Résumé : A strong absorption profile was reported by the EDGES Collaboration, which indicates the hydrogen gas being colder than expected. It could be signatures of non-gravitational interactions between normal matter and dark matter (DM), and a potential explanation is that a small fraction of millicharged DM scatters with normal matter, with the DM mass in tens of MeV. To obtain the small fraction of millicharged DM and meanwhile being tolerant with by the constraints, the dark photon portal scalar and vector millicharged DM are explored in this paper. We consider that the mass of dark photon is slightly above twice of the millicharged DM mass, and thus the millicharged DM predominantly annihilates in p-wave during the freeze-out period, with the annihilation being enhanced near the resonance. The dark photon mainly decays into millicharged DM, and couplings of dark photon with SM particles could be allowed by the lepton collision experiments. The corresponding parameter spaces are derived. Future lepton collision experiments can be employed to search for millicharged DM via the production of the invisible dark photon.