Résumé : We study the stellar population properties of 182 spectroscopically-confirmed (MUSE/VLT) Lyman-$\alpha$ emitters (LAEs) and 450 photometrically-selected Lyman-Break galaxies (LBGs) at z = 2.8 - 6.7 in the Hubble eXtreme Deep Field (XDF). Leveraging the combined power of HST and JWST NIRCam and MIRI observations, we analyse their rest-frame UV-through-near-IR spectral energy distributions (SEDs) with MIRI playing a crucial role in robustly assessing the LAE's stellar mass and ages. Our LAEs are low-mass objects (log$_{10}$(M$_\star$[M$_\odot$]) ~ 7.5), with little or no dust extinction (E(B - V) ~ 0.1) and a blue UV continuum slope ($\beta$ ~ -2.2). While 75% of our LAEs are young (< 100 Myr), the remaining 25% have significantly older stellar populations (> 100 Myr). These old LAEs are statistically more massive, less extinct and have lower specific star formation rate (sSFR) compared to young LAEs. Besides, they populate the M$_\star$ - SFR plane along the main-sequence (MS) of star-forming galaxies, while young LAEs populate the starburst region. The comparison between the LAEs properties to those of a stellar-mass matched sample of LBGs shows no statistical difference between these objects, except for the LBGs redder UV continuum slope and marginally larger E(B - V) values. Interestingly, 48% of the LBGs have ages < 10 Myr and are classified as starbursts, but lack detectable Ly$\alpha$ emission. This is likely due to HI resonant scattering and/or selective dust extinction. Overall, we find that JWST observations are crucial in determining the properties of LAEs and shedding light on the properties and similarities between LAEs and LBGs.