Résumé : Context. Asteroseismic studies showed that cores of post main-sequence stars rotate slower than theoretically predicted by stellar models with purely hydrodynamical transport processes. Recent studies on main sequence stars, particularly Gamma Doradus ($\gamma$ Dor) stars, revealed their internal rotation rate for hundreds of stars, offering a counterpart on the main sequence for studies of angular momentum transport. Aims. We investigate whether such a disagreement between observed and predicted internal rotation rates is present in main sequence stars by studying angular momentum transport in $\gamma$ Dor stars. Furthermore, we test whether models of rotating stars with internal magnetic fields can reproduce their rotational properties. Methods. We compute rotating models with the Geneva stellar evolution code taking into account meridional circulation and the shear instability. We also compute models with internal magnetic fields using a general formalism for transport by the Tayler-Spruit dynamo. We then compare these models to observational constraints for $\gamma$ Dor stars that we compiled from the literature, combining so the core rotation rates, projected rotational velocities from spectroscopy, and constraints on their fundamental parameters. Results. We show that combining the different observational constraints available for $\gamma$ Dor stars enable to clearly distinguish the different scenarios for internal angular momentum transport. Stellar models with purely hydrodynamical processes are in disagreement with the data whereas models with internal magnetic fields can reproduce both core and surface constraints simultaneously. Conclusions. Similarly to results obtained for subgiant and red giant stars, angular momentum transport in radiative regions of $\gamma$ Dor stars is highly efficient, in good agreement with predictions of models with internal magnetic fields.