Résumé : We present the results of a systematic search for candidate quiescent galaxies in the distant Universe in eleven $JWST$ fields with publicly available observations collected during the first three months of operations and covering an effective sky area of $\sim145$ arcmin$^2$. We homogeneously reduce the new $JWST$ data and combine them with existing observations from the $Hubble\,Space\,Telescope$. We select a robust sample of $\sim80$ candidate quiescent and quenching galaxies at $3 < z < 5$ using two methods: (1) based on their rest-frame $UVJ$ colors, and (2) a novel quantitative approach based on Gaussian Mixture Modeling of the $NUV-U$, $U-V$, and $V-J$ rest-frame color space, which is more sensitive to recently quenched objects. We measure comoving number densities of massive ($M_\star\geq 10^{10.6} M_\odot$) quiescent galaxies consistent with previous estimates relying on ground-based observations, after homogenizing the results in the literature with our mass and redshift intervals. However, we find significant field-to-field variations of the number densities up to a factor of $2-3$, highlighting the effect of cosmic variance and suggesting the presence of overdensities of red quiescent galaxies at $z>3$, as it could be expected for highly clustered massive systems. Importantly, $JWST$ enables the robust identification of quenching/quiescent galaxy candidates at lower masses and higher redshifts than before, challenging standard formation scenarios. All data products, including the literature compilation, are made publicly available.