Résumé : In self-gravitating $N$-body systems, small perturbations introduced at the start, or infinitesimal errors that are produced by the numerical integrator or are due to limited precision in the computer, grow exponentially with time. For Newton's gravity, we confirm earlier results that for relatively homogeneous systems, this rate of growth per crossing time increases with $N$ up to $N \sim 30$, but that for larger systems, the growth rate has a weaker scaling with $N$. For concentrated systems, however, the rate of exponential growth continues to scale with $N$. In relativistic self-gravitating systems, the rate of growth is almost independent of $N$. This effect, however, is only noticeable when the system's mean velocity approaches the speed of light to within three orders of magnitude. The chaotic behavior of systems with more than a dozen bodies for the usually adopted approximation of only solving the pairwise interactions in the Einstein-Infeld-Hoffmann equation of motion is qualitatively different than when the interaction terms (or cross terms) are taken into account. This result provides a strong motivation for follow-up studies on the microscopic effect of general relativity on orbital chaos, and on the influence of higher-order cross-terms in the Taylor-series expansion of the Einstein-Infeld-Hoffmann equations of motion.