Résumé : We develop a theory for the phonon magnetic moment in doped Dirac materials, treating phonons as emergent gauge and gravitational fields coupled to Dirac fermions in curved space. By classifying electron-phonon coupling into angular momentum channels of Fermi surface deformation, we show that the phonon moment arises from two mechanisms: proportional to the electron Hall conductivity through the emergent gauge field coupling, and to the Hall viscosity through the frame field coupling. Applying our theory to Cd$_3$As$_2$ with first-principles calculations, we find quantitative agreement with experiment. Our results reveal a general mechanism for dynamically generating large phonon magnetism in metals and suggest a new route for probing Hall viscosity via phonon dynamics.