Résumé : Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, Machine learning interatomic potentials (MLIP) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIP have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIP is lacking. Here, we benchmark popular MLIP using the anharmonic vibrational Hamiltonian of ThO$_2$ in the fluorite crystal structure. This anharmonic Hamiltonian was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods, and then used to generate molecular dynamics trajectories. This data set was used to train three classes of MLIP: Gaussian Approximation Potentials, Artificial Neural Networks (ANN), and Graph Neural Networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT molecular dynamics dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIP have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.