Results of the summarizing process for the arXiv paper: 2205.09332v6

The paper presents a new technique for accelerating the training of physics-informed neural networks (PINNs) called discretely-trained PINNs (DT-PINNs). DT-PINNs are trained by replacing exact spatial derivatives with high-order accurate numerical discretizations computed using meshless radial basis function-finite differences (RBF-FD) and applied via sparse-matrix vector multiplication. This approach allows for DT-PINNs to be trained even on point cloud samples placed on irregular domain geometries. The authors show that using fp64 on the GPU leads to significantly faster training times than fp32 vanilla-PINNs with comparable accuracy. The efficiency and accuracy of DT-PINNs are demonstrated through a series of experiments. First, the authors explore the effect of network depth and show that RBF-FD approximations of third order accuracy and above are more efficient while being sufficiently accurate. Next, they compare DT-PINNs to vanilla-PINNs on both linear and nonlinear Poisson equations and show that DT-PINNs achieve similar losses with 2–4x faster training times on a consumer GPU. Finally, they demonstrate that similar results can be obtained for the PINN solution to the heat equation by discretizing spatial derivatives using RBF-FD and using automatic differentiation for temporal derivatives. Overall, their results show that fp64 DT-PINNs offer a superior cost–accuracy profile compared to fp32 vanilla–PINNs. The authors conclude by highlighting how this work could have implications for accelerating scientific computing tasks that rely on PINNs such as fluid dynamics simulations and inverse problems in medical imaging.

• The paper presents a new technique called discretely-trained PINNs (DT-PINNs) for accelerating the training of physics-informed neural networks (PINNs).
• DT-PINNs are trained using high-order accurate numerical discretizations computed using meshless radial basis function-finite differences (RBF-FD) and applied via sparse-matrix vector multiplication.
• This approach allows for DT-PINNs to be trained even on point cloud samples placed on irregular domain geometries.
• Using fp64 on the GPU leads to significantly faster training times than fp32 vanilla-PINNs with comparable accuracy.
• RBF-FD approximations of third order accuracy and above are more efficient while being sufficiently accurate for network depth.
• DT-PINNs achieve similar losses with 2–4x faster training times compared to vanilla-PINNs on both linear and nonlinear Poisson equations on a consumer GPU.
• Similar results can be obtained for the PINN solution to the heat equation by discretizing spatial derivatives using RBF-FD and using automatic differentiation for temporal derivatives.
• Fp64 DT-PINNs offer a superior cost–accuracy profile compared to fp32 vanilla–PINNs, which could have implications for accelerating scientific computing tasks that rely on PINNs such as fluid dynamics simulations and inverse problems in medical imaging.

Accelerating the Training of Physics-Informed Neural Networks with Discretely-Trained PINNs

Recent advances in machine learning have enabled researchers to develop powerful models for solving complex scientific problems. One such model is the physics-informed neural network (PINN), which combines deep learning and numerical methods to solve partial differential equations (PDEs). However, training these networks can be computationally expensive due to their large number of parameters. In this paper, we present a new technique for accelerating the training of PINNs called discretely-trained PINNs (DT-PINNs).

Background

A PINN is a type of artificial neural network that uses physical laws as constraints during training. This allows them to learn accurate solutions to PDEs without requiring large amounts of labeled data. The main challenge with using PINNs is that they require exact derivatives when computing gradients during backpropagation. This can lead to long training times due to the need for repeated numerical integration or differentiation operations.

Discretely Trained PINNS

To address this issue, we propose DT-PINNs, which are trained by replacing exact spatial derivatives with high-order accurate numerical discretizations computed using meshless radial basis function–finite differences (RBF–FD) and applied via sparse matrix vector multiplication. This approach allows us to train DT–PINNs even on point cloud samples placed on irregular domain geometries. We show that using fp64 on the GPU leads to significantly faster training times than fp32 vanilla–PINNs with comparable accuracy.

Experimental Results

We demonstrate the efficiency and accuracy of DT–PINNs through a series of experiments. First, we explore the effect of network depth and show that RBF–FD approximations of third order accuracy and above are more efficient while being sufficiently accurate. Next, we compare DT–PINNs with vanilla–PINNS on both linear and nonlinear Poisson equations and show that DT–PINNs achieve similar losses with 2x – 4x faster training times on a consumer GPU compared to vanilla – PINNS . Finally, we demonstrate similar results can be obtained for the solution to heat equation by discretizing spatial derivatives using RBF – FD combined with automatic differentiation for temporal derivatives .

Conclusion

Overall our results show that fp64 DT - PINNS offer superior cost - accuracy profile compared to fp32 vanilla - PINNS . Our work could have implications for accelerating scientific computing tasks such as fluid dynamics simulations or inverse problems in medical imaging which rely heavily on physics informed neural networks .