Researchers have introduced a new computational framework called the Interpolating Neural Network (INN), designed to merge classical interpolation theory with modern machine learning. This new architecture aims to address significant challenges in scientific computing, including low accuracy on sparse datasets, poor scalability, and high computational costs associated with complex system simulations.
The INN model has demonstrated substantial performance gains, particularly in simulating metal additive manufacturing. In one test, it modeled a heat transfer process 5 to 8 orders of magnitude faster than competing machine learning models, completing a high-resolution simulation in under 15 minutes on a single GPU.
Key Takeaways
- A new framework, the Interpolating Neural Network (INN), combines principles from physics-based simulations and data-driven machine learning.
- INN aims to improve accuracy, reduce computational cost, and enhance scalability for complex scientific problems.
- The model uses interpolation theory, similar to the Finite Element Method (FEM), and tensor decomposition to dramatically reduce the number of trainable parameters.
- In a key application, INN simulated a metal additive manufacturing process orders of magnitude faster than traditional methods and other AI models.
A New Approach to Scientific Computing
Modern computational science is increasingly shifting from traditional, explicitly programmed solvers to data-centric artificial intelligence models. This transition, sometimes called the move from "Software 1.0 to Software 2.0," has powered advancements like large language models.
However, applying this approach to scientific and engineering problems presents unique hurdles. Machine learning models often require vast amounts of data, struggle with scalability for high-dimensional problems, and can be computationally expensive, raising concerns about energy consumption and sustainability.
The Interpolating Neural Network was developed to address these issues. It creates a hybrid system that leverages the strengths of both numerical analysis methods, like the Finite Element Method (FEM), and deep neural networks (DNNs).
How Interpolating Neural Networks Work
At their core, both FEM and DNNs are function approximators. FEM uses interpolation functions based on discrete points (nodes) to approximate a solution, while DNNs create a complex function using layers of neurons with trainable weights and biases.
INN formalizes this connection. It operates in three main steps:
- Domain Discretization: The input domain of a problem is divided into segments defined by interpolation nodes. This is conceptually similar to creating a mesh in traditional engineering simulations.
- Graph Construction: A computational graph is formed using these nodes. The model then uses a process similar to message passing in graph neural networks to construct interpolation functions for each node.
- Optimization: The values and coordinates of these nodes are optimized to minimize a given loss function, effectively training the model to solve the problem.
Bridging Two Worlds
The INN framework reinterprets components of numerical analysis, such as shape functions in FEM, as trainable parameters in a machine learning model. This allows it to act as a data-driven model, a solver for partial differential equations (PDEs), or a tool for solving inverse problems (calibration) within the same architecture.
Achieving Scalability with Tensor Decomposition
One of the most significant challenges in high-dimensional problems is the "curse of dimensionality," where computational costs grow exponentially with the number of input variables. INNs address this by employing tensor decomposition when the input domain is structured on a regular grid.
By using techniques like Tucker decomposition or CANDECOMP/PARAFAC (CP) decomposition, INN breaks down a high-dimensional problem into a series of one-dimensional ones. This dramatically reduces the number of trainable parameters needed.
From Exponential to Linear Growth
Without decomposition, the number of parameters in a full interpolation model scales exponentially with the input dimension. With tensor decomposition, the INN's parameter count scales linearly. This makes it feasible to tackle problems with many variables that would be intractable for other methods.
This reduction in model complexity not only accelerates computation but also lowers memory requirements, making INNs a more sustainable alternative to massive, resource-intensive AI models.
Performance Benchmarks and Comparisons
To validate its effectiveness, the INN framework was tested against established models like Multi-Layer Perceptrons (MLPs) and Physics-Informed Neural Networks (PINNs).
Training Speed and Efficiency
In a regression task with 10 inputs and 5 outputs, INNs were compared to MLPs. The results showed that for a similar number of trainable parameters, INNs converged significantly faster. For example, an INN with 2,500 parameters reached the target accuracy in 19 epochs, whereas an MLP with 2,285 parameters required 253 epochs.
This efficiency means INNs can be trained more quickly and with fewer computational resources, a key advantage for practical applications.
Solving Physics Equations
When used as a PDE solver, the INN demonstrated predictable and stable convergence, a critical requirement for engineering-grade tools. In a benchmark solving a 1D Poisson's equation, the INN's error decreased at a predictable rate as the model's complexity increased, consistent with established numerical analysis theory.
"A PDE solver needs to have a stable and predictable convergence rate because it guides an engineer in choosing the mesh resolution and other hyperparameters for achieving the desired level of accuracy. INN solvers meet this requirement."
In contrast, PINNs built with standard MLPs did not show a clear convergence pattern in the same test. This reliability makes INNs a more trustworthy tool for applications where accuracy is paramount.
Application in Additive Manufacturing
The power of the INN framework was demonstrated in the complex field of metal additive manufacturing, specifically Laser Powder Bed Fusion (L-PBF). This process involves using a laser to melt and fuse metal powder layer by layer, a process governed by a high-dimensional heat transfer equation.
The simulation involved an 8-dimensional problem, including three spatial dimensions, one time dimension, and four variable process parameters (like laser power and thermal conductivity). Accurately modeling this is computationally prohibitive for most methods.
Breakthrough Simulation Speeds
The INN solver was used to create a surrogate model for a 10 mm single-track L-PBF scan with a resolution of 8 micrometers. The results were striking:
- Time: The model was projected to be up to 100 million times faster than a finite-difference method (FDM) solver for offline data generation.
- Storage: The INN model required only gigabytes of storage, compared to the petabytes estimated for storing data from traditional simulation runs.
- Accuracy: The resulting surrogate model achieved an R² accuracy of 0.9969 when compared to high-fidelity solutions.
These capabilities open the door for real-time online control and parameter calibration in advanced manufacturing. For instance, an INN-based model could predict melt pool temperatures and adjust laser power on the fly to ensure part quality, a task that is impossible with slower simulation tools.
Future Directions and Summary
The introduction of Interpolating Neural Networks presents a promising path forward for scientific machine learning. By grounding AI in the robust principles of interpolation theory and leveraging the efficiency of tensor decomposition, INNs offer a scalable, accurate, and computationally efficient tool.
Future research will explore the model's performance on problems with high uncertainty, investigate its interpretability, and extend its application to multiscale analysis. The framework's ability to function as a trainer, solver, and calibrator makes it a versatile platform for tackling some of the most challenging problems in science and engineering.
The computer code for INN has been made publicly available on GitHub, allowing other researchers to build upon this work.