Week 45 10 papers Coverage: 2026-10-25 → 2026-10-30

Week 45 · PINN Trainability Catch-up, and Neural Operators Going Operational

Four of the ten papers this week are doing PINN fundamentals: Lie-symmetry-generated sample points with exact solutions, a hard constraint projection that pins nonlinear Burgers to its discrete solution hyperplane, holomorphic analytic structure that sidesteps the nonhomogeneous term, and a study tracing buoyancy-driven PINN failure to a non-dimensionalization-induced gradient imbalance. The rest push neural operators into operational use - as convergence-guaranteed preconditioners for nonlinear optimization, as forecasting and spectral operators, and a reminder that the published The Well baselines were never architecture ceilings.

Highlights

  • A root cause of PINN collapse is pinned down: in buoyancy-driven flow the two non-dimensionalizations make the loss-gradient gap scale as (Ra·Pr)², and neither adaptive weighting nor sparse supervision helps
  • Hard constraints cross into the nonlinear regime: discretizing Burgers and decomposing it into a constraint matrix, an untrainable projection pins the network output to the discrete solution hyperplane - about an order of magnitude lower data error than a matched PINN
  • Holomorphic networks get practical: split the solution into particular plus homogeneous, the particular part via spectral method, the homogeneous part via a boundary-trained PIHNN - shorter training and lower error on nonhomogeneous Poisson/Stokes/biharmonic/elasticity/Helmholtz
  • Neural operators do more than fit: SNAP uses a pretrained operator as the search direction for nonlinear optimization, accepting only a posteriori descent-satisfying corrections - proven global convergence and preserved rate
  • Baselines misread: on The Well, a single light TFNO tuning pass using only one input timestep beats the published FNO/TFNO baselines on 15 of 17 tasks

In this issue

1

Lie Symmetry-Guided Adaptive Sampling for Physics-Informed Neural Networks

Authors Runze Zhao
Why read it

For PDEs with Lie symmetry groups, symmetry transformations give exact functional relations between the solution and its variables, so the authors build basis points that carry exact solutions and span the domain analytically. An adaptive strategy then weights sampling by absolute error and invariant-surface conditions, plus a dual-index loss. Experiments show accuracy consistently better than existing sampling methods.

arXiv · 2610.34345 ↗ physics-informedpinnadaptive-samplinglie-symmetry
2

Hard Constraint Projection in a Physics Informed Neural Network (Burgers)

Authors Miranda J S Horne, Peter K Jimack, Amirul Khan et al. 6 pages, 3 figures, In proceedings of 1st international Symposium on AI and Fluid Mechanics 2025 (Paper No. S11P2)
Why read it

Extends the 2021 hard constraint projection (HCP) method of Chen et al. from a linear PDE to the nonlinear 1D Burgers equation. The PDE is discretized locally, decomposed into a constraint matrix and a prediction matrix, and a final untrainable layer projects the velocity prediction onto the hyperplane of exact solutions to the discretized Burgers equation. Compared with a matched PINN, it is more robust to collocation-point selection and reaches a data error about an order of magnitude lower.

arXiv · 2610.42106 ↗ physics informedpinnhard-constraintburgers
3

A Hybrid Physics-informed holomorphic neural networks Method

Authors Zhijie Su, Qiaolin He, Zhiping Mao et al. 34 pages, 14 figures
Why read it

Physics-informed holomorphic neural networks satisfy homogeneous PDEs by construction via complex-analytic representations, but stall on nonhomogeneous problems. This splits the solution into a particular component (computed by a spectral method on an extended domain) and a homogeneous component (approximated by a PIHNN trained only on the boundary with modified data). On nonhomogeneous Poisson, Stokes, biharmonic, linear elasticity and Helmholtz it gives lower error and shorter training than standard PINNs.

arXiv · 2610.38269 ↗ physics-informedpinnholomorphicnonhomogeneous
4

Safeguarded Neurally-Accelerated Preconditioning (SNAP) framework for Nonlinear Optimization

Authors Youngkyu Lee, George Em Karniadakis, Alena Kopaničáková 26 pages
Why read it

Hybridizes nonlinear optimization with pretrained neural operators: each cycle alternates a classical optimization step with a search direction from the operator, and an a posteriori safeguard accepts only corrections that satisfy prescribed descent and acceptance conditions. Global convergence is proved, the underlying method rate is preserved, and it converges faster when the operator is accurate. FNO and Transolver are tested on minimal surface, Allen-Cahn and nonlinear Helmholtz problems, with a large drop in iteration counts.

arXiv · 2610.38790 ↗ neural operatoroptimizationpreconditionerfourier neural
5

Learning Kolmogorov Backward Operators for Spectral Analysis and Forecasting: A Data-Driven Variational Galerkin Framework

Authors Yuanchao Xu, Claire Valva, Ben Gao et al.
Why read it

For a time-homogeneous Markov process, even when the dynamics are stochastic, the fixed-lag solution operator evolves observables linearly and deterministically. A two-stage method first approximates the operator directly from data with no governing equation (e.g. a Fourier neural operator), then compresses via a Galerkin projection onto a finite-dimensional observable space fixed in advance or chosen variationally. The error analysis separates learning, sampling and invariance/reconstruction errors, and gives bounds on subspace distance and forecast error that stay uniform in the horizon near an isolated spectral cluster.

arXiv · 2610.39580 ↗ neural operatorkolmogorovspectralforecasting
6

Inherent Failure Modes of Physics-Informed Neural Networks for Buoyancy-Driven Flow: Role of Non-Dimensionalization in Network Training

Authors Krishan Chand, Ahmad Peyvan, Haoxiang Luo 32 pages, 12 figures
Why read it

Using a differentially heated square cavity at Pr=0.71 and Ra=10³–10⁵, this quantifies PINN trainability for buoyancy-driven flow. Free-fall scaling (unit buoyancy coefficient) keeps the mean Nusselt error below 1.1% for Ra≤10⁴, while thermal-diffusive scaling (buoyancy coefficient Ra·Pr) gives 157–650% error and diverges at Ra=10⁵. The culprit is a coefficient-induced gradient imbalance whose dominant term scales as (Ra·Pr)², confirmed by direct gradient measurement; neither adaptive loss weighting nor sparse-data supervision resolves it. For unsteady convection the free-fall PINN forecasts long-time heat transfer but misses the transient Nusselt overshoot.

arXiv · 2610.38889 ↗ physics-informedpinnbuoyancynondimensionalization
7

Predicting Sparse Observables in Parameterized Unsteady Flows Using Coarse-Mesh Graph Neural Networks

Authors Brener Ramos, William R. Wolf, Tarcísio C. D. Oliveira et al.
Why read it

Deploys graph neural networks directly on coarse meshes to predict only sparse probes and integrated quantities, bypassing full-state reconstruction. On parameterized 2D/3D confined cylinder flows and an unevenly heated cavity, coarse-mesh GNNs match or beat finer-mesh surrogates at lower training time and memory; a frequency-spectrum-guided adaptive sampling cuts the maximum error by about an order of magnitude. Noise-augmented multi-step unrolling also enables long-horizon autoregressive forecasting.

arXiv · 2610.37986 ↗ neural operatorgraph neural networkreduced-ordercoarse-mesh
8

Application of PDE emulators for a multi-material, multi-physics simulation problem

Authors Diane Oyen, Morgan Jones, Adithan Kathirgamaraju et al. Accepted at ICMLA 2026, Rochester, MI, USA
Why read it

Evaluates computer-vision-based architectures, including transformers, as surrogate models on a challenging multi-material, multi-physics task, and exposes the real friction of adapting pretrained models to such data: algorithms for sparse and imbalanced data, adaptable spatial encodings, physical-consistency metrics, and cheaper training-data generation all matter far more than on standard PDE benchmarks. A sober checklist for anyone hoping to repurpose off-the-shelf large models for industrial simulation.

arXiv · 2610.37972 ↗ surrogate modelpdemulti-physicsmulti-material
9

ADACT: PDE-Constrained Thermal Hardware/Software Co-Design Adjoint-Driven Optimal Control via Finite Difference Discretization

Authors Don Li
Why read it

A physics-informed PDE-constrained dynamic thermal management for AI/HPC multicore chips. The temperature field is a 2D nonhomogeneous parabolic reaction-diffusion equation with time-varying power sources, discretized by a Crank-Nicolson five-point Laplacian (second-order, unconditionally stable), with exact gradients from a continuous backward-in-time adjoint state and L-BFGS. The method of manufactured solutions and directional-derivative Taylor tests verify both solvers. ADACT then throttles software power and cooling to uphold physical thermal thresholds while keeping core throughput.

arXiv · 2610.35569 ↗ physics-informedpdeadjointthermal-management
10

Performance of Neural Operator Baselines on 'The Well' Dataset

Authors Yash Pincha, Andy Dimnaku, Andrew Chiang et al.
Why read it

The Well is a 15TB benchmark across 16 physical simulation domains, but its released baselines were illustrative results obtained at small cost, with zero to minimal tuning and a 12-hour per-run wall-clock limit, later treated as fully tuned architecture benchmarks. Under the same 12-hour limit, using only a single input timestep instead of the four used by the published baselines, a short non-exhaustive TFNO tuning pass still beats the published FNO and TFNO baselines on 15 of 17 tasks, sometimes by orders of magnitude. The lesson: the released neural-operator baselines are not architecture ceilings.

arXiv · 2610.33967 ↗ neural operatorbenchmarkthe-wellfourier neural

Overview

Four of the ten papers this week are doing PINN fundamentals: Lie-symmetry-generated sample points with exact solutions, a hard constraint projection that pins nonlinear Burgers to its discrete solution hyperplane, holomorphic analytic structure that sidesteps the nonhomogeneous term, and a study tracing buoyancy-driven PINN failure to a non-dimensionalization-induced gradient imbalance. The rest push neural operators into operational use - as convergence-guaranteed preconditioners for nonlinear optimization, as forecasting and spectral operators, and a reminder that the published The Well baselines were never architecture ceilings.

1. Lie Symmetry-Guided Adaptive Sampling for Physics-Informed Neural Networks

For PDEs with Lie symmetry groups, symmetry transformations give exact functional relations between the solution and its variables, so the authors build basis points that carry exact solutions and span the domain analytically. An adaptive strategy then weights sampling by absolute error and invariant-surface conditions, plus a dual-index loss. Experiments show accuracy consistently better than existing sampling methods.

2. Hard Constraint Projection in a Physics Informed Neural Network (Burgers)

Extends the 2021 hard constraint projection (HCP) method of Chen et al. from a linear PDE to the nonlinear 1D Burgers equation. The PDE is discretized locally, decomposed into a constraint matrix and a prediction matrix, and a final untrainable layer projects the velocity prediction onto the hyperplane of exact solutions to the discretized Burgers equation. Compared with a matched PINN, it is more robust to collocation-point selection and reaches a data error about an order of magnitude lower.

3. A Hybrid Physics-informed holomorphic neural networks Method

Physics-informed holomorphic neural networks satisfy homogeneous PDEs by construction via complex-analytic representations, but stall on nonhomogeneous problems. This splits the solution into a particular component (computed by a spectral method on an extended domain) and a homogeneous component (approximated by a PIHNN trained only on the boundary with modified data). On nonhomogeneous Poisson, Stokes, biharmonic, linear elasticity and Helmholtz it gives lower error and shorter training than standard PINNs.

4. Safeguarded Neurally-Accelerated Preconditioning (SNAP) framework for Nonlinear Optimization

Hybridizes nonlinear optimization with pretrained neural operators: each cycle alternates a classical optimization step with a search direction from the operator, and an a posteriori safeguard accepts only corrections that satisfy prescribed descent and acceptance conditions. Global convergence is proved, the underlying method rate is preserved, and it converges faster when the operator is accurate. FNO and Transolver are tested on minimal surface, Allen-Cahn and nonlinear Helmholtz problems, with a large drop in iteration counts.

5. Learning Kolmogorov Backward Operators for Spectral Analysis and Forecasting: A Data-Driven Variational Galerkin Framework

For a time-homogeneous Markov process, even when the dynamics are stochastic, the fixed-lag solution operator evolves observables linearly and deterministically. A two-stage method first approximates the operator directly from data with no governing equation (e.g. a Fourier neural operator), then compresses via a Galerkin projection onto a finite-dimensional observable space fixed in advance or chosen variationally. The error analysis separates learning, sampling and invariance/reconstruction errors, and gives bounds on subspace distance and forecast error that stay uniform in the horizon near an isolated spectral cluster.

6. Inherent Failure Modes of Physics-Informed Neural Networks for Buoyancy-Driven Flow: Role of Non-Dimensionalization in Network Training

Using a differentially heated square cavity at Pr=0.71 and Ra=10³–10⁵, this quantifies PINN trainability for buoyancy-driven flow. Free-fall scaling (unit buoyancy coefficient) keeps the mean Nusselt error below 1.1% for Ra≤10⁴, while thermal-diffusive scaling (buoyancy coefficient Ra·Pr) gives 157–650% error and diverges at Ra=10⁵. The culprit is a coefficient-induced gradient imbalance whose dominant term scales as (Ra·Pr)², confirmed by direct gradient measurement; neither adaptive loss weighting nor sparse-data supervision resolves it. For unsteady convection the free-fall PINN forecasts long-time heat transfer but misses the transient Nusselt overshoot.

7. Predicting Sparse Observables in Parameterized Unsteady Flows Using Coarse-Mesh Graph Neural Networks

Deploys graph neural networks directly on coarse meshes to predict only sparse probes and integrated quantities, bypassing full-state reconstruction. On parameterized 2D/3D confined cylinder flows and an unevenly heated cavity, coarse-mesh GNNs match or beat finer-mesh surrogates at lower training time and memory; a frequency-spectrum-guided adaptive sampling cuts the maximum error by about an order of magnitude. Noise-augmented multi-step unrolling also enables long-horizon autoregressive forecasting.

8. Application of PDE emulators for a multi-material, multi-physics simulation problem

Evaluates computer-vision-based architectures, including transformers, as surrogate models on a challenging multi-material, multi-physics task, and exposes the real friction of adapting pretrained models to such data: algorithms for sparse and imbalanced data, adaptable spatial encodings, physical-consistency metrics, and cheaper training-data generation all matter far more than on standard PDE benchmarks. A sober checklist for anyone hoping to repurpose off-the-shelf large models for industrial simulation.

9. ADACT: PDE-Constrained Thermal Hardware/Software Co-Design Adjoint-Driven Optimal Control via Finite Difference Discretization

A physics-informed PDE-constrained dynamic thermal management for AI/HPC multicore chips. The temperature field is a 2D nonhomogeneous parabolic reaction-diffusion equation with time-varying power sources, discretized by a Crank-Nicolson five-point Laplacian (second-order, unconditionally stable), with exact gradients from a continuous backward-in-time adjoint state and L-BFGS. The method of manufactured solutions and directional-derivative Taylor tests verify both solvers. ADACT then throttles software power and cooling to uphold physical thermal thresholds while keeping core throughput.

10. Performance of Neural Operator Baselines on 'The Well' Dataset

The Well is a 15TB benchmark across 16 physical simulation domains, but its released baselines were illustrative results obtained at small cost, with zero to minimal tuning and a 12-hour per-run wall-clock limit, later treated as fully tuned architecture benchmarks. Under the same 12-hour limit, using only a single input timestep instead of the four used by the published baselines, a short non-exhaustive TFNO tuning pass still beats the published FNO and TFNO baselines on 15 of 17 tasks, sometimes by orders of magnitude. The lesson: the released neural-operator baselines are not architecture ceilings.