Tokamak Plasma Magnetic Confinement Control
New1. Overview
Magnetic confinement research has produced many simulation-to-real demonstrations that stop at offline metric parity. This work is different because the loop is closed on hardware. A controller is trained purely against a physics simulator, compiled into a real-time executable, loaded into the control framework of an actual tokamak, and then drives nineteen magnetic coils directly at 10 kHz while holding a requested plasma configuration across repeated discharges.
The tokamak is a well-chosen testbed for this claim. The plasma equilibrium is described by the Grad-Shafranov equation, in which the balance between Lorentz force and the pressure gradient inside the plasma determines the configuration. That equation has mature numerical solutions, which makes a physically credible fast simulator possible. At the same time, shape control couples many objectives at once: boundary shape, position, elongation, triangularity, divertor X-point location, plasma current, and vertical stability growth rate. Each objective alone is simple, but their combination strains conventional single-variable feedback design.
The industrial relevance lies in the cost figures. The system trains with 5,000 parallel actors over one to three days, a learner on a single tensor processing unit, and a policy network of 266,280 parameters compiled for real-time execution. Those numbers move the discussion from demonstration to engineering feasibility, and they are the reason this case is worth studying in detail rather than only citing.
2. How It Works
The pipeline has four layers, and the paper states a concrete design choice for each one.
The first layer is the physics simulator. The FGE (Free-boundary Grad-Shafranov Equilibrium) package solves the free-boundary Grad-Shafranov equation while the total plasma current evolves under a lumped circuit equation. Free boundary means the plasma boundary and the vacuum magnetic field boundary are solved together, so geometric quantities such as X-points and divertor leg positions fall directly out of the solution. The simulation time step is 50 kHz while the control rate is 10 kHz, so one policy evaluation corresponds to five simulator substeps with the coil voltage held constant across them. This decoupling lets the simulator step be chosen on numerical grounds alone. A later study loosened the residual tolerance of the FGE Newton-Krylov solver from 1e-8 to 1e-4 and obtained a two- to threefold training speedup with no significant change in simulated control outcome.
The second layer is the observation and action space. The policy consumes raw magnetic measurements and does not rely on real-time equilibrium reconstruction. The real-time sensors used are 34 wire loops measuring magnetic flux, 38 probes measuring the local magnetic field, and 19 measurements of current in the active control coils, augmented by an explicit measurement of the current difference between the ohmic coils. The action is the voltage command for the nineteen coils.
The third layer is the algorithm. Maximum a posteriori policy optimization (MPO) was chosen as an actor-critic method that supports collection across distributed parallel data streams and is data-efficient. The work also exploits the inherent asymmetry of the actor-critic design: the actor is constrained by the control rate while the critic is not, so the critic uses a larger recurrent network with 718,337 parameters and a wider receptive field, while the actor stays lightweight at 266,280 parameters. A symmetric control design with a multilayer perceptron critic of 266,497 parameters clearly underperformed the asymmetric arrangement.
The fourth layer covers training and deployment. Training uses 5,000 actors in parallel, each running an independent simulator instance, with one learner on a tensor processing unit. The policy is then bundled with the experiment control targets into an executable compiled for 10 kHz real-time control, minimizing dependencies and eliminating unnecessary computation. At deployment, standard plasma-formation procedures run first, with a conventional controller maintaining plasma position and total current, and handover to the learned controller occurs at a prespecified time.
3. Scope and Assumptions
The boundaries are stated clearly enough to check against your own situation.
A real-time control system on an actual device is required. The 10 kHz rate, simultaneous action on nineteen coils, and millisecond-scale handover all depend on an existing real-time control framework. A team without that infrastructure cannot reproduce the demonstrated path.
A trustworthy fast simulator is required. FGE solves magnetic equilibrium only. It contains no turbulent transport, no tearing instability growth rate prediction, and no disruption forecasting. The paper states plainly that the controller does not cause outcomes corresponding to disruption criteria in simulation because disruption physics was never modelled in the training simulator. Disruption avoidance capability was therefore absent from training.
Uncertainty must be covered by domain randomization. The randomized ranges include normalized pressure, safety factor, growth rate, plasma current, and heating power, and these are listed in the paper tables and compared against measured experimental values. Behaviour outside those trained intervals carries no guarantee.
Objectives are given by a human operator. The framework optimizes a set of objectives supplied by the operator, and it does not cover automatic generation of those objectives or the engineering trade-offs behind them.
A hardware protection layer cannot be omitted. The paper explicitly recommends keeping a simpler fallback controller or interlock system in place during experiments, because the simulator does not guarantee avoidance of all disruption forms.
Objectives also differ in difficulty. The droplet experiment shows that the architecture can adapt to previously unseen configurations, while the paper states that bridging the sim-to-real gap for maintaining highly accurate perfect snowflakes still requires work.
4. Getting Started
Decide which control problem you are solving. Shape control, vertical stability control, and disruption avoidance are distinct problem domains. The strength of this work lies in shape constraints, while disruption avoidance belongs to a different research line built on a multimodal dynamic model. The two do not share modeling objects or cost functions.
Build or obtain a simulator. If the target device differs substantially from TCV, a free-boundary equilibrium solver must be written or an existing open-source tool integrated. The pattern of placing a fast physical kernel underneath reinforcement learning appears across domains, and the acceleration and surrogate coupling practices in JAX-based fluid frameworks and in PhysicsNeMo are directly transferable in spirit.
Confirm which observation channels are available in real time. Because the policy consumes raw magnetic measurements rather than a reconstructed state, sensor selection becomes a design decision at the machine level. The number and placement of wire loops and probes must be designed together with the coil layout, and that is an infrastructure change that no algorithm can work around.
Set up the training infrastructure. A distributed actor pool with a single learner is the standard configuration at this scale. Existing parallel simulation frameworks can be reused directly, and scheduling experience from multiphysics coupling tools applies here.
Run reward shaping and observation augmentation experiments. The numbers reported by the practical follow-up are specific: reward shaping reduced X-point tracking RMSE by 59.7 percent against a previous TCV experiment, with the simulation value falling from 3.46 cm to 0.27 cm and the hardware value from 3.66 cm to 1.74 cm, while chunked parallel training cut learning time from about 40 hours to about 10 hours.
Design the handover and fallback procedures. Use a conventional controller to establish the plasma and hold position and total current, hand over to the learned controller at a prespecified moment, and keep the fallback controller and interlocks active throughout.
5. Comparison with Alternatives
Against classical magnetic control, the classical approach uses a set of sub-controllers for radial position, vertical position, shape, and divertor position, each closing its own loop. This work instead uses a single neural network that outputs voltages for all nineteen coils, leaving the question of how to trade off conflicting objectives to the training process. The distinction is not only structural simplification. In a multi-controller scheme the arbitration rule is written by an engineer; in the neural scheme the rule is encoded in a reward function, which is still written by a person but expresses a preference rather than a coordination procedure.
Against PID and linear controllers, the argument advanced here is that classical methods depend on equilibrium reconstruction and linearized models, which limits adaptability. A later study on the mid-size DIII-D device pushes this further, using Soft Actor-Critic to remove the need for real-time equilibrium reconstruction and training on NSFsim, a two-dimensional Grad-Shafranov equilibration solver with a circuit equation plus a one-dimensional transport solver. That work reports that reproducing the kinetic parameter evolution alongside the magnetic equilibrium evolution is a key factor affecting control quality, and it validated the approach experimentally on DIII-D, preserving control performance during transients and reaching target parameters from the first discharge without extra tuning.
Against disruption avoidance, the Princeton group uses a multimodal dynamic model as the reinforcement learning training environment to avoid tearing instability, published in Nature in 2024. The difference is one of objective type: shape control is a continuous accuracy problem, while disruption avoidance is a sparse and extremely costly safety event.
Against replication on other devices, the WEST machine in France published magnetic control by deep reinforcement learning in IEEE Transactions on Plasma Science in 2024. Such cross-device work suggests the method is transferable, and it also shows that transfer requires retraining or adaptation.
Against other reinforcement learning landings in fluid mechanics, flow control and neural operator work address a different path, usually geometric optimization or load control rather than real-time safety-relevant shape holding.
6. Benchmarks and Evidence
The control metrics from the Nature paper are specific. In the snowflake experiment, the policy first established a snowflake configuration with X-points separated by 34 cm, then manipulated the far X-point toward the limiting X-point until the separation reached 6.6 cm, tracking the time-varying X-point targets with a combined RMSE of 3.7 cm while maintaining plasma current RMSE of 0.50 kA and shape RMSE of 0.65 cm. In the droplet experiment, two independent droplets were stabilized over the entire 200 ms control window while the current was ramped within each domain. Target shape points had a 2 cm radius and were compared against the equilibrium-reconstructed plasma boundary across several configurations, including elongation 1.9 with a vertical instability growth rate of 1.4 kHz, an ITER-like shape with neutral beam heating, a diverted negative triangularity of minus 0.8, and a snowflake with time-varying control of the bottom X-point.
The practical follow-up reports a comparison table for snowflake X-point tracking. On simulation, the previous policy achieved a mean X-point RMSE of 3.46 cm and a mean LCFS RMSE of 0.078, while the reward-shaped policy achieved 0.27 cm and 0.0069. On hardware, the previous TCV shot 77505 achieved 3.66 cm and 0.0110, and the reward-shaped TCV shot 70755 achieved 1.74 cm and 0.0119. LCFS accuracy stayed essentially unchanged on hardware while X-point accuracy improved substantially.
Training scale and cost are also reported: 5,000 actors in parallel, training times of one to three days and longer for complex targets, with an actor-count sweep showing that far fewer actors can reach comparable performance at the cost of longer training. The chunked parallel scheme reaches a reward of 96 out of 100 in roughly 10 hours, against about 40 hours for the baseline.
Simulator and network specifications: simulator time step 50 kHz, control rate 10 kHz, five substeps per policy evaluation, actor of 266,280 parameters, critic of 718,337 parameters with a recurrent LSTM, and a symmetric-design control critic of 266,497 parameters. The device geometry is given as a vessel 1.5 m high with a minor radius of 0.88 m and a vessel half-width of 0.26 m, with 19 actuated coils.
7. Role in Physics-AI Workflows
One use is to hand objectives that are given and mutually coupled to a learned controller. Shape control is a textbook case where the operator specifies objectives and many constraints interact, and reinforcement learning contributes by expressing preferences in the reward function instead of hard-coding arbitration rules in the controller structure.
A second use is to train on simulator data rather than experimental data. Because experimental data is expensive and scarce, essentially all training data comes from the simulator. This data-engine-first organization is a common pattern in physical AI: build a trustworthy kernel, then layer algorithms on top of it.
A third use is to leave non-differentiable physics to the simulator and hand differentiable pieces to surrogates. The virtual circuit idea learns an approximation of the sensitivity matrix from shape emulators for real-time magnetic control. Recent work quantifies the real-time applicability of such emulators: agreement between Jacobian-based target displacements and finite-difference Jacobians on exact Grad-Shafranov solutions reaches roughly 5 to 10 percent on selected geometric targets with neural emulators of about 1e5 parameters, training requires about 1e5 to 1e6 synthetic equilibria, and container currents that are not measured in real time can be inferred by linear regression over a trailing window of active coil currents with residuals of only a few amperes.
A fourth use is to provide an algorithm baseline that transfers to other thermal-fluid systems. Non-symmetric actor-critic design, reward shaping, integral observations, chunked parallelism, and domain randomization are device-independent techniques.
A fifth use is to anchor surrogate model work in fusion. Neural surrogates and differentiable simulators for tokamak physics are already catalogued here, covering fast electromagnetic solution and kinetic transport approximation.
8. Limitations and Failure Modes
Disruption avoidance cannot be expected from this scheme. The training simulator solves magnetic equilibrium only and contains no disruption criterion model, so the learned behaviour carries no guarantee of avoiding disruptions. When avoidance is required, the path is to plug a multimodal dynamic model in as the training environment.
Targets outside the trained envelope carry no guarantee. The randomized parameter intervals are explicitly bounded, and behaviour beyond them is unverified. A new target means retraining, which is an inherent property of the current paradigm.
Perfect snowflake shapes still show an accuracy gap on hardware. After reward shaping the hardware X-point RMSE is 1.74 cm, still visibly separated from the 0.27 cm simulation value, and the paper acknowledges this.
LCFS accuracy and X-point accuracy trade against each other. In the reported table the reward-shaped policy has a mean LCFS RMSE of 0.0119 against 0.0110 for the previous policy, so gains in X-point accuracy do not automatically bring gains in boundary accuracy.
Deployment depends on device-specific real-time infrastructure. The trained product is a real-time executable tightly coupled to one machine, and moving to another device requires retraining or substantial adaptation.
The technology sits at the research validation stage. FGE and LIUQE are made available subject to a license agreement from the Swiss Plasma Center at EPFL, while the MPO reference implementation and the libraries used, including launchpad, dm_env, sonnet, tensorflow, and reverb, are provided under open-source licenses. This level of open access differs from what industrial software typically offers.
9. Version and Source Notes
The primary paper is by Degrave and colleagues, published online on 16 February 2022 in Nature, volume 602, pages 414 to 419, DOI 10.1038/s41586-021-04301-9, and it claims the first verification of deep reinforcement learning based magnetic control on a real tokamak. The practical follow-up, Towards practical reinforcement learning for tokamak magnetic control, has 17 authors led by Tracey and appeared in Fusion Engineering and Design, volume 200, article 114161, in March 2024, DOI 10.1016/j.fusengdes.2024.114161, with the public preprint at arXiv:2307.11546.
One correction matters for anyone reusing this entry. An earlier version pointed its source URL at arXiv:2104.06277, an identifier that resolves to a paper on network intrusion detection and has nothing to do with plasma control. Checking the arXiv index shows that the 2022 Nature paper has no arXiv preprint, so the available public preprint is the 2023 practical follow-up, and the source URL now points to that identifier.
The parallel disruption line is the Nature volume 626, pages 746 to 751, dated 21 February 2024, DOI 10.1038/s41586-024-07024-9, with ten authors. The WEST replication appeared in IEEE Transactions on Plasma Science, volume 52, pages 3698 to 3703, in September 2024, DOI 10.1109/tps.2024.3377811.
On software availability, the primary paper states that the MPO reference implementation is available under an open-source license and that launchpad, dm_env, sonnet, tensorflow, and reverb are also open source, with the code computing control targets, rewards, and termination conditions provided in the supplementary information. FGE and LIUQE require a license agreement from the Swiss Plasma Center at EPFL. TCV experimental data corresponding to the figures are available in the supplementary information, with source data provided with the paper.
The training stack also changed between the two papers. The primary work uses the TensorFlow ecosystem, while the practical follow-up uses JAX and Haiku for neural network construction and parameter training throughout. Related preprints include arXiv:2506.13267 from June 2025 on reconstruction-free magnetic control at DIII-D, arXiv:2509.01789 from September 2025 on real-time applicability of emulated virtual circuits, and arXiv:2406.06718 from June 2024 describing TORAX, a fast differentiable tokamak transport simulator built on JAX and verified against the established RAPTOR code.
10. References
- Degrave N., Minciac E., Lele A., Yu A., Yadsabadi M., et al., "Magnetic control of tokamak plasmas through deep reinforcement learning", Nature, Vol. 602, 16 February 2022, pp. 414-419, DOI 10.1038/s41586-021-04301-9
- Tracey J., Michi K., Chervonyi V., Davies T., Paduraru R., et al., "Towards practical reinforcement learning for tokamak magnetic control", Fusion Engineering and Design, Vol. 200, March 2024, article 114161, DOI 10.1016/j.fusengdes.2024.114161, arXiv:2307.11546
- Seo J., Kim D., Jalalvand M., Conlin M., Rothstein A., et al., "Avoiding fusion plasma tearing instability with deep reinforcement learning", Nature, Vol. 626, 21 February 2024, pp. 746-751, DOI 10.1038/s41586-024-07024-9
- Kerboua-Benlarbi S., Nouailletas A., Faugeras B., Nardon E., Moreau P., "Magnetic Control of WEST Plasmas Through Deep Reinforcement Learning", IEEE Transactions on Plasma Science, Vol. 52, September 2024, pp. 3698-3703, DOI 10.1109/tps.2024.3377811
- "Reconstruction-free magnetic control of DIII-D plasma with deep reinforcement learning", arXiv:2506.13267, June 2025, https://arxiv.org/abs/2506.13267
- "Real-Time Applicability of Emulated Virtual Circuits for Tokamak Plasma Shape Control", arXiv:2509.01789, September 2025, https://arxiv.org/abs/2509.01789
- "TORAX: A Fast and Differentiable Tokamak Transport Simulator in JAX", arXiv:2406.06718, June 2024, https://arxiv.org/abs/2406.06718
- MPO maximum a posteriori policy optimization reference implementation under an open-source license, together with the libraries launchpad, dm_env, sonnet, tensorflow, and reverb listed in the code availability statement of the primary paper, https://github.com/deepmind/acme
- Swiss Plasma Center at EPFL license statement for the FGE and LIUQE software, referenced in the code availability paragraph of the primary paper, https://www.epfl.ch/labs/spc/
- Public technical description of the TCV tokamak including machine parameters, quoted in Figure 1 of the primary paper and published by the Swiss Plasma Center