EPFL’s Swiss Plasma Center (SPC) and DeepMind, dedicated to “solving intelligence to advance science and humanity,” developed a new magnetic control method for plasmas based on deep reinforcement learning and applied it for the first time to a real-world plasma in the SPC’s tokamak research facility, TCV.
Tokamaks are donut-shaped devices used for nuclear fusion research, and the SPC is one of the few research facilities in the world that has one in operation. These devices employ a massive magnetic field to confine plasma at extremely high temperatures—hundreds of millions of degrees Celsius, hotter than the sun’s core, so nuclear fusion between hydrogen atoms may occur. In addition, the energy generated by fusion is being researched for application in electricity generation.
The SPC’s tokamak supports many plasma configurations, thereby known as variable-configuration tokamak (TCV). As a result, scientists may utilize it to test novel methods for containing and controlling plasmas. The configuration of a plasma refers to its form and location within the apparatus.
Tokamaks generate and sustain plasmas using a sequence of magnetic coils, and the voltage must be constantly monitored. Otherwise, the plasma may clash with the vessel walls, causing it to degrade. To avoid this, SPC researchers first test their control system setups on a simulator before utilizing them in the TCV tokamak.
“Our simulator is based on more than 20 years of research and is updated continuously,” says Federico Felici, an SPC scientist and co-author of the study. “But even so, lengthy calculations are still needed to determine the right value for each variable in the control system. That’s where our joint research project with DeepMind comes in.”
DeepMind experts created and trained an AI system to produce and maintain certain plasma configurations on the SPC’s simulator. This entailed first having the algorithm test various control tactics in simulation and gaining expertise. Next, the program produced a control strategy to achieve the targeted plasma configuration based on collected experience. This included first running the algorithm through various parameters and then analyzing the plasma configurations that occurred from each one. The system was then asked to generate a specific plasma configuration by selecting the appropriate conditions.
After being trained, the AI-based system was able to develop a broad range of plasma forms and complex configurations, including one in which two independent plasmas are kept in the vessel simultaneously. Finally, the study team put their new system through its paces on the tokamak to test how it would do under real-world situations.
“DeepMind was immediately interested in the prospect of testing their AI technology in a field such as nuclear fusion, and especially on a real-world system like a tokamak,” says Felici.
Martin Riedmiller, DeepMind’s control team head and co-author of the study, states that “our team’s mission is to research a new generation of AI systems—closed-loop controllers—that can learn in complex dynamic environments completely from scratch. Controlling a fusion plasma in the real world offers fantastic, albeit extremely challenging and complex, opportunities.”
DeepMind volunteered to collaborate with the SPC to design an AI-based control system for its tokamak after consulting with Felici.
“We agreed to the idea right away because we saw the huge potential for innovation,” says Ambrogio Fasoli, the director of the SPC and a co-author of the study. “All the DeepMind scientists we worked with were highly enthusiastic and knew a lot about implementing AI in control systems.”
DeepMind benefited greatly from the collaborative research work, demonstrating the value of a multidisciplinary approach to both sides.
“The collaboration with the SPC pushes us to improve our reinforcement learning algorithms, and as a result can accelerate research on fusing plasmas,” says Brendan Tracey, a senior research engineer at DeepMind and co-author of the study.
This collaboration should open the way for EPFL to pursue future cooperative R&D possibilities with outside firms.
“We’re always open to innovative win-win collaborations where we can share ideas and explore new perspectives, thereby speeding the pace of technological development,” says Fasoli.
The research has been published here.