| Title: |
High-Fidelity Data-Driven Dynamics Model for Reinforcement Learning-based Magnetic Control in HL-3 Tokamak |
| Authors: |
Wu, Niannian, Yang, Zongyu, Li, Rongpeng, Wei, Ning, Chen, Yihang, Dong, Qianyun, Li, Jiyuan, Zheng, Guohui, Gong, Xinwen, Gao, Feng, Li, Bo, Xu, Min, Zhao, Zhifeng, Zhong, Wulyu |
| Publication Year: |
2024 |
| Collection: |
Physics (Other) |
| Subject Terms: |
Physics - Plasma Physics |
| More Details: |
The drive to control tokamaks, a prominent technology in nuclear fusion, is essential due to its potential to provide a virtually unlimited source of clean energy. Reinforcement learning (RL) promises improved flexibility to manage the intricate and non-linear dynamics of the plasma encapsulated in a tokamak. However, RL typically requires substantial interaction with a simulator capable of accurately evolving the high-dimensional plasma state. Compared to first-principle-based simulators, whose intense computations lead to sluggish RL training, we devise an effective method to acquire a fully data-driven simulator, by mitigating the arising compounding error issue due to the underlying autoregressive nature. With high accuracy and appealing extrapolation capability, this high-fidelity dynamics model subsequently enables the rapid training of a qualified RL agent to directly generate engineering-reasonable magnetic coil commands, aiming at the desired long-term targets of plasma current and last closed flux surface. Together with a surrogate magnetic equilibrium reconstruction model EFITNN, the RL agent successfully maintains a $100$-ms, $1$ kHz trajectory control with accurate waveform tracking on the HL-3 tokamak. Furthermore, it also demonstrates the feasibility of zero-shot adaptation to changed triangularity targets, confirming the robustness of the developed data-driven dynamics model. Our work underscores the advantage of fully data-driven dynamics models in yielding RL-based trajectory control policies at a sufficiently fast pace, an anticipated engineering requirement in daily discharge practices for the upcoming ITER device. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2409.09238 |
| Accession Number: |
edsarx.2409.09238 |
| Database: |
arXiv |
