Pangu-Weather: A 3D Earth-Specific Transformer for Global Medium-Range Weather Forecasting
New1. Overview
Pangu-Weather was developed by a six-person team at Huawei Cloud, namely Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu and Qi Tian. It is the first global medium-range weather forecasting model whose accuracy exceeds the ECMWF operational numerical prediction system across every tested variable and every forecast lead time. It produces global forecasts from one hour to one week on a 0.25 degree grid, and a five-day forecast takes roughly 1400 milliseconds on a single GPU.
Two formal versions of the paper exist. The Nature article "Accurate medium-range global weather forecasting with 3D neural networks" appeared on 5 July 2023 in volume 619, pages 533 to 538, DOI 10.1038/s41586-023-06185-3, under a CC BY 4.0 licence. The earlier arXiv preprint 2211.02556 was published on 3 November 2022 and has remained at version 1; the README notes that the arXiv version carries more technical detail while the Nature version adds some figures. The repository also links a Zenodo archive, DOI 10.5281/zenodo.7678849, labelled v1.0 and issued on 26 February 2023.
The official repository is 198808xc/Pangu-Weather, created on 11 January 2023, with the last code push on 12 January 2024. At the time of verification it carries roughly 1402 stars, 242 forks and 54 open issues, is written mainly in Python, is not archived, and uses main as the default branch. The repository is small, containing eleven entries only: a README, three constant mask files under constant_masks, three inference scripts, a pseudocode.py file describing the algorithm, and two requirements files.
Licensing needs separate attention. The repository ships no LICENSE file, and the GitHub API reports a null license field. The README states only that the trained parameters are made available under BY-NC-SA 4.0 and that commercial use of the models is forbidden. The weight licence is therefore explicit and non-commercial, while the licence status of the code is undeclared and must be checked separately before use. The README also reminds users that all models were trained on ERA5 data from ECMWF and that the corresponding Copernicus policy applies.
2. How It Works
Pangu-Weather rests on two technical contributions: a 3D Earth-Specific Transformer architecture that introduces the pressure level dimension explicitly, and a hierarchical temporal aggregation algorithm that reduces accumulated iteration error.
The input is three-dimensional. Four surface variables are used, namely mean sea level pressure, the two components of 10 metre wind and 2 metre temperature, plus five upper-air variables, geopotential, specific humidity, temperature and the two wind components, spread over thirteen pressure levels on a 1440 by 721 grid at 0.25 degree spacing. The paper notes that full ERA5 provides thirty-seven pressure levels with twenty-one variables each plus 262 surface variables, and that only the reduced set is used to keep the computational cost manageable.
Patches and the encoder-decoder structure follow the vision transformer convention. Upper-air patches are 2 by 4 by 4 in height, latitude and longitude, surface patches are 4 by 4, and the sliding window stride equals the patch size with zero padding where needed. Encoder and decoder each have eight layers. The first two operate at full resolution on 8 by 360 by 181 feature maps, and the remaining six operate at half resolution on 8 by 180 by 91 maps while raising the channel count from 192 to 384. Patch embedding and patch recovery hold separate parameters.
To control cost, the model inherits window attention from the vision transformer, partitioning feature maps into windows of at most 2 by 12 by 6 tokens with standard self-attention inside each window and alternating shifted windows. The key modification is an Earth-specific positional bias, learned in blocks by latitude, height and longitude so that token relationships differ across latitudes and pressure layers. The paper quantifies the effect: in the first block the learnable parameter count rises from a product of window extents to one roughly 527 times larger. The change does not increase FLOPs, and because useful priors are introduced, the model converges faster in training.
Hierarchical temporal aggregation avoids training one long-lead model. Four independent networks are trained for lead times of 1, 3, 6 and 24 hours, and at inference time a greedy algorithm combines them to reach a target lead time with the fewest iterations. A seven-day forecast runs the 24-hour model seven times; a 23-hour forecast runs the 6-hour model three times, the 3-hour model once and the 1-hour model twice. Compared with approaches that compute both a single step and a recursive step and stack two loss terms, this saves memory for the same model size and avoids the instability of training a recurrent network.
Training details are explicit. The loss is an MAE term with the surface component weighted 0.25. The optimizer is Adam with learning rate 5e-4 and weight decay 3e-6, and the DropPath rate rises linearly from 0 to 0.2 with depth. Each of the four models is trained for 100 epochs, taking 16 days on 192 NVIDIA Tesla V100 GPUs, with a README estimate of roughly 60 TB of ERA5 data and about 3000 V100 GPU days per model. The paper states plainly that training had not fully converged at 100 epochs because of the compute budget.
For ensemble forecasting the initial state is perturbed with random Perlin noise: 99 noise vectors of the same size as the initial field are generated, the amplitude coefficient is 0.2, and the perturbed members plus one control member give 100 members whose outputs are averaged. The paper explains why the alternative of perturbing model parameters was not attempted, noting that a deep network holds hundreds of millions of parameters that are highly sensitive to random seeds and sampling strategies.
3. Scope and Assumptions
The licence constraint comes first. Weights are released under CC BY-NC-SA 4.0 with commercial use explicitly forbidden, the repository has no LICENSE file so the code licence is undeclared, and the training data remains subject to the Copernicus terms, which together make a commercial product path unavailable under the current arrangement.
Initial conditions are restricted. The README supports ERA5 reanalysis and ECMWF operational initial fields such as the initial state of an HRES forecast, the latter usually giving slightly different accuracy. Other kinds of initial fields are explicitly not supported, because fields may differ dramatically when geopotential takes negative values.
Data formats are fixed. Arrays must be single precision, so a float32 cast is required. The surface array has shape 4 by 721 by 1440 and the upper-air array 5 by 13 by 721 by 1440, with latitude running from 90 to minus 90 and longitude from 0 to 359.75. Variable and level ordering is hard-coded: the surface order is MSLP, U10, V10 and T2M, the upper-air order is Z, Q, T, U and V, and the thirteen pressure levels run 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100 and 50 hPa. Geopotential Z is the geopotential itself, so raw geopotential height must be multiplied by 9.80665.
Hardware and environment assumptions follow. The GPU path requires CUDA 11.6 with cuDNN 8.2.4 on Linux or 8.5.0.96 on Windows, while a CPU path installs from a separate requirements file. Community issue 63 reports insufficient memory during inference on an RTX 3060 with 12 GB, showing that actual inference memory exceeds the 1.1 GB weight size.
Training capability is the most important limitation. The repository publishes inference code and pseudocode only, with no training implementation. Issue 58, opened on 12 January 2024 with one reply, asks whether training code and lite checkpoints will be released, and it remains open at the time of verification with no timetable in the documentation. The README gives the data volume and GPU day budget but no runnable training script.
The lite version carries its own assumptions. The README reduces training data to eleven years from 2007 to 2017, uses only the 00 UTC time point per day, changes the first-stage downsampling from 2 by 4 by 4 to 2 by 8 by 8, trains a single 24-hour model, and allows the epoch count to stay at 100 or drop to 50 with the cosine annealing schedule adjusted accordingly. The team states that the lite models were trained and tested only at 00 UTC, that performance at other times of day is not guaranteed, and that the version exists to let researchers with limited resources explore the method.
4. Getting Started
The directory layout described in the README is flat. A root directory holds an input_data folder containing input_surface.npy and input_upper.npy, an output_data folder, the four ONNX models pangu_weather_1, 3, 6 and 24 onnx, and three inference scripts.
Weights are distributed as four ONNX files of roughly 1.1 GB each, offered through Google Drive and Baidu netdisk links. Because the format is ONNX, the models can be called from C++, C#, Java and other languages without a Python dependency.
Input preparation uses netCDF4 to convert ERA5 netCDF files and pygrib to convert ECMWF GRIB initial fields. The README also provides ready-made example inputs corresponding to the ERA5 initial state of 27 September 2018 at 12 UTC.
Inference is covered by three scripts. inference_cpu.py performs a single 24-hour forecast with the 24-hour model on CPU, inference_gpu.py is the GPU equivalent, and inference_iterative.py demonstrates producing every 6-hour forecast across one week. Each requires its matching requirements file first.
For independent implementation, pseudocode.py spells out the algorithm in Python while importing operators from a deep learning library such as PyTorch or TensorFlow. It states the 100-epoch schedule, the MAE loss with surface weight 0.25, the learning rate of 5e-4 and the weight decay of 3e-6.
Third-party tooling is available. ECMWF published the ecmwf-lab/ai-models-panguweather repository on 12 May 2023 under Apache-2.0, last pushed on 24 September 2024 with roughly 116 stars, offering a toolkit for running the model. The README also points to ECMWF open data, which distributes models and initial fields so that users can run forecasts themselves.
The operational status has changed and must be accounted for. ECMWF published a blog post titled "Farewell to the external AI models" on 11 May 2026 announcing that real-time operation of external AI models stops with the implementation of IFS cycle 50r1. At the beginning of 2026 ECMWF ran Pangu-Weather from Huawei, GraphCast from Google DeepMind, Aurora from Microsoft Research and FourCastNet from NVIDIA, twice daily at 00 and 12 UTC. The blog notes that charts based on PanguWeather, such as T850, are no longer on openCharts, although users can still obtain forecasts by running the models from ECMWF open data. Any earlier claim that the model entered ECMWF operational service therefore needs this timeline attached.
5. Comparison with Alternatives
The comparison with the ECMWF operational IFS is the core claim of the paper. For Z500, the 3-day and 5-day RMSE of operational IFS at 152.8 and 333.7 are reduced to 134.5 and 296.7. For T850, IFS values of 1.37 and 2.06 fall to 1.14 and 1.79. For T2M, 1.34 and 1.75 fall to 1.05 and 1.53, and for U10, 1.94 and 2.90 fall to 1.61 and 2.53. The paper states that the relative RMSE reduction exceeds 10 percent in every scenario, which corresponds to a forecast time gain of 10 to 15 hours.
Against FourCastNet the gain is larger. The paper reports that FourCastNet gives 5-day Z500 RMSE of 484.5 for a single model and 462.5 for a 100-member ensemble, whereas Pangu-Weather reduces RMSE by more than 30 percent in the scenarios above, with the forecast time gain widening beyond 36 hours.
Against other AI models of the same generation, GraphCast published in Science in 2023 works on a spherical graph grid with message passing, while Pangu-Weather uses a regular grid with a three-dimensional transformer; earlier evaluations gave Pangu-Weather the accuracy advantage.
The hybrid route is a different design choice. NeuralGCM keeps a differentiable dynamical core and learns only the unresolved parameterization, which gives more control over long integrations and climate simulation, while Pangu-Weather is purely data-driven, so conservation laws cannot automatically suppress iteration error, which is exactly why the hierarchical temporal aggregation algorithm was needed.
For ensembles, the 100-member mean is slightly worse than the single deterministic forecast at short lead times such as one day and clearly better beyond five days. Over seven days the latitude-weighted RMSE of Z500 falls from 500.3 to 450.6 and that of U10 from 3.48 to 2.96, relative reductions of 10 and 15 percent, with larger benefits for less smooth quantities such as 500 hPa specific humidity and 10 metre wind.
The independent evaluation by ECMWF is two-sided. The technical report arXiv 2307.10128, released in July 2023, compared PanguWeather with the operational IFS from identical initial conditions in an operational-like context. Accuracy on global metrics and extreme events was comparable, discrimination of summer temperature extremes measured by AUC exceeded that of IFS with statistical significance between the 75 and 90 percentiles, and three drawbacks were identified: forecasts are overly smooth with clear small-scale damping, bias grows with lead time faster than for IFS and ERA5, and tropical cyclone intensity is markedly underestimated.
6. Benchmarks and Evidence
The data split uses hourly ERA5 reanalysis from 1979 to 2017, 39 years in total, for training, with 2019 for validation and 2018, 2020 and 2021 for testing. The paper also refers to downloading 43 years of data spanning 1979 to 2021, which describes the download range rather than the training range, and the two numbers should not be conflated.
The verification metrics are latitude-weighted RMSE and ACC, with climatology estimated from the long-term mean over the 39 years of training data.
Key numbers compare 3-day and 5-day RMSE. For Z500 the operational IFS gives 152.8 and 333.7 against Pangu-Weather at 134.5 and 296.7. For T850 the IFS gives 1.37 and 2.06 against 1.14 and 1.79, for T2M 1.34 and 1.75 against 1.05 and 1.53, and for U10 1.94 and 2.90 against 1.61 and 2.53. The paper also reports the values with December 2018 included, giving 133.9 and 294 for Z500 and 1.13 and 1.77 for T850.
For tropical cyclone tracks, the evaluation set TC2018 consists of the 88 named cyclones of 2018 that appear in both IBTrACS and the ECMWF-HRES forecasts. The mean direct position errors of Pangu-Weather at 3 and 5 days are 120.29 km and 195.65 km, against 162.28 km and 272.10 km for ECMWF-HRES. In a case study the track of Typhoon Yutu was predicted correctly 48 hours earlier.
On inference speed the paper gives about 1400 milliseconds for a single inference on one GPU, more than 10000 times faster than operational IFS and on par with FourCastNet. The direction matters: Pangu-Weather is faster than IFS, not the reverse.
Parameter counts and training budget are also given. The four models together hold about 256 million parameters, roughly 64 million per network, each trained for 16 days on 192 V100 GPUs.
The lite results in the README table are conditioned on testing at 00 UTC only. For Z500 at 3 and 5 days the operational IFS gives 152.8 and 333.7, the full Pangu-Weather gives 134.5 and 296.7 with 39 years of data, 2 by 4 by 4 downsampling, 100 epochs and 192 by 16 GPU days, Lite1 gives 163.1 and 338.2 with 11 years, 2 by 8 by 8, 100 epochs and 8 by 6 GPU days, and Lite2 gives 177.9 and 357.5 with 11 years, 2 by 8 by 8, 50 epochs and 8 by 3 GPU days. For T850 the lite values are 1.29 and 1.96 and 1.36 and 2.05, for T2M 1.16 and 1.64 and 1.24 and 1.71, and for U10 1.80 and 2.74 and 1.90 and 2.84. The README conclusion is that, when tested only at 00 UTC, the lite version can surpass operational IFS on T850, T2M and U10 while using less than one percent of the computational cost.
For the ensemble, the 7-day latitude-weighted RMSE of Z500 falls from 500.3 for the deterministic forecast to 450.6 for the 100-member mean, and U10 falls from 3.48 to 2.96.
7. Role in Physics-AI Workflows
The most direct use is as a fast surrogate model. A five-day forecast costing about 1400 milliseconds, distributed as 1.1 GB ONNX files, fits naturally into parameter sweeps, sensitivity studies and scenario experiments that require hundreds or thousands of global integrations, a volume at which traditional IFS is unaffordable. This substitutes a trained model for an expensive numerical solve, which is the central theme of surrogate modelling in this field.
The ensemble capability is underappreciated. Because a single inference is cheap, a 100-member ensemble has negligible marginal cost, and the paper shows the ensemble clearly outperforms the deterministic forecast beyond five days. For downstream applications that need probabilistic output, such as shipping windows, agricultural scheduling and energy load prediction, this path is more useful than single-member output.
The three-dimensional field modelling pattern also transfers. Treating the pressure level as an explicit third dimension, and learning latitude-dependent positional bias so that token relationships vary by latitude and height, is not specific to meteorology. For coupled multiphysics engineering design on volumetric grids, the combination of regular grid, height dimension and latitude-dependent bias offers a directly reusable structure, while hierarchical temporal aggregation is a general engineering answer to accumulated error in multi-step simulation.
For extreme-event tracking, the quantified improvement in cyclone track error suggests that gains on path-type questions, which are relative position quantities, can exceed gains on intensity questions. Decision processes that mainly ask whether a region will be affected can exploit that division of labour.
8. Limitations and Failure Modes
Commercial use is restricted by the licence. The weights are CC BY-NC-SA 4.0 with commercial use explicitly forbidden, and the absence of a LICENSE file leaves the code licence undeclared, which differs from the common arrangement in which code is open and weights are commercially usable.
Small-scale damping and over-smoothing are measured. The ECMWF evaluation finds statistically detectable smoothing of PanguWeather forecasts at small scales, and a case study shows PGW maps that are smoother than IFS and deprived of smaller-scale structure. The mechanism is consistent with mean squared or absolute error losses discouraging bold forecasts, and the practical consequence is difficulty representing local severe weather such as convective snow and gusts.
Bias grows faster with lead time. The same evaluation finds PGW bias rising more quickly than IFS and ERA5, so systematic bias correction learned from IFS experience cannot be transferred directly.
Tropical cyclone intensity is clearly underestimated. ECMWF lists this as a separate defect, and the paper's own comparison covers only track error without an intensity metric, so the relative advantage on tracks cannot be extended to intensity.
Precipitation forecasts are absent. The 2026 ECMWF blog names the lack of precipitation output as one reason limiting the practical usefulness of Aurora and PanguWeather. For scenarios where precipitation is the key input this is a capability boundary rather than a configuration issue.
Training and further development are constrained. Training code is unreleased under issue 58 and lite weights are also unreleased, so secondary training, custom fine-tuning and the paper's proposed child-model route all lack a starting point.
Runtime consistency is open to doubt. Issue 63 reports insufficient memory on a 12 GB device, issue 66 points out that the three constant mask files in the repository, land_mask.npy, soil_type.npy and topography.npy, appear unused during inference with no documented explanation, and issue 74 reports an anomaly in the magnitude of surface output. All three remain open community reports and deserve checking during reproduction.
Maintenance activity is low. The last code push was 12 January 2024 with no push since at the time of verification, and 54 issues remain open. For a model that entered commercial use and was run in parallel by an official institution for years, this cadence does not match its influence.
The operational endorsement has changed on a timeline. ECMWF announced on 11 May 2026 that real-time operation of external AI models stops with IFS cycle 50r1, and the PanguWeather charts left openCharts. The strongest external validation source, official parallel operational testing, has ended, and subsequent model quality must be assessed independently.
Training had not converged. The paper itself states that training was still unconverged after 100 epochs, and that no exhaustive ablation was performed, with the team noting that higher-accuracy configurations likely exist.
9. Version and Source Notes
Three titles coexist. The Nature title reads "Accurate medium-range global weather forecasting with 3D neural networks". The arXiv metadata title reads "Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast", while the arXiv PDF running head uses the word "System" in place of "Model", and the README cites the variant containing "Model". A citation must name the specific version.
A correction exists. The Methods section on mathematical settings had the longitude and latitude dimensions swapped, and the Author Correction published in Nature volume 621, page E45, on 14 September 2023, DOI 10.1038/s41586-023-06545-z, states that both the HTML and PDF have been revised.
The training-year count needs care. The 43 years in the abstract describes the download range 1979 to 2021, while training actually uses 39 years from 1979 to 2017, with 2019 for validation and 2018, 2020 and 2021 for testing.
The direction of the inference-speed claim matters. The 1400 milliseconds is the time for one Pangu-Weather inference on a single GPU, meaning Pangu-Weather is more than 10000 times faster than operational IFS and on par with FourCastNet.
Forecast time gains have distinct baselines. The gain is 10 to 15 hours relative to IFS and beyond 36 hours relative to FourCastNet, and the two figures cannot be interchanged.
Licensing is layered across objects. The paper text is CC BY 4.0, the trained weights are CC BY-NC-SA 4.0 with commercial use forbidden, the repository root has no LICENSE file and the GitHub API license field is null, and the ECMWF runtime toolkit ecmwf-lab/ai-models-panguweather is Apache-2.0. Each must be checked separately before citation or deployment.
The ensemble composition is 99 Perlin-noise perturbed members plus one control member, with amplitude coefficient 0.2.
The ECMWF operational timeline runs as follows. On 12 May 2023 ECMWF released the runtime toolkit repository. The README announced on 17 July 2023 that Pangu-Weather went live in the ECMWF operational suite with ten-day global forecasts viewable without running code. At the beginning of 2026 four external models ran twice daily at 00 and 12 UTC, namely Pangu-Weather, GraphCast, Aurora and FourCastNet. On 11 May 2026 ECMWF announced the end of real-time operation with IFS cycle 50r1, the relevant openCharts charts were withdrawn, and the models remain runnable from ECMWF open data.
The cycle-upgrade experiment produced a grouped result. ECMWF initialized models with the 50r1 pre-operational analysis between 21 January and 20 March 2026 and verified against the 50r1 analysis, comparing with 49r1 initialization and verification. PanguWeather and FourCastNet were affected neutrally, while GraphCast, Aurora and AIFS v1.1 degraded, because models with a fine-tuning step are more sensitive to the exact initializing analysis. PanguWeather's lower sensitivity to initial conditions is an advantageous feature in that round.
The lite evaluation condition is that all lite results derive from training and testing at the 00 UTC time point alone, with no guarantee for other times of day.
10. References
- Nature article "Accurate medium-range global weather forecasting with 3D neural networks", volume 619, pages 533 to 538, 5 July 2023, DOI 10.1038/s41586-023-06185-3, CC BY 4.0.
- Author Correction "Correction to: Accurate medium-range global weather forecasting with 3D neural networks", Nature volume 621, page E45, 14 September 2023, DOI 10.1038/s41586-023-06545-z.
- arXiv preprint "Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast", arXiv 2211.02556 v1, 3 November 2022.
- Official Pangu-Weather repository at https://github.com/198808xc/Pangu-Weather , containing inference scripts, pseudocode and constant masks, with no LICENSE file.
- Zenodo software archive, DOI 10.5281/zenodo.7678849 , titled 198808xc/Pangu-Weather: v1.0, issued 26 February 2023.
- ECMWF runtime toolkit repository at https://github.com/ecmwf-lab/ai-models-panguweather , licensed Apache-2.0.
- ECMWF technical report "The rise of data-driven weather forecasting", arXiv 2307.10128, 19 July 2023.
- ECMWF blog post "Farewell to the external AI models", 11 May 2026.
- ERA5 reanalysis dataset paper "The ERA5 global reanalysis", Quarterly Journal of the Royal Meteorological Society, 2020.
- Pathak J. et al., "FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators", arXiv 2202.11214, 2022.
- GraphCast paper "Learning skillful medium-range global weather forecasting", Science volume 382, pages 1416 to 1421, 2023.
- IBTrACS international best track archive for climate stewardship, fourth version, used for tropical cyclone track verification.
- ECMWF TIGGE archive, source of operational IFS and ECMWF-HRES forecasts.
- Copernicus dataset licence terms at https://apps.ecmwf.int/datasets/licences/copernicus/ .
- Creative Commons BY-NC-SA 4.0 licence text at https://creativecommons.org/licenses/by-nc-sa/4.0/ .
- ECMWF open data distribution page, providing models and initial states for self-run forecasts.