MainQuantum computers are fundamentally analogue machines, which makes them extremely fragile compared with the digital classical computers17. Quantum error correction (QEC) mitigates this vulnerability by effectively digitizing the errors: through repetitive error detection embedded in QEC protocols, spurious analogue evolution branches into a sequence of ‘error’ or ‘no error’ events. These binary error-detection signals can then be decoded to correct the logical quantum state, providing a mechanism for reaching low logical error rates (LERs) required for practical applications.However, the digitization of errors alone is insufficient, and the practical success of QEC is critically dependent on the analogue control of the constituent qubits: QEC is effective only if the physical gate error rate is kept significantly below a certain threshold, around 10−3−10−2 (ref. 18). The process of precisely tuning the control parameters of the system to meet this prerequisite is known as calibration19,20. Although traditional calibration techniques have been able to achieve the necessary precision8,10, the analogue nature of the control makes the system performance susceptible to drift. Consequently, the challenge expands from merely reaching deep below the QEC threshold to continuously maintaining this performance amidst the non-stationarity.The solution to the drift problem used in previous experiments5,6,7,8,9,10 is to terminate the entire QEC process for intermittent system recalibration. However, this complete decoupling of computation and calibration represents a fundamental bottleneck for useful future algorithms that require continuous execution times on the order of days or even months3,4. Although theoretical proposals based on logical swaps or code deformation21 have been put forward to reconcile this dichotomy, they entail a large overhead in the circuit runtime, footprint and operational complexity. Yet, the very process of error detection embedded in the QEC algorithm offers a more direct solution, because errors from imperfect calibrations produce detectable syndromes just like all other errors. A pioneering attempt in ref. 22 to leverage this by engineering a direct feedback loop from error detection to physical control relied on a heuristic approach that is difficult to scale. In this work, we propose a different paradigm: repurposing error-detection events as a learning signal for an artificial intelligence agent (Fig. 1).Fig. 1: Overview of RL control.Quantum computation on logically encoded states is physically realized by analogue control signals. The quantum errors are digitized by the QEC process, and error-detection events are used by the decoder to infer logical corrections. In our control framework, they are also repurposed as a learning signal, teaching the RL agent to continuously steer the physical control parameters and stabilize the quantum system during computation.Our framework is based on reinforcement learning (RL), an approach that has demonstrated remarkable success in automating solutions to complex control problems across diverse fields. It has powered systems that achieve superhuman performance in intricate games12,13, enabled substantial advances in robotics14,15 and has recently become integral to refining the behaviour of large language models16. In the field of quantum control, RL was theoretically explored for various problems23,24,25,26,27 and applied experimentally to improve the performance of isolated gates28,29,30 and bosonic codes7. Building on these foundations, here we demonstrate the effectiveness of RL for the system-level challenge of calibrating and steering a large error-corrected quantum processor.We perform our experiments on the distance-5 and -7 surface codes and the distance-5 colour code, focusing on a quantum memory algorithm, in which the logical state is preserved over time by repetitive application of QEC cycles. Our RL agent manages more than 1,000 control parameters, which specify how an abstract QEC circuit is translated into analogue waveforms that control the quantum system. The agent successfully steers the system against injected drift, improving the LER stability 2.4-fold, a figure that increases to 3.5-fold with complementary decoder steering. Moreover, RL fine-tuning of an already well-calibrated processor yields an additional 20% suppression of the LER, pushing performance beyond the limits of traditional physics-based calibration and human expert tuning. The agent is able to reach high performance even starting from randomized initial control parameters, suggesting the potential to augment or replace elements of the traditional calibration stack.Having demonstrated RL control of the present-day state-of-the-art QEC processor, we further showcase the future potential of this framework using numerical simulations. They establish the compatibility of RL control with uninterrupted quantum computation and its scalability to distance-15 surface code with tens of thousands of control parameters. Our approach is fundamentally general, requiring only error-detection signals and tunable controls. Although demonstrated here using planar QEC codes implemented with superconducting circuits, it is directly applicable to any physical qubit modality and QEC architecture, including those with spatially nonlocal connectivity31. Our findings thus firmly establish RL as a promising path towards automating the control of large-scale error-corrected quantum systems.Learning from errorsThe main guiding principle in the construction of fault-tolerant QEC circuits is to ensure that errors leave detectable signatures in the measurement outcomes of the code stabilizers. These signatures are most conveniently represented using the notion of detectors32, defined as sets of measurements that have deterministic total parity in the absence of errors. A detection event, corresponding to flipped parity, signals the occurrence of errors in a certain space–time region of the circuit, called the detecting region33 (Fig. 2a). Detecting regions are usually sensitive to multiple kinds of errors, and a decoder is needed to infer the likely error patterns from any given pattern of detection events34,35. After the correction of the logical state by the decoder, the remaining error is quantified by the LER—the principal measure of quality of the QEC process.Fig. 2: Learning from errors.a, A small space-time chunk of the QEC circuit for the repetition code, highlighting two overlapping detecting regions. b, One iteration of the learning process. In each epoch, a batch of control policy candidates is sampled from the policy distribution. A certain number of QEC cycles is executed with each policy candidate (shades of red and blue). The acquired QEC data are used to compute rewards by estimating error-detection rates for each detector. This information, indicating the relative performance of each policy candidate, is converted by the learning algorithm into a small gradient step for the policy distribution. Then, a new batch of policy candidates is sampled, and the process repeats. c, Finite-difference partial derivatives experimentally evaluated along random directions in the high-dimensional control space confirm the linear relation between the gradient of the true and surrogate objectives, with proportionality coefficient (d + 1)/2 (black line). d, The surrogate objective C allows us to effectively use the sparse dependence of error-detection rates on the system control parameters, represented here as a factor graph. Each detector node is connected to the learnable control parameters of the gates within its detecting region. In our distance-5 surface code experiment, on average, each detector node is connected to 302 parameter nodes, and each parameter node to 18 detector nodes.To execute QEC, the parameters of the quantum gates are stored in the memory of a classical controller (Fig. 1). These parameters are used by the pulse compiler to convert an abstract QEC logical circuit to its physical implementation. In our learning-based approach, this process is modified: we deliberately apply small, simultaneous perturbations to all control parameters during the computation to explore the control space. These perturbations translate into subtle changes in the statistics of error-detection events. The goal of the learning algorithm is to disentangle cause and effect, correlating specific perturbations with changes in performance to continuously calibrate the system.The performance measure of interest in this process is the LER, εL. However, using it directly as an optimization objective faces fundamental obstacles to scalability (Methods). Instead, we construct a surrogate objective C, which is an efficiently computable local proxy for LER that enables high-dimensional optimization. Our surrogate objective is defined as the average rate of error-detection events, \(C=\widehat{{\mathbb{E}}}[D]\), where D denotes all detectors in the circuit, and the empirical expectation value is computed over a sufficient number of QEC cycles. This surrogate objective function is inspired by a simple scaling model of the surface code εL ∝ Λ−d/2, where d is the code distance and Λ = εth /ε quantifies the suppression of the physical error rate ε relative to the QEC threshold εth. As C ∝ ε, the gradients of LER and of the surrogate objective in this model are simply related as \(\nabla \log {\varepsilon }_{{\rm{L}}}=(d+1)/2\cdot \nabla \log C\). To confirm that this relation is approximately satisfied in a real experimental setting, we sample Gaussian perturbations in the control parameter space and evaluate finite-difference partial derivatives (Fig. 2c). We observe good empirical agreement in the regime of small perturbations, which is sufficient for our RL approach that relies on Monte Carlo gradient estimation.To optimize C, we use a multi-objective policy-gradient RL algorithm (Methods). Key to its performance and scalability is an observation that components of C have sparse dependence on the system control parameters owing to the locality of detecting regions in the QEC circuit (Fig. 2d). Our algorithm maintains a parameterized probability distribution over all system control parameters36. Each sample from this distribution, the so-called control policy, is a candidate solution to the optimization problem. As shown in Fig. 2b, the learning proceeds iteratively by repeating the same sequence of steps in every learning epoch. First, a batch of policy candidates is sampled and ranked according to C: policies that lead to a lower rate of error-detection events receive a higher reward. In practice, the error-detection rate is estimated by acquiring data from a finite number of QEC cycles executed with a given control policy and then switching to another policy.Next, the reward information, reflecting the relative performance of all policy candidates, is used by the learning algorithm to compute the gradient of the policy distribution, pushing it towards a more optimal region in the control space. Owing to the limited data rate that currently hampers our ability to learn complex policy distributions such as neural networks, we resort to a simple factorized multivariate Gaussian distribution. Its mean μ(t) represents the best guess of the optimal control policy during the learning epoch t, and the diagonal covariance σ(t)2 controls the degree of exploration of the control space. It evolves during the learning process, typically shrinking over time to finely localize the solution25. Eventually, the distribution converges to a local optimum of the optimization objective, in which detection event probabilities are minimized, indicating that the occurrence of errors is suppressed to its minimum. In a non-stationary setting, the stochastic gradients steer the control parameters to follow the system drift. In this case, μ(t) learns to track the optimal policy over time, whereas σ(t)2 maintains finite spread to never cease exploring.RL fine-tuning of control policyTo demonstrate the practical utility of our RL framework, we first show that it can push system performance beyond the limits achievable by the traditional calibration stack and human expert tuning.The central idea in the traditional approach to gate calibration is to orthogonalize the space of control parameters, using an accurate physical model of the system, and then construct dedicated ‘calibration experiments’ that target a few parameters at a time. For example, a control parameter corresponding to the frequency of a microwave pulse for XY-rotation of a qubit is calibrated by spectroscopy, and then the amplitude of this pulse is calibrated by Rabi oscillations. Control parameters can also be jointly fine-tuned in calibration experiments that measure an aggregate performance metric of the gate, such as randomized or cross-entropy benchmarking37,38. Recently, this approach has culminated in a framework for automated calibration of large quantum systems based on the traversal of a directed acyclic graph20, which encodes the order of calibration of various control parameters, distilled from decades of research in the field of quantum control39. It has been successfully applied in numerous experiments8,11,38,40 reaching the scale of O(102) qubits.Using this traditional approach and extensive human-expert tuning, we calibrate the Willow quantum processor for QEC of the surface code and colour code, as in similar previous experiments8,11,41,42. Subsequent RL fine-tuning of the system consistently yields about 20% additional suppression of the LER, as shown in Fig. 3a for multiple runs on the distance-5 codes of both types. We achieve record QEC performance demonstrated across any physical qubit modality: our distance-7 surface code reaches the LER of εL = 7.72(9) × 10−4 with the AlphaQubit2 neural network decoder43 (averaged over X and Z logical bases), whereas the distance-5 colour code reaches εL = 8.19(14) × 10−3 with the Tesseract most-likely-error decoder35 (Fig. 3b). Our RL fine-tuning technique was also applied in the recent magic-state cultivation experiment44, which combines the elements of both surface code and colour code QEC, in which it provided an order of magnitude improvement in the cultivation error and the post-selection rate. Beyond fine-tuning, in Supplementary Information section IV, we show that RL successfully recovers the performance even after we artificially randomize the control policy to fully scramble the logical observable.Fig. 3: RL fine-tuning of QEC performance.a, Systematic improvement of LER from RL fine-tuning applied after exhaustive conventional calibration process, with five independent runs for surface code and colour code each (grey), mean performance (teal) and 1σ deviation (shaded region). b, Decay of the logical observable in a quantum memory experiment, averaged over X and Z logical bases, featuring enhanced performance from RL fine-tuning. The distance-7 surface code (left) was decoded with AlphaQubit2, and the distance-5 colour code (right) with Tesseract.Generally, as gate error rates are pushed further below the QEC threshold, they will become limited by an increasing number of low-probability error channels. These can be related to the unavoidable simplifying assumptions and approximations in the system models used by the standard calibration approach, or to yet unknown device physics. We anticipate that in the deep below-threshold regime, achieving and maintaining optimal system performance through metrology and targeted calibrations alone will become infeasible, and holistic model-free in-context fine-tuning of the QEC system will become necessary. Our results indicate that RL from error-detection events could fill this important role.RL steering of control policyNext, we demonstrate that RL is not only able to optimize the system performance, but also steer the control policy in the presence of drift. This is achieved by the entropy regularization technique45, which ensures that the policy distribution never becomes deterministic, enabling the agent to continuously explore and adapt to changes in a non-stationary environment. To systematically study this effect, we inject artificial drift on several control parameters simultaneously, as shown in Fig. 4a (red symbols). We choose different temporal drift profiles (step-like, sinusoidal and stroboscopic), different control parameters (CZ coupling strength, XY pulse amplitude and frequency) and different locations of drifting gates on the qubit grid.Fig. 4: Demonstration of RL steering.a, The data qubits (yellow diamonds) and measure qubits (panels with data) are arranged in the layout of a distance-5 surface code. We inject artificial drift with various temporal profiles on the gates indicated with red shapes (circles, diamonds and bars) and observe elevated error-detection signals where expected (coloured background). The detection rate associated with each measure qubit is normalized for visualization to remove the effect of the natural system drift. Performance of the fixed control policy (maroon) degrades over time because of the injected drift, whereas RL steering (blue) stabilizes and maintains the error-detection rate (EDR) below its initial level (white lines). b, Time dependence of the injected drift in system control parameters (dashed) and RL steering (solid). An exponential fit of RL response to step-like drift in XY pulse amplitude yields the characteristic learning time of 130 epochs. c, Periodic evaluation of the logical performance indicates that RL steering of the system significantly suppresses and stabilizes the LER, see main text. Incorporating the decoder steering (black) further improves these results.Following the already-described methodology, we start by calibrating the control policy at t = 0. The performance of this policy (maroon) degrades over time because of the injected drift, as the fixed values of the control parameters become ‘outdated’, leading to additional errors. These control errors lead to an elevated detection event signal, which appears exactly in those detectors in which it is expected based on the constructed factor graph, see tiles highlighted with coloured background in Fig. 4a. In contrast to a fixed control policy, RL steering (blue) maintains a significantly suppressed rate of error-detection events that consistently remains below the initial level (white line), except for brief moments when the drift is too fast. In Fig. 4b, we show the evolution of the learned control parameters. The recovery from a step-like drift in the XY pulse amplitude (red circle) allows us to estimate the response time of the steering process of about 130 epochs. This also sets the time scale for the policy lag in the case of slow continuous drift, as in XY pulse frequency (red diamond).To confirm that suppression of detection events is not due to hindered detection capability but is due to suppressed errors, we evaluate the logical performance in Fig. 4c. Compared with the fixed policy, we find on average a 24% reduction of LER and a 2.4× improvement of its stability (quantified here by the standard deviation of the LER distribution). These figures of merit further improve to 31% and 3.5×, respectively, by additionally steering the decoder parameters. Decoder steering is achieved within the same RL framework by reweighting the matching graph as described in ref. 46. Although the steering of the classical controller is done via the surrogate objective C that relies exclusively on the error-detection probabilities, our decoder steering process relies on LER estimation, which is not straightforwardly scalable to the real-time setting. However, alternative approaches to decoder steering have been proposed47,48,49 that, in principle, do not suffer from this limitation.We also analysed the RL performance under natural system drift, which arises from sources ranging from material defects near the quantum system50 to temperature fluctuations in the classical control instruments and, unlike our previous demonstration, rarely has a simple time dependence. Fourier analysis of multiple experimental RL runs from Fig. 3a shows that the effect of steering can be understood as a filter that provides about 4 dB of suppression of low-frequency LER fluctuations originating from these natural sources (Supplementary Information section III).We have thus far demonstrated that the mean μ(t) of the Gaussian policy distribution learns to track the optimal policy over time in the presence of drift. As a result, the learned policy μ(t) outperforms the fixed policy, with lower EDR and LER. However, as the learning process requires exploration of the parameter space, the algorithm inevitably samples policy candidates whose performance is worse than that of the μ(t) policy. This ‘exploration noise’ is irrelevant in our experimental setting in which RL steering relies on repeated executions of a short logical algorithm and the quantum state is independently re-prepared in every shot. However, in the future, this steering must be done in real time during the single-shot execution of a long logical algorithm. In that case, the exploration noise, although necessary for learning, will be detrimental to the performance of the logical algorithm. Generally, balancing the exploration of parameter space and the exploitation of the learned policy μ(t) is a central challenge in many applications of RL51. In our case, the favourable resolution of the exploration–exploitation trade-off would mean that the aggregate performance of all sampled policy candidates, most of which are worse than μ(t), is still better than the performance without RL steering.To study this trade-off, we conducted numerical simulations of real-time steering of the distance-3 surface code subject to sinusoidal parameter drift at different frequencies (Fig. 5a and Supplementary Information section VIA). We count the total number of error-detection events generated in a 1.8 × 109-cycle window of QEC, and normalize it so that level 1 corresponds to the performance of the optimal policy (known in the simulation), and level 0 corresponds to the performance of a fixed policy (optimal at t = 0). We control the exploration–exploitation trade-off by changing the amount of entropy regularization45 in our RL algorithm. Our findings indicate that there is a critical drift frequency, about 1/150 epochs, below which the system becomes real-time steerable: the performance with exploration noise is better than the performance of a fixed policy. This critical frequency is consistent with the response time of the learning algorithm observed in the experiment (Fig. 4b). When drift is too fast, the real-time steering is not able to keep up—this drift must be mitigated at the hardware level. In particular, correlated drift caused by rare high-energy particle impacts in superconducting devices8,41 typically equilibrates on a much shorter time scale and is not real-time steerable in our current implementation. By contrast, at low drift frequencies, the exploration and exploitation can be successfully balanced to closely approach the performance of the optimal policy at all times.Fig. 5: Real-time steering and scaling simulations.a, Normalized improvement (colour) of error-detection rate in the real-time steering simulation of the distance-3 surface code subject to sinusoidal drift at different frequencies. Level 1 indicates the performance of the optimal policy. Isoline at level 0 demarcates the boundary beyond which real-time steering results in better performance than a fixed policy, approaching the performance of an optimal policy in the regime of slow drift. b, Simulation of scalability of RL control of large surface codes. The algorithm learns the parameters of single-qubit and CZ gates, with 30 control parameters per gate, amounting to almost 40,000 control parameters in total for the distance-15 code. During the learning process, the LER reduces over time (colour) until it reaches the floor (red bars) set by the irreducible physical error rates and characterized by the optimal error suppression factor Λ*. c, Point estimates of Λ at every code distance and learning epoch from b confirm that the speed, ∂t Λ/Λ* × 102, at which the error suppression factor approaches the local optimum, is proportional to the distance from the optimum, 1 − Λ/Λ*. The convergence rate γ (see main text) is independent of the system size but depends on the number of control parameters per gate, with three beams corresponding to 1, 10 and 30 parameters, and the linear fits (red) indicating the convergence rates.Thus, our simulations establish that RL is able to effectively use the information concealed in the error-detection events to calibrate the system while continuing the logical computation. This ability offers a substantial advantage over approaches based on the synthesis of traditional calibration and code deformation21, because RL steering does not introduce any resource overhead.Scaling and outlookFinally, we demonstrate the scalability of RL control with simulations extending to a distance-15 surface code with approximately 40,000 control parameters (Fig. 5b). We extract point estimates of the instantaneous error suppression factor Λ at each code distance and learning epoch, verifying in Fig. 5c that its convergence rate ∂t Λ/Λ* is proportional to the distance from the optimal Λ*, set in our simulation by the irreducible control-independent physical error rates. This empirical result leads to the exponential convergence law, 1 − Λ/Λ* ∝ e−γ t, characteristic of gradient-descent-based algorithms in the vicinity of a local optimum52, with the convergence rate γ that depends on properties such as the number of gates per detecting region and the number of control parameters per gate. Crucially, this rate is independent of the system size, a direct consequence of the ability of our algorithm to effectively use the sparsity of the factor-graph representation of the control problem.Multiple possible extensions of our RL control framework leave ample room to further enhance its abilities. For example, instead of relying on a simple Gaussian policy distribution, it could be advantageous to use deep neural networks13 and condition the policy distribution on the observations from the environment, for example, on some suitable statistics of the error-detection events. Learning the system model in addition to the policy53 can lead to better sample efficiency and further improve the quality of solutions by discovering unexpected correlations in the data. It is conceivable that in the future, with sufficient enhancement of our learning framework, a quantum processor could be calibrated for QEC ab initio fully by RL, with no reliance on the traditional calibration paradigm or human experts.In conclusion, by empowering the quantum computer to learn from its errors, we unlock a scalable pathway to optimize performance in real time, replacing disruptive calibration routines with uninterrupted computation. Our work suggests that the path to fault tolerance will be built not only on better hardware but on more intelligent control.MethodsObjective functionUsing LER as an optimization objective faces fundamental obstacles to scalability, particularly in multi-qubit stabilizer codes such as the surface code, for the following reasons: (1) The LER is suppressed with the code distance d as εL ∝ Λ−d/2, where the error suppression factor Λ > 2 has already been demonstrated in experiments with superconducting circuits8 and neutral atoms10. Thus, accurately resolving the LER would require an exponentially large number of QEC cycles. (2) The optimization involves a vast number of parameters—already more than 2,000 in our d = 7 experiment, and scaling as O(d2)—which renders global optimization from a single scalar metric impractical. (3) The LER is unsuitable for real-time calibration and steering during a quantum computation, as the logical state is generally unknown. Although direct LER optimization has proven effective in specific contexts, such as for small bosonic codes7, these scaling challenges necessitate a different approach for large multi-qubit codes.The surrogate objective C, defined in the main text, immediately alleviates the third limitation. It also alleviates the first limitation because resolving C to a constant relative accuracy requires accumulation of data from a number of QEC cycles that scale as O(ε−1) and is independent of the code distance. To alleviate the second limitation and enable efficient high-dimensional optimization, we harness the sparse structure of the surrogate objective. We rely on the locality of detectors in the QEC circuit to narrow down the dependence of each term in C to only a small subset of control parameters—those related to the gates within the respective detecting region. These relations can be conveniently represented using a factor graph, a bipartite graph in which the factor nodes correspond to detectors, and the variable nodes correspond to the system control parameters (Fig. 2d and Supplementary Information section II for more information).Learning algorithmDirectly optimizing the surrogate objective C is intractable for standard off-the-shelf optimizers. Although some algorithms54 are compatible with large-scale nonlinear optimization problems represented as sparse factor graphs, in practice, additional considerations play an important part. Specifically, our objective function C is highly stochastic, as it is obtained by averaging the binary detector values from a finite (and preferably small) number of QEC cycles. Furthermore, C is non-stationary due to inherent system drift. Furthermore, evaluating C is expensive because it requires updating the classical controller and acquiring data from the QEC detectors. Finally, in the future, C must be optimized in real time during the quantum computation, favouring smooth and gradual improvements over disruptive large parameter steps that might lead to catastrophic consequences for the logical algorithm.Given these properties, we adopted a multi-objective, policy-gradient RL approach, in which signals from QEC detectors comprising C serve as multiple objectives optimized simultaneously. Our learning algorithm is built on parameter-exploring policy gradients36, an approach in which an entire control policy is sampled in one piece, which provides a convenient interface with the classical controller. For robustness in the stochastic and non-stationary setting, we integrate several key techniques: proximal policy optimization55 for stability, entropy regularization45 to maintain exploration and a replay buffer13 for improved sample efficiency. Crucially, the algorithm harnesses the sparse structure of the factor graph representation of C for efficient variance reduction of the Monte Carlo gradient estimator56 by gradient masking. The latter technique is adopted from appendix G of ref. 46, in which it was derived from the point of view of multi-agent learning. For completeness, a detailed exposition of our RL algorithm is provided in Supplementary Information section VIII.