Search safe reinforcement learning for industrial process control and you get a reading list. Conference abstracts, arXiv preprints, a funded German research project, two literature reviews. Every one of them proposes a method and most of them work. What none of them supplies is the artefact an operator actually needs, which is the record that lets somebody sign off on a learned policy moving a real valve.
That is not a small gap. A PRISMA systematic review of reinforcement learning in the process industries reaches the same conclusion from the other direction: methodological maturity is improving, algorithm selection is increasingly tailored to the problem, and the majority of applications remain confined to simulated environments. Digital-twin-based approaches account for 6.4% of the work surveyed. The review's own list of priority actions puts controlled pilot deployment on physical systems first, ahead of any algorithmic wish.
The blocker is not the algorithm. This article sets out what has to be demonstrated, in the order a project meets it.
Three ways to make reinforcement learning safe, and they are not interchangeable
The clearest taxonomy comes from Bettina Koenighofer's account of shielding, published in Communications of the ACM in October 2025. Safe RL splits three ways: shape the reward so unsafe actions are penalised, add a second cost function so safety is constrained rather than merely discouraged, or block unsafe actions outright at runtime.
Only the third gives you a guarantee, and the reason matters. Penalising unsafe actions during learning changes what the agent prefers, not what it can do. Even during exploitation, when the agent is choosing the best action it knows, nothing rules out an unsafe one.
A shield sits between the agent and the plant. Post-shielding watches the action the agent selected, classifies it, and replaces it if unsafe. Pre-shielding hands the agent the list of permitted actions and lets it choose from inside that list. Post-shielding is easier to retrofit, which is usually decisive on an existing plant, but it has a subtle failure mode worth knowing before you build one: overwriting an action breaks the link between the action and the reward the agent receives. The fix is small and specific, tie the reward to the action the shield actually executed rather than to the one the agent proposed.
The control-theoretic instance of this family is the barrier function. The AAAI paper that put barrier functions into end-to-end safe RL opens with the sentence the rest of the field keeps rediscovering: RL algorithms have found limited success beyond simulated applications, and real world systems would realistically fail or break before an optimal controller can be learned. Its architecture combines a model-free RL controller, model-based controllers using control barrier functions, and online Gaussian process learning of the unknown dynamics. The guarantee holds with high probability throughout learning, and it holds regardless of which RL algorithm you put inside it. The demonstrations were an inverted pendulum and autonomous car-following, which is worth remembering before assuming the result transfers to a reactor.
Proof obligation one: the constraint has to exist in physical units before the agent does
A shield is computed from a formal specification of the safety-critical property plus an abstract model of the environment. That specification is independent of the learning objective. If your safety requirement cannot be written down without reference to the reward, there is nothing to compute, and what you have is reward shaping wearing a safety layer's clothes.
In practice this is the step that exposes how much of a plant's safety knowledge lives in operator habit rather than in a document. The DFG project on safe RL for chemical processes at TU Dortmund, running 2022 to 2025 inside SPP 2331, describes the status quo plainly: complex operations are performed by following recipes and manual adaptations by operators, which leads to suboptimal operation and severely limits flexibility, because start-up and shut-down procedures get avoided for lack of systematic strategies to handle them. The knowledge exists. It is not in a form a shield can be synthesised from.
There is a softer version of the same obligation, and it is the more useful one when the model is uncertain. Work on safe chance constrained RL for batch process control uses a Gaussian process state-space model and takes its posterior predictive variance in two directions at once: to tighten the constraint with backoffs, holding the nominal trajectory away from the boundary by an amount that scales with how uncertain the model is there, and to penalise exploration of regions where epistemic uncertainty is high. The constraint stays hard. The distance you keep from it becomes a function of how much you know.
Proof obligation two: the safety layer has to answer inside the control period
A guarantee that arrives after the control step has passed is not a guarantee. The safety check runs on every step, so its latency budget is set by the process dynamics, not by the model's convenience, and this is where a great deal of published work quietly stops being deployable.
The figures from our own control stack, published on our research page, are these. The safety layer answers in under 2 ms. End-to-end edge latency is 285 ms, reduced from 1.2 seconds. Growth-model accuracy is R-squared above 0.95, validated on a 500 L pilot basin with zero safety violations across 400 simulated years.
The accuracy number is the modelling result. The latency numbers are what made it a controller, and they came from a separate discipline: compressing the model to fit a latency budget, and choosing deliberately between distillation and quantization rather than applying both by reflex. Note the asymmetry between the two figures. The safety layer is two orders of magnitude faster than the policy it guards, which is the correct arrangement, because the shield has to be able to overrule a late or bad action rather than queue behind it.
Proof obligation three: the gap between the simulator and the plant has to be measured
Offline training moved the problem rather than solving it
Training in simulation and deploying on the plant runs into the sim-to-real gap. Training on historic plant data avoids it and buys a different problem.
The case for going offline is made bluntly in work on safe deployment of offline RL via input convex action correction: high-fidelity dynamic simulation models are expensive to develop and maintain, will not be available for most processes, and are slow enough to drive up the compute cost of training a policy. The same paper is equally blunt about what offline training does not fix. Deployment of RL policies in high-stakes settings remains rare, because a poorly chosen control action can damage equipment, violate regulatory constraints or cause cascading system instability, and an agent optimised on historical performance produces exactly that when it reaches a state its training data underrepresents. Their benchmark, an exothermic polymerisation reactor across startup and grade-change scenarios, showed behaviour cloning and implicit Q-learning suffering steady-state offsets and degraded performance near setpoints. Both are unremarkable in a research plot and unacceptable in a product spec.
A middle route has industrial appeal because it starts from something the plant already owns. Pre-training the agent from an existing linear MPC formulation gives it a competent starting policy, after which restricted exploration on the real plant is defensible in a way random exploration never is. The 2025 AIChE work applied this to a polymerisation reactor and to a C2 splitter distillation column with output and system safety constraints incorporated.
Plant-model mismatch is a quantity, not a caveat
Whichever route you take, the mismatch has to be a number in the validation record. The chance-constrained approach above makes it one directly, satisfying joint chance constraints under mismatch and benchmarking against nonlinear MPC rather than against an unconstrained baseline.
The real-plant version of this is less tidy, and the most honest catalogue of it comes from a team that actually deployed. Reporting on the challenges of controlling industrial energy systems with RL, Lademann and colleagues describe a system state that is only partially observable, where the state of charge of the thermal storage cannot be accurately determined because of strong temperature stratification and too few temperature sensors. The action space is constrained by what the existing building automation interface will accept. And the plant does not hold still: gradual efficiency degradation and outright modifications force the simulation model to be re-adapted and the agent retrained, hyperparameter search included. That last point is the one that turns a deployment into a maintenance commitment, and it belongs in the proposal rather than in the first post-handover invoice. The measurement side of this problem, where instruments run on different cadences and drop out, is a modelling decision in its own right, and continuous-time models of an irregularly sampled process exist precisely for it.
Between the fitted model and the trusted controller sits sim-to-real transfer, and skipping it is how a model that validated cleanly meets sensor noise and actuator lag for the first time in production.
Proof obligation four: there has to be a defined fallback when no safe action exists
This is the least discussed point in the literature and the first one a plant engineer asks about.
It is formally not possible to compute a shield if no safe action exists. Koenighofer states it and states the practical response, which is to lower the safety requirements if you can. There is a second version of the same problem at runtime: shields are computed over a winning region, and if the system enters a state outside that region during operation, a fallback strategy has to have been defined in advance. A predefined action, or the least-bad available one.
The control-theoretic literature reaches the same structure independently. Control invariant set enhanced RL folds the invariant set into reward design, initial state sampling and state reset during offline training, then uses membership of that set as the online stability criterion, retraining whenever the state falls outside it. Because online search can fail to find a safe action within a bounded number of iterations, the method carries an explicit backup table of safe actions for exactly those states. Applied to a nonlinear CSTR, it produced a lower failure rate and better sampling efficiency than RL without it.
Read that as an engineering instruction rather than a paper. Every safe RL architecture that survives contact with a plant has a defined behaviour for the case where its guarantee does not apply, and that behaviour is an interlock decision, not a machine learning one.
Proof obligation five: the record has to survive somebody else's audit
Sample efficiency is a safety argument, because every episode on real equipment costs product, time or risk. It is also more attainable than the field's reputation suggests when uncertainty is modelled explicitly rather than averaged away. Using Gaussian processes as the function approximator for a non-isothermal semi-batch reactor, analytical uncertainty from the GP predictions drives the exploration and exploitation trade-off and lets probabilistic constraint violation be modelled directly. The result was a safe, effective policy from 100 evaluations of process trajectory, with no prior knowledge of the process dynamics, where a neural network approach would have needed substantially more.
Then there is the documentation, which is where funded programmes and their reporting obligations bite. On our OptiVX programme the record ran to 29 accounted work phases and 400 simulated years certified. On IntelliBot it included more than 400 pages of transfer documentation. That volume is not ceremony. It is what makes the validation reproducible by somebody who was not in the room, and it is a defining feature of how a contract research programme differs from a proof of concept.
When the process is alive rather than chemical
A cultivation basin is slower than a reactor and less forgiving. Slower, because the dynamics run on hours and days rather than seconds. Less forgiving, because the failure is not off-specification material that gets reprocessed. A crashed culture does not restart on the next batch, and recovery is measured on the same timescale as the growth it interrupted.
Observability is worse too. Concentration, health and composition are inferred from optical and spectral proxies rather than measured directly, so the state estimate carries uncertainty that a temperature reading does not. That is why the growth model, the safety layer and the latency budget are three separate deliverables in our own stack rather than one system judged on a single accuracy figure.
Two things about reinforcement learning in this setting are easy to lose. It is derived from optimal control theory, not opposed to it, and its early results were inferior to a PID controller and harder to run. Its real advantage arrives after training, as adaptive control at low computational cost, and that advantage is conditional on the theory carrying process constraints and stability rather than assuming them.
Derive the equations first. Then decide what is allowed to act on them.
FAQ
Is safe RL better than model predictive control?
Usually the wrong comparison. The pattern that recurs across the process-control literature is hybrid: MPC contributes constraint handling and often the pre-trained starting policy, RL contributes adaptation and cheap evaluation once trained. Industries under strong regulatory constraint favour exactly these hybrid architectures with model-based safety layers.
Which safe RL method should I start with on an existing plant?
Start from what the plant already has. If a linear MPC is running, pre-training against it costs least and avoids random exploration entirely. If the safety property can be specified formally and the state space is tractable, a shield gives a stronger guarantee. If the dynamics are unknown but smooth, the barrier-function-plus-Gaussian-process route learns them online while staying safe.
Does a shield slow down learning?
It can, in both directions. Shields often speed learning by keeping the agent in productive regions, but a restrictive shield hinders the exploration that helps an agent generalise, and post-shielding in particular can degrade learning performance if the reward is still attributed to the overwritten action.
How large a system can a shield actually cover?
State-of-the-art shielding approaches handle environments with billions of states, which sounds ample until you note that real-world applications may demand far more. Shielding inherits the scalability limits of model-based verification, so the practical question is whether your safety property can be expressed over an abstraction small enough to check.
Who signs off on a controller that acts on a live process?
Not the modelling team alone. The fallback behaviour is an interlock decision, the latency budget is a process decision, and the validation record has to be legible to whoever carries the operational risk. Building that record is the work, and it starts before the first policy is trained.