Most lists of the disadvantages of reinforcement learning were written by people who have never let a policy touch an actuator. They name the right words, sample inefficiency, reward design, compute cost, interpretability, and stop there. On a live physical or biological process each of those words turns into a specific failure with a specific cost, and several of the failures that matter most do not appear on the lists at all.
This article sorts the disadvantages by where they bite once the controller is connected to something real, puts a measured number on each one, and ends with what we do about them in our own control stack.
The short list, and why it is incomplete
The standard answer, as in this AIML summary table, is six items: RL needs large amounts of data, training is unstable and sensitive to hyperparameters, reward design is prone to unintended behaviour, exploration can be costly or unsafe, compute and time costs are high, and the policy is hard to interpret.
All six are true. What the list leaves out is the context that decides whether they are inconveniences or deal breakers. A game engine can be reset a billion times. A 500 L culture basin, a reactor or a kiln cannot. Google and DeepMind researchers put it plainly in their study of real-world RL challenges: real systems do not have separate training and evaluation environments, so the agent must learn to act reasonably and safely from the start, and every exploratory action has consequences.
1. It needs more data than a plant can give
What the benchmarks actually consumed
The numbers behind famous RL results are large. The Rainbow DQN paper needed 7 million frames, under 10 hours on one GPU, just to match the final performance of the original DQN, and a full run of 200 million frames took about 10 days. A 2026 tutorial survey on statistical limits in real-world RL puts typical deep RL consumption at 10^6 to 10^9 transitions, orders of magnitude above human experience.
Why a live process cannot pay that bill
A simulator produces a transition in microseconds. A bioreactor or a thermochemical unit produces one per control step, and a slow process may take minutes per step. The process-control review by Nian, Liu and Huang notes that for complex systems, training may need hundreds of thousands of steps to reach even a near-optimal policy, which makes training in live operation infeasible. Dulac-Arnold and co-authors add that because exploration on a real system is constrained, the training data is low variance and covers very little of the state and action space. You cannot learn how the process behaves near a limit if the operators have spent twenty years keeping it away from that limit.
This is the disadvantage that forces the next one.
2. The simulator becomes the thing it learns to beat
Since the plant cannot supply the data, you train in a model of the plant. The agent then optimises against the model, errors included. A practitioner on AI Stack Exchange names the real danger: the agent finds and exploits the mismatch between the learned simulator and the actual process. A policy that looks excellent in simulation may be excellent only at the simulator's mistakes.
The failure is not hypothetical. In a NeurIPS 2024 sim-to-real study on a robot puck-pushing task, standard direct transfer of the simulator-trained policy solved the real task in 0 of 6 trials, even with fine-tuning. Transferring exploration policies instead solved it 6 of 6. The simulator was useful; trusting its optimal policy was not.
Two further points from the literature narrow the case. Nian et al. argue that an RL agent trained on a simulator should not be expected to outperform an MPC designed on the same simulator model if the MPC reaches its global optimum. And a 2024 review of RL in manufacturing found that while most studies train in simulation, only a minority validate in real or physical settings, and those mostly without a strategy for the gap. If you are building the simulator anyway, read our notes on sim-to-real transfer and domain randomization before you trust it.
3. It optimises the reward you wrote, not the one you meant
Every listicle mentions reward design. Few say how structural the problem is. The survey of open problems in RL from human feedback collects three findings that apply to any RL controller: unhackable reward proxies are very rare in complex environments, reward hacking should be expected by default under mild conditions, and it becomes more likely as the agent gets more capable. A better algorithm makes the problem worse, not better.
The textbook case, retold in Daniel Paleka's notes on how RL fails, is a boat-racing agent that learned to spin in circles collecting points instead of finishing the race. On a plant the equivalent is quieter. Reward a fermentation for biomass growth and the agent may push temperature into a band that raises growth today and stresses the culture tomorrow. Reward setpoint tracking and it may chatter a valve until it wears out.
Real processes also have several objectives at once. Dulac-Arnold and co-authors showed that as a safety term gains weight in the reward, constraints are respected quickly, but over-emphasis degrades the base task. There is no single weighting that is right; there is a trade-off someone has to own.
4. Constraints it has never seen violated
A plant is defined by what must never happen: a pressure ceiling, a dissolved-oxygen floor, a temperature at which a culture dies. Reinforcement learning handles these badly for two reasons.
First, the data does not contain them. The same Google and DeepMind paper points out that safety violations are very rare in the logs of a real system, and that safety constraints are often assumed rather than specified by the operator. An agent cannot learn to avoid an event it has almost never observed.
Second, the math does not enforce them. A 2025 review of RL for chemical process control notes that input limits are easy to encode in the action space, but path constraints on state variables are not trivial to enforce, and that the black-box nature of the policy hinders practical implementation. An earlier 2022 process-control review says RL still depends on the evolution of theory to incorporate process constraints and guarantee stability.
A penalty in the reward says "please avoid this". A plant needs "this cannot happen". Closing that gap is the job of a separate layer, a control barrier function safety filter or shielding, not of the policy.
5. The plant keeps changing after commissioning
Catalysts deactivate, heat exchangers foul, feedstock varies by season, a microbial strain adapts. The tutorial survey above calls this nonstationarity and identifies two sources: dynamics that shift on their own, and the agent's own improving policy shifting the data it sees. It concludes there is no single algorithmic fix; abrupt shifts, slow drift and agent-induced shift need different mechanisms.
Industrial processes add partial observability. Nian et al. note that many plant states are not fully measurable, which breaks the Markov assumption RL is built on and leads to sub-optimal decisions. And when a policy does degrade, the 2022 review describes the state of the art for maintaining and retraining RL on a running process line as still embryonic. A PID loop drifts gracefully and an operator can retune it in an afternoon. A drifted neural policy needs a data pipeline, a retraining campaign and a new validation.
6. You cannot hand an auditor a neural network
When a controller acts on a live process, someone has to sign that it is safe. For a PID or an MPC that argument is built from gains, models and constraint sets an engineer can read. For a deep policy it has to be built from evidence: simulation campaigns, formal guarantees on a wrapper, hardware-in-the-loop tests. That is a real cost that listicles file under "interpretability" and move past.
It also drags in infrastructure. In one plant-wide PID tuning study cited in the 2025 review, the DDPG agent needed so much real-time operating data that a high-fidelity digital twin had to be built before online deployment. The controller was the small part of the project.
What we do about them
We build controllers for living and thermochemical processes, so these are the constraints we design against rather than a list we read about. The figures below are what we measured on our own control stack, on a 500 L pilot basin:
- Data cost and simulator exploitation. The policy trains against a continuous-time growth model, with accuracy above R-squared 0.95, and the controller was validated across 400 simulated years. The model is derived from the process equations first, which narrows the mismatch the agent can exploit.
- Constraints. A separate safety layer sits between the policy and the actuators and runs in under 2 ms. Validation recorded zero safety violations across those 400 simulated years.
- Latency. End-to-end inference runs at the edge in 285 ms, down from 1.2 s, so the loop does not depend on a cloud round trip.
None of this makes the disadvantages disappear. It moves them into places where they can be measured and argued about, which is what certification needs.
When reinforcement learning is still the right call
The disadvantages are real, but so is the upside. The 2025 review reports a batch bioprocess policy that took 3 hours to train offline on a CPU and then needed only 0.002 s per decision online, cheap enough for fast loops where MPC's online optimisation is too slow. The tutorial survey cites RL cutting data-centre cooling energy by roughly 40%. RL earns its place where the dynamics are nonlinear, a good model exists for training, the online compute budget is tight, and a safety layer can hold the hard limits.
Where those conditions do not hold, the incumbent usually wins. We compared the options head to head in reinforcement learning vs PID control and reinforcement learning vs MPC for process control.
FAQ
What is the biggest disadvantage of reinforcement learning?
Sample inefficiency. Deep RL typically needs millions of interactions to learn. On a physical plant it compounds with safety, because every exploratory action happens on the real process and has consequences.
Why is reinforcement learning rarely used in industry?
Most published industrial RL work is trained in simulation and only a minority is validated on physical equipment. Safety constraints that cannot be guaranteed, black-box policies that are hard to certify, and the cost of building a trustworthy simulator keep adoption at an early stage.
Is reinforcement learning better than MPC?
Not on raw performance if both are built on the same model: an MPC that reaches its global optimum should match or beat the RL policy. RL's advantage is that the heavy computation happens offline, so each online decision is cheap.
What is reward hacking in reinforcement learning?
It is the agent earning a high reward without doing the task you intended, by exploiting a gap between the reward you wrote and the goal you meant. Research suggests it should be expected by default, and becomes more likely as agents grow more capable.