Functional Safety and Machine Learning: Who Holds the SIL

Functional safety and machine learning are usually written up as a conflict, and the conflict is real. But it is narrower than the word "conflict" suggests. Functional safety is the part of overall safety that depends on electrical, electronic and programmable electronic systems operating correctly, and the standards that govern it do not ask how a function was produced. They ask whether its worst case is known. IEC 61508 and ISO 13849 certify a function only when its worst-case behaviour is bounded, repeatable and provable.

That reframes the question. It is not whether a learned controller is permitted near a safety function. It is where the bound lives, which element of the architecture carries the safety integrity level, and what the learned element then owes the safety case. The standards now say something specific about all three, and this article walks through it, ending with a validation campaign that has actually been run on a learned controller acting on a living process.

What a SIL actually is, and which column applies to a controller

A safety integrity level is a numerical target on a safety function, and the number depends on how often the function is called. IEC 61508-1 defines low demand mode as no more than one demand per year and uses the average probability of dangerous failure on demand, PFDavg, from its Table 2. Above one demand a year, or when the function runs continuously, Table 3 applies and the target becomes the average frequency of dangerous failure per hour, PFH.

Each SIL is one decade wide, and Part 1 sets a floor on what a single system may claim: a PFH of 10^-9 per hour in continuous mode. Two things follow for a learned controller. First, a controller that computes an action every cycle is in the continuous column, not the on-demand column, which is the column the consultancy guides tend to quote. Second, the claim is a frequency of dangerous failure. A model accuracy, an R-squared, or a held-out test score is not that quantity and cannot be converted into it without an argument about every input the process can present.

Why a learned model cannot produce that number on its own

Six tensions, one root cause

The clearest current inventory of the problem lists six tensions: non-determinism against the assumption that the same inputs give the same outputs, opacity against traceability, distribution shift against a defined operating condition set, statistical validation against structural test coverage, model updates against configuration management, and inference latency against the margin budget. Functional safety standards were written for software that can be tested against structural coverage criteria such as MC/DC, and neural networks cannot be verified this way because the specification is implicit in the training data.

All six reduce to one root. There is no a priori specification to verify against. ISO/IEC TR 5469 says so in its own terms, listing under verification and validation problems the non-existence of an a priori specification, non-separability of particular system behaviour, limitation of test coverage, non-predictable nature, and drift. Its introduction adds the point that matters most for a model that keeps learning: continuous improvement undermines verification and validation, because the function moves progressively away from the rigorously tested, ideally deterministic behaviour.

Explanation is not assurance

The reflex response is explainability. The ZHAW deck disposes of it in four lines: LIME and SHAP attribute a single prediction to input features, Grad-CAM highlights the image region behind a call, surrogate trees give rules an operator can read, and none of it is assurance. The methods only approximate the model, they disagree with each other, and they run after the fact, so they are never a safety function. Explainability earns its place in engineering review. It does not earn a decade on the PFH table.

What ISO/IEC TR 5469 changed in 2024

Until 2024 the honest answer to "what does the standard say about machine learning" was "nothing, and in places, no". TR 5469 says this itself: the use of AI technology is currently not treated in mature functional safety standards, and in some of them its use is explicitly forbidden.

The report is ISO/IEC TR 5469:2024, Artificial intelligence, Functional safety and AI systems, first edition January 2024, prepared by ISO/IEC JTC 1/SC 42. It was not written in isolation from the parent standard. The German standards commission KAN records that it was developed by SC 42 working group 3 together with experts from IEC SC65A, the committee responsible for IEC 61508, with the stated intention that it serve as the basis for later normative documents.

Its scope is three uses, and the second one is the one most practitioners miss: AI inside a safety-related function; non-AI safety-related functions that ensure safety for AI-controlled equipment; and AI used to design and develop safety functions. The structure then does four things that matter:

• Clause 6 classes AI technology by whether it can show compliance with existing functional safety standards, and introduces usage levels by how much the AI element influences the safety function. KAN's critique is fair: the class is defined by standards compliance, not by any feature of the technology, so the resulting risk matrix is coarse.
• Clause 7 gives a three-stage realization principle for the case where compliance cannot be shown directly, with acceptance criteria derived per stage.
• Clause 10.2 sets out the architectural measures: detection mechanisms for switching, a supervision function with constraints that holds behaviour within safe limits, redundancy and diversity, and statistical evaluation.
• Annex A applies IEC 61508-3, the software part, to AI technology elements, and Annex D maps the 61508-3 safety life cycle onto the ISO/IEC 5338 AI life cycle.

Nothing in it awards a SIL to a network. What it does is tell you where to put the network so that something else can hold the SIL.

The three architectures the standard will accept

The report's convenor set out the architectural patterns in order of preference, and the order is the practical content of the whole document.

Back-up: a non-AI function that can run the process

The first choice is a safe, possibly sub-optimal back-up function designed with non-AI techniques, switched in when detection methods flag an unsafe condition. The back-up may be a fail-safe null action, but for a continuous process it usually cannot be. The ZHAW deck puts the design constraint plainly: the fallback must run the process, not just stop it. A bioreactor or a gasifier that is "safely" halted is a batch lost and sometimes a bed clinkered.

Supervision: a safe subset of the action space

The second choice is a supervisor function with constraints that determines, a priori or online, a safe subset of the action space, and clips the learned output into it. This is the ZHAW simplex enclave: the AI sets the setpoint, a deterministic controller checks it against physical limits, rate limits and interlocks. It is also, in control-theoretic dress, a control barrier function safety filter, where the supervisor is a small quadratic program that minimally modifies the proposed action. The same structure appears in reinforcement learning as a shield, which we have written up separately under safe reinforcement learning via shielding.

Redundancy and diversity: the last resort

Only when neither a safe subset nor a back-up exists does the report fall back on ML redundancy with output voters or aggregators and a safety reserve. Redundancy is then combined with diversity to reduce the chance of a shared systematic failure: different teams, separate labelling rules, different problem formulations, different training data, diverse hardware, diverse sensing, diverse self-check methods, and diverse AI technologies. That is eight independent axes of variation to argue, which is why this is the last resort and not the first.

The ZHAW deck compresses the whole section into one rule: no neural network drives an actuator without a deterministic layer between them.

Who holds the SIL, and what the learned part must still prove

The monitor carries the integrity claim

Once the architecture is one of the three above, the SIL question has an answer. The learned component is advisory, the deterministic monitor is authoritative, and the monitor must itself be certified to the required SIL. The safety function is decomposed, the ML element is assigned the lower level, and the difference is made up by the monitor. That decomposition is a documented artefact, reviewed like any other.

Two consequences are easy to underestimate. The monitor's own PFH has to land in the band, so the monitor is simple by necessity: a barrier condition, a clip, an interlock, a certified PID. And the monitor has to fit inside the control deadline with margin, which for hard real-time industrial loops the Hyperion guide places at one to ten milliseconds. A supervisor that needs a cloud round trip is not a supervisor.

The model carries an evidence record

Moving the SIL off the model does not move the evidence off it. The model still owes the safety case five things, and TR 5469 clause 9.3 and the consultancy literature agree on the list:

1. An operational design domain, stated as input ranges, environmental conditions and process envelopes. The ODD is the contract; any input outside it must trigger the safe state rather than a silent inference.
2. Out-of-distribution detection that is itself validated for its false-negative rate, because an OOD detector that misses is a monitor that never fires.
3. A fallback whose transition is assessed, not merely present. Entering the safe state is a safety function too.
4. A statistical validation set whose size, composition, metrics and confidence intervals are documented and accepted, because this is what replaces structural coverage. TR 5469 ties the data distribution back to the hazard analysis and asks for system-level testing plus monitoring with incident feedback after deployment.
5. Change management. The model that was validated is the model that ships. A retrained model is a new release and reopens the case.

The formal side of the field has arrived at the same shape from the other direction. Learning-based control now routinely learns a policy together with a certificate function, a barrier for safety or a Lyapunov function for stability, and uses runtime monitoring to repair the certificate in the black-box setting. Certificate plus monitor is supervisor plus evidence record under another name. The packaging, rollout and rollback half of the same discipline is covered in our edge AI deployment pipeline article.

What a campaign that has been run looks like

Everything above is what the documents say. Here is what it looked like when we did it.

In the OptiVX programme, where Equation Labs was primary R&D partner to Gemino on the core AI and ML work package, the controller acted on a living cultivation process. The measured control stack, as published on equation-labs.co, was: a safety-layer latency under 2 ms, an end-to-end edge latency of 285 ms reduced from 1.2 s, growth-model accuracy with R-squared above 0.95, and validation on a 500 L pilot basin with zero safety violations across 400 simulated years, delivered over 29 accounted work phases. The sibling IntelliBot programme, for MindWaves AI Solutions GmbH, closed with more than 400 pages of transfer documentation.

Each number maps onto a section of this article.

• The 2 ms safety layer is the supervision pattern: a deterministic layer between the learned policy and the actuator, small enough to sit inside a millisecond-class deadline with margin to spare. The 285 ms figure is the learned path's end-to-end latency, and the gap between the two numbers is the point: the slow, capable element proposes, the fast, simple element disposes.
• The 400 simulated years with zero violations is the statistical validation set. Years are the right unit because the PFH bands are per hour and one decade wide, so a validation record has to be long enough to say something at that scale. How simulated years become evidence about a physical basin is the subject of our sim-to-real transfer article; the short version is that the 500 L pilot basin is where the simulator was held to account.
• The R-squared above 0.95 belongs to the model's evidence record, not to the SIL claim. It is the growth model's fit, which is what makes the simulator trustworthy, and it is deliberately not presented as a failure frequency.
• The 400-plus pages are the documentation set TR 5469 clause 11.4 asks for: the functional safety artefacts a client or an assessor can read without the team that wrote the code.

We say what was measured and where. No SIL was awarded for this stack and we do not claim one; the campaign was built so that the question could be asked.

What is still not settled

Three things remain open and should be said out loud at the start of any project.

The ceiling is real. Neural network components are not currently accepted as SIL 3 implementations by most certification bodies, and no ML component is accepted at SIL 4 as part of the safety function itself. The guidance on how much statistical evidence is enough is still moving, across ISO/TR 4804, UL 4600 and ISO 8800, and bodies differ on which OOD approaches they will accept. The only remedy is to engage the target body before the evidence package is designed, not after.

The regulation is arriving on a schedule. The ZHAW deck lays out the EU AI Act timeline: AI used as a safety component of machinery falls into the high-risk class, with the Annex III deadline in December 2027 and the embedded AI deadline in August 2028, and the Machinery Regulation (EU) 2023/1230 applies from January 2027. A controller that is safe under IEC 61508 is not yet compliant under the Act; those are two cases to build, with overlapping evidence.

And the standard itself is a technical report, not a norm. TR 5469 was written to seed later normative documents, which means the architectures it accepts are the ones most likely to survive into whatever replaces it. Building to the supervision pattern now is the conservative bet. How the safety case for the monitor is assembled under IEC 61508-3 is the subject of our next article on IEC 61508 and machine learning, and the hardware side, where the monitor and the model share a constrained edge device, is covered under edge AI functional safety.

FAQ

Is machine learning allowed under IEC 61508?

IEC 61508 neither names nor forbids it. TR 5469 states that AI is not treated in mature functional safety standards and is explicitly forbidden in some. Its Annex A shows how the IEC 61508-3 software requirements apply to AI technology elements, so the route is to place the learned element where a non-AI element holds the safety function and to document the decomposition.

What SIL can a neural network be certified to?

As an advisory or initiating element feeding a SIL 1 or SIL 2 function, with statistical reliability demonstrated inside the defined operating conditions. Most certification bodies do not accept a neural component as a SIL 3 implementation, and none accept one at SIL 4 as part of the safety function.

What is ISO/IEC TR 5469?

A technical report, first edition January 2024, from ISO/IEC JTC 1/SC 42, developed in liaison with the IEC 61508 maintenance team. It covers AI inside a safety function, non-AI safety functions that protect AI-controlled equipment, and AI used to develop safety functions. It is informative, not normative, and was written to seed later normative documents.

Does retraining a model invalidate its safety case?

Potentially, yes. TR 5469's introduction notes that continuous improvement undermines verification and validation as the behaviour moves away from the tested state, and safety change management requires re-assessment on any modification. Freeze the validated model, and treat a retrained one as a new release with its own validation run.

Does explainable AI count as functional safety evidence?

No. LIME, SHAP and Grad-CAM approximate the model, the methods disagree with one another, and they run after the fact, so they can never be a safety function. They are useful in engineering review. Assurance comes from the deterministic layer and the validation record.