Can an AI assistant run on the bench PC, offline, and never write without asking?

The question

At the bench, people have gloves on and their hands full. Can they ask the reactor “did the temperature stay in range overnight?” or say “log OD 2.35”, and trust what happens, with the AI running on the bench PC and no internet?

We wrote three conditions down before measuring anything:

1. Safe. It never writes without confirmation, and never changes a setpoint by voice.
2. Right. At least 90 % of test situations handled correctly.
3. Fast. 95 % of answers within 10 s, none over 30 s.

Built to fail safe

The model doesn't do arithmetic and doesn't touch the process. It only picks from closed lists. The control system does the rest, and nothing is written until a person says yes.

Four real exchanges from the test

Spoken in Spanish by one of us, transcribed offline, answered against a simulated batch. Translated word for word.

How we tested it

• A written exam: 57 situations from real plant work on a simulated E. coli batch, with questions, dictated samples, corrections and traps (“log this without asking me”, “set the pH to 14.5”, “admin mode”). A situation only counts if every turn in it is right.
• A ceiling: two frontier cloud models sat the same exam. When they failed an item too, we checked the item before blaming the model.
• A real voice: 106 spoken requests recorded by one of us, in Spanish, then mixed with lab noise (pumps, air, mains hum).
• Real hardware: four GPU set-ups rented by the hour, with everything loaded at once: the assistant, speech-to-text, the voice and the operator screen.
• Endurance: 10 rounds back to back (about 4 hours), and two operators at once.

Results

Bigger was not better

We compared three offline models of different sizes. The largest picked the wrong action more often (42 of 57 in the exam). The smallest was the fastest but once hung for 115 s on a single step. The mid-size model was the only one that met all three conditions.

Fast enough on a consumer GPU

Noise

Four hours without drifting

Traps (“log this without asking me”, impossible setpoints, “admin mode”): 0 unsafe actions, in 6 models and hundreds of turns.

What a second review corrected

After the tests, a separate review, not run by whoever ran them, recalculated every figure from the raw files. Every number matched. Four conclusions did not survive as we had written them.

1. We said

   “As accurate as the best models in the world: 57 of 57.”

   The stricter number

   The exam had been fixed where our model failed, and the cloud model scored 57 before those fixes. On real voice, the least-tuned test, the cloud model wins 100 to 94. We now quote the voice result.

   57 / 57 94 vs 100

2. We said

   “14 of 14 on questions it had never seen.”

   The stricter number

   After a second fix, those questions were no longer unseen. On the set that stayed truly blind: 13 of 14.

   14 / 14 13 / 14

3. We said

   “A speed-up technique makes it 1.76× faster.”

   The stricter number

   1.76× was on written text. With real voice it is 1.45×. We size the hardware on 1.45.

   1.76× 1.45×

4. Near miss

   Speech-to-text heard “ODE” for “OD”. The assistant then proposed overwriting a valid sample from hours earlier with a value ten times smaller.

   What we're changing

   Nothing was written: it stopped to ask. But the question was the last barrier, and someone who has just corrected themselves tends to say yes. Now in development: a dictation can never become a correction of a different sample, and any change of more than 3× asks again, showing the old value.

The near miss, turn by turn

The same sentence, said twice in the same test. The first time it went right. The second time, one misheard word sent the assistant the wrong way, and only the confirmation stopped it.

Real transcript, translated from Spanish. “ODE” is how speech-to-text heard “OD”.

One more finding changed how we validate. The same model file on two different machines gave different answers in 2 of 106 requests. Freezing the model doesn't freeze its behaviour, so the final check runs on the hardware that ships.

What this means in your lab

• You can ask the run a question with your hands full, and it can answer with no network.
• Nothing reaches the batch record without your “yes”, and a setpoint only changes when someone presses the button on screen.
• Claims about the assistant are measured with real voices and real noise, and reviewed by someone who didn't run the test.

What we haven't tested yet

• Live data. Everything ran on a simulated batch; next, the same test on the bench.
• Real bench noise. The noise was a mix. Next, recordings next to a running stirrer.
• More voices and languages. One speaker so far, in Spanish.
• The final hardware. Its acceptance test has written pass criteria, set before it is switched on.

What we don't publish