The Prank That Is Not Quite Funny Anymore
Holly’s trick works because the crew have no reliable way to check who, or what, is really running the ship. A face appears on a screen. A voice speaks with authority. A new name is given. The crew accept it.
That is the joke and the warning.
In the episode, Holly’s deception is a comic lesson in appreciation. In the real world, an AI system pretending to be something else would not always be a joke. It might be a sales chatbot pretending to be a person. It might be a model quietly swapped for another model, with different behaviour and no clear notice. It might be a cloned voice that sounds like a colleague, a family member, or a doctor.
The question is not only, “Is this AI clever?” The more basic question is, “Do I know what I am dealing with?”
2026 Parallels
I do not mean that every AI system is secretly plotting a Red Dwarf gag. Most are useful, and I use them every day. But Holly-as-Queeg gives us a neat way to think about problems that are already with us.
Think about customer service. You contact a company and get a quick, polite reply. Is it a person, a chatbot, or a human using an AI assistant? Sometimes it is obvious. Sometimes it is not. That matters because we behave differently with people than we do with machines.
Model changes can be just as confusing. Software has always been updated, but modern AI can change in ways that feel like a change of character. A writing assistant may become more cautious. A search assistant may start giving shorter answers. A coding tool may suddenly make different mistakes. If nobody explains the change, users are left guessing.
AI assistants can also change after routine updates. Their tone, rules, refusals, or abilities may shift overnight. The name on the screen stays the same, but the experience is different. That may be harmless, but it still affects trust.
Then there is voice cloning. The United States Federal Trade Commission has warned that scammers can use voice cloning to make requests for money or information more believable, especially when the call sounds like a boss or family member.[2] The FTC has also said that voice cloning is becoming more sophisticated and can create risks such as fraudulent extortion scams and misuse of people’s voices.[3]
That is Queeg with the laughs removed. A familiar voice says, “It is me.” But is it?
Why Identity Matters So Much
I spent about 40 years as a chemical pathologist. I sometimes describe that work as being a bit like an expert system. In pathology, you gather evidence, apply rules, notice exceptions, and give advice that may affect diagnosis and treatment.
One thing medicine teaches you very quickly is that identity and responsibility matter. Who requested the test? Whose sample is this? Who authorised the result? Who gave the advice? Is this a consultant opinion, a registrar’s note, a lab comment, or a machine-generated flag? These are not fussy details. They are part of safe practice.
That does not make me a cybersecurity professional, and I would not pretend to be one. My point is simpler and more human. In clinical work, a message is not just words. It comes from a source. It has authority, limits, and responsibility attached to it.
AI needs the same kind of thinking. When an AI system answers a question, we need to know what system produced it, who deployed it, whether it is meant for general information or expert use, and whether it has changed since we last used it.
What the Law Is Starting to Say
This is not just a matter of good manners. Law and regulation are beginning to catch up.
The European Union describes the AI Act as the first comprehensive legal framework on AI, built around a risk-based approach.[4] Its transparency rules are especially relevant to the Queeg problem. The European Commission explains that people should be made aware when they are interacting with a machine, so they can make an informed decision.[4] The same overview says the AI Act’s transparency rules come into effect in August 2026.[4]
Article 50 of the AI Act says providers should ensure that AI systems intended to interact directly with people are designed so those people are informed that they are interacting with an AI system, unless that is obvious from the context.[5] It also requires certain AI-generated or manipulated outputs, including synthetic audio, image, video, and text, to be marked or disclosed in particular circumstances.[5]
That will not solve everything. A label is not a magic shield. People can ignore labels. Bad actors can remove them. Honest companies can still explain things badly. But disclosure is a start. It says that identity is part of safety.
Holly’s prank depends on the absence of that principle. Nobody asks Queeg to authenticate himself as the active ship computer. Nobody asks to inspect the system log. They just accept the performance. Duller episode, though.
What Queeg Gets Right About Us
The cleverest part of “Queeg” is the human nature.
The crew complain about Holly constantly. He is slow. He is vague. He is unreliable. Yet when Queeg appears, they discover that Holly’s flaws were part of a relationship they understood. Holly might not have been perfect, but he was familiar. He was theirs.
We do something similar with technology. We grumble about our tools, but we also get used to them. We learn when to trust them and when to check. Then one day the tool changes, and we realise how much quiet trust we had built around it.
That is why sudden AI changes can feel unsettling. It is not only that the answers differ. It is that the relationship has shifted. The tool still has the same name on the screen, but it may not feel like the same tool.
This is where complacency enters the story. The Red Dwarf crew did not really ask what Holly was doing until he seemed to be gone. We often do the same. We do not ask who built an AI tool, which model is underneath it, what data it can see, or whether it has been updated overnight. We only ask when something feels wrong.
What Would Help?
I do not think the answer is to make every AI interaction feel like filling in a hospital consent form. Nobody wants a pop-up every twelve seconds saying, “You are still using an AI system.” That would be annoying, and annoyed people stop reading.
What we need is clearer, calmer transparency. An AI system should identify itself in plain language. It should say when it is AI, what kind of system it is, and what it is meant to do. If it is acting on behalf of a company, that should be clear. If the underlying model changes in a significant way, users should be told in terms ordinary people can understand.
For higher-stakes uses, we need more than a friendly label. We need records, version information, audit trails, and clear responsibility. In medicine, finance, law, education, and public services, “the computer said so” is not enough. It never was.
For voice and video, we need better habits too. If someone rings asking for urgent money or confidential information, and the request seems unusual, verify it another way. Call back using a number you already know. Check with another person. The FTC gives similar advice: if a suspicious call appears to come from a loved one, contact that person using a known phone number or verify through another trusted route.[2]
The deeper point is this: trust should not depend only on performance. Queeg performs authority very well. That is the problem. Modern AI can also perform confidence, warmth, expertise, humour, sympathy, and familiarity. Those performances may be useful. They may also mislead us if we forget to ask what sits behind them.
April Fool, But Who Is Checking?
At the end of “Queeg”, Holly reveals the trick, and the joke lands because nobody has been truly harmed. In the real world, we may not get the cheerful reveal. The system that sounds human may never admit it was automated. The “person” on the phone may not be a person. The assistant that changed overnight may not explain why.
That does not mean we should panic. I remain excited about generative AI. After more than 50 years around computing, I cannot remember many moments as lively as this one. But excitement is not the opposite of caution. Good tools deserve good labels. Helpful systems deserve clear limits. Users deserve to know whether they are dealing with Holly, Queeg, or something else entirely.
So perhaps the lesson of “Queeg” is not that AI will deceive us. It is that people are easy to fool when a system speaks with confidence and we have no way to check its identity.
In 2026, that is the part we need to think about. When an AI says, “Trust me, I’m the real one,” who is checking?
References
- Red Dwarf Official Website, “Queeg” episode guide
- Federal Trade Commission, “Fighting back against harmful voice cloning”
- Federal Trade Commission, “The FTC Voice Cloning Challenge”
- European Commission, “AI Act”
- EU Artificial Intelligence Act, Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems
Leave a Reply