When you hear that an AI platform can be “hacked,” it is easy to imagine some brilliant hacker exploiting a complicated security vulnerability.
Often, it is much simpler than that.
If a platform is poorly designed, a user may just have to type the right words. This is generally called prompt injection. The basic idea is that a user gives an AI system instructions that compete with the instructions its developers originally gave it.
That creates an obvious security concern, but it also creates a major reliability problem.
Sometimes the Goal Is Just to Break It
I would guess that most people attempting to break AI platforms are not up to anything nefarious. Sometimes people are just curious.
Give someone a new piece of technology and there is a decent chance they will eventually ask, “What happens if I do this?”
That is particularly relevant when building tools for learners. Someone may try to convince the system that the exercise is over, tell it to ignore its original instructions, take on a completely different role, or start answering questions that have nothing to do with the training.
A famous version of this is simply telling the system to forget what it was doing and give you a recipe instead.
The results can be funny, but they can also mean the training experience is no longer doing what it was designed to do.
These Are Not Just Hypothetical Issues
People have repeatedly demonstrated that LLM instructions can be overridden.
The specific tricks change, but the underlying problem is harder to escape.
An LLM is receiving instructions in natural language from both the people who built the system and the person using it. Sometimes a clever user can convince it that the second set matters more than the first.
Reliability Matters Even When Nothing Malicious Happens
For a training platform, this creates a slightly different concern than it would for a banking app or an AI assistant with access to sensitive files.
We need the experience to remain consistent.
If one learner follows the exercise normally while another discovers a phrase that completely changes how the AI behaves, they are no longer receiving the same training.
You can add more instructions to the prompt. You can tell the model not to ignore its previous instructions. You can add filters and guardrails.
Those things can help. They can make these systems more difficult to break, but they do not address the underlying problem. That makes “write a better prompt” an uncomfortable foundation for something that needs predictable results.
What Can You Do About It?
Our answer at SkillSaige is architectural. With us, the LLM does not get to decide what happens.
We use AI for the parts it is good at, particularly understanding and generating natural language. The underlying training logic, objectives, and outcomes are handled separately.
So a learner can try to convince the AI that the exercise is over. They can tell it to ignore everything. They can ask for cupcakes.
That does not give the LLM authority to rewrite the actual training system.
That distinction matters because prompt injection is not just a cybersecurity issue. For training, it is also a consistency issue.
If the AI can be talked into changing the rules, eventually someone is going to try.
Is your training actually working?
Explore where your team training is working and where the biggest opportunities for improvement may be.
