There is a tempting idea behind a lot of AI products.

If the model is doing something wrong, the prompt just needs to be better. Add more instructions. Give it more examples. Tell it what not to do. Eventually, the thinking goes, you can prompt your way into reliable behavior.

Prompts matter, but there is a limit to how much responsibility we should place on them.

That becomes especially important in training.

A prompt is not a training system

Suppose you want an AI character to play an upset customer.

You could give an LLM a detailed prompt explaining who the customer is, why they are upset, what they want, how angry they should be, what information they know, how they should respond to different approaches from the learner, and so on.

That can produce results that look convincing, but there may still be some problems.

Maybe the customer becomes cooperative too quickly. Maybe they suddenly introduce information that was never part of the scenario. Maybe they praise the learner for a response that should actually make the situation worse.

The obvious response is to keep adding instructions.

Now the prompt says not to become cooperative too quickly. Then you add instructions about what cooperation means. Then another instruction explains when cooperation is allowed. Pretty soon, the prompt is doing an enormous amount of work.

And you still have the same underlying problem.

The system responsible for interpreting the rules is also responsible for deciding whether it followed them.

I have some firsthand experience with this.

Before we built SkillSaige, I was the project manager on a proof of concept for a more traditional prompt-based AI training system. I was (unfortunately) heavily involved in both the prompt engineering and the QA testing.

It very quickly became a game of whack-a-mole.

You would test the system, find an unwanted behavior, add something to the prompt to address it, and test again. That might solve the original problem, but now something else behaves differently. So you add another instruction. Then another.

The prompt gets longer. The testing surface gets larger. And every change gives you more things to verify.

That experience had a major influence on how we approached SkillSaige. From the beginning, we knew we did not want reliability to depend on endlessly patching a prompt.

Reliability requires structure

Our CEO Sayre recently wrote more about this problem on her blog, particularly why testing an LLM is different from testing conventional software.

Traditional software gives us something extremely useful. Under the same conditions, we generally expect the same logic to run.

An LLM does not naturally behave that way.

That does not make LLMs bad. In fact, some variability is exactly what makes them useful for roleplaying exercises, but it means we need to be careful about which parts of the experience we allow to vary.

In a training simulation, the learner might phrase the same idea in dozens of different ways. The AI character might respond with different wording each time. That is useful variability.

Whether the learner successfully accomplished the objective is a different question.

If the goal is to practice de-escalation, the system should have some dependable way to determine whether the learner is actually de-escalating the situation. That decision should not change simply because the model happened to interpret a sentence differently on Tuesday.

Separate language from logic

This is a big part of how we think about AI at SkillSaige.

LLMs are extremely useful for language. They can interpret what someone says, generate natural responses, and make a simulated conversation feel much less rigid than a traditional branching exercise.

But language generation does not have to control the entire simulation.

The underlying system can still track objectives, state, behaviors, and consequences separately. Maybe the learner acknowledges the other person’s concern. Maybe they avoid answering the question. Maybe they become defensive.

Those things can affect what happens next without asking the LLM to independently invent the rules every time.

The AI can then focus on something it is exceptionally good at. Talking.

Better prompts are still useful

None of this means prompting is unimportant.

A good prompt can dramatically improve how an AI character sounds and behaves. It can establish personality, context, vocabulary, tone, and countless other details that make a simulation believable.

The mistake is treating the prompt as the entire application. I have already worked on a system where improving reliability meant repeatedly testing the prompt, finding the next edge case, and adding another rule to deal with it.

We did not want to build SkillSaige that way.

Training needs room for variation because real conversations vary, but the learning objectives should not drift along with them. Reliable AI training comes from deciding which parts of the experience should be flexible and which parts absolutely should not.