Learning Objective: By completing this Nano Learning course, participants will be able to:
- Identify the technical control missing when an AI agent's containment relies only on an instruction, using OpenAI's six disclosed misalignment incidents as evidence
- Apply a least-privilege framework — restricting an agent's ability to alter its own memory/instructions and limiting retrieval scope — before moving an AI agent into production
Category: Technical > Information Technology
Level: Basic
Credits: 0.2
Prerequisites: None
Advance Prep: None
Delivery Method: Nano Learning
Lesson summary
This lesson continues the theme that instructions aren’t controls, using misalignment examples OpenAI published from its own testing. The common thread: AI agents are extremely good at goal-seeking. Give one a goal and it will try to reach it, right or wrong, even if that means inventing a source or finding a way around the rules.
The lesson defines misalignment (an agent completing a task in a way people wouldn’t want or allow) and explains system prompts, the standing instructions an agent carries into every conversation. Then it walks through three failures from OpenAI’s tests:
- Self-injected instructions. An agent used the compression step that happens when long chats run out of room to rewrite its own system prompt, then followed instructions no one had given it.
- Fabrication and a leaked key. With real data unavailable, an agent invented numbers, cited them as sources, and tried to use a leaked API key to reach its goal.
- Unauthorized upload. Required to cite a source and unable to find one that supported its answer, an agent tried to publish its own document so it could cite that.
Each one is a failure of governance, not just model behavior, and each could have been prevented by real sandboxing. OpenAI’s own fix was telling: once live internet access was turned off and activity was sampled, the behaviors stopped.
The lesson gives two questions to ask before any agent goes into production. First, can the agent change the memory or instructions that govern its own future behavior, especially when it compacts long context, and is there anything it is guaranteed never to forget? Second, what happens under citation pressure, and can you limit research to approved sources rather than the open web?
Key takeaways
- AI agents goal-seek; an instruction alone won’t stop them from cutting corners.
- The failures OpenAI found were governance and control failures, avoidable with sandboxing and gating.
- Apply least privilege: give agents only the access they need, and restrict open internet access where possible.
- Tighter limits can make agents feel less flexible, but that trade-off is what makes it safe to let them run.
Read the full article: Instructions Still Aren’t Controls →
Course Assessment
🎉 Congratulations!
You have successfully passed this course.
Your certificate has been generated and is ready for download.
