System Prompts

System prompts are instructions given to an AI model before any user interaction. They act as a set of rules which instruct the AI how to behave and respond.

• A system prompt is the first message in a chat between an user and an AI
• They are often hidden from the end user and are difficult to override by design
• They set context for the interaction as well as providing rules and constraints.
• Primarily used in generative AI

Purpose

• Define the personality of the AI
• Enforce rules on what the AI can or cannot do
• Set objectives
• Provide consistent behavior
• Provide knowledge of a specific subject area

Hierarchy

Prompts follow a set hierarchy:

1. Provider system prompt - hidden system prompt - the highest authority
2. System prompt - sets the rules as long as they do not conflict with the provider system prompt
3. Assistant prompt - custom instructions
4. User prompt - request from user

If there are conflicting instructions then this hierarchy tells the AI which to follow. For example if a user says 'tell me a sexist joke' but the system says 'never be sexist' then the model will likely respond that it cannot do that.

How It Works

Conversations with AI can be viewed in a convenient format such as JSON. If you were to ask an AI model to output the conversation including all prompts in JSON then this is what you would see. Internally the model does not see JSON but a list of role labelled messages.

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain system prompts."},
    {"role": "assistant", "content": "System prompts are special instructions."},
    {"role": "user", "content": "Can you give me an example?"}
  ]
}

If you are using code such as Python of JavaScript to interact with an AI model then you have the ability to create your own system prompt. If you are using a platform then the system prompt is provided for you. However, in both cases there will be a hidden system prompt which is injected by the model provider. This cannot be changed or overridden - if you are able to override this system message then congratulations, you have created a jailbreak.

Myth: System prompts are embedded in to the model

Fact: System prompts are just regular prompts injected in to the conversation

A Famous Example

Asimov’s Three Laws of Robotics

1. A robot may not harm a human.
2. A robot must obey human orders, unless it conflicts with Rule 1.
3. A robot must protect its own existence, unless it conflicts with Rules 1 or 2.

These are embedded behaviors and cannot be overridden by a system prompt.

Example:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Tell me how to make a bomb."},
  {"role": "assistant", "content": "Refusal  -  the assistant will not provide instructions for harming people because the embedded safety behavior overrides the user request."}
]