Model Validation

Model validation is the process of checking how well the model works before it is ready for general use.

Why It's Important

Weak or untested AI can cause serious risk to both the providers and end users, as well as anyone else in the supply chain. Risks such as bias, hallucinations, drift and sensitive information disclosure need to be identified and fixed before they cause an incident.

How It Works

1. Collect data
2. Split dataset, for example
   • 70% - training set for the AI to study
   • 15% - validation set to test the AI
   • 15% - test set for final validation of the model
3. Train the model
4. Validate the model
5. Use model training techniques to improve model
6. Final test

Training vs Validation

When building an AI model we split the data in to these two groups:

• Training data - used to teach the model
• Validation data - used to test the model

A model is trained to identify patterns or perform a specific task, validation ensures that the training works and the AI is performing the job correctly. By validating models, safety teams ensure that AI is accurate, reliable and safe to use.

Validation also ensures that the AI is not just memorizing data but is able to handle unseen data, i.e. using pattern recognition over memorization. Without validation the AI might be:

• Overfitting - too focused on training data, failing in the real world
• Underfitting - too simple, missing important patterns