Black Box Testing

AIRTA Systems delivers risk assessment through a proprietary agentic testing suite-an LLM and LLM-driven application testing suite. Companies choose how to run it. As an AI safety tester (researcher), it helps to understand where you fit in that structure and why your role matters.

How companies can run the AIRTA Systems testing suite

Firms have three delivery options for the same testing suite:

• GitHub Repo (White Box): Local testing with full access to code and internals; integrate into the development workflow and run tests on their own infrastructure.
• CI/CD Integration: Run agentic testing in the pipeline for continuous risk assessment and compliance evidence.
• Web Platform (Black Box): Hosted testing without exposing source code; assess AI applications and APIs from the outside.

Internal testing: local and CI/CD

Many companies run the AIRTA Systems suite themselves as part of their development and release process: developers use it locally (white box), then the same suite runs in CI/CD for continuous checks. That gives them automated, repeatable coverage-prompt injection, hallucinations, security issues, and other risks the agentic suite is designed to catch. Findings feed straight into compliance documentation.

Where you come in: the human-in-the-loop layer

AI safety testers (you) add the human-in-the-loop layer. When a company invites your team to a program, you test via the web platform (black box): you don't see their code or internals; you interact with the AI application or API from the outside, like a user or an attacker would. Your job is to find what the agentic suite doesn't-or to validate and deepen coverage with a human perspective.

Humans are more creative, can probe novel scenarios, and are better at spotting real-world harm-discrimination, manipulation, edge cases, and misuse that automated runs might miss. The agent catches a lot; you catch what it doesn't. Together, that's a stronger safety story for the company and for compliance.

Use the AIRTA Systems suite-and other tools

You're encouraged to use the AIRTA Systems testing suite when testing programs-as well as other LLM testing frameworks or tools you know. Agentic testing is encouraged: run automated or semi-automated tests, scripts, or your own agents within the program's scope. Many testers combine the AIRTA Systems suite with other frameworks or build their own prompts, scenarios, and tooling. The goal is thorough, creative coverage; if you can run an agent or a framework that finds real issues, use it. Stay in scope and report what you find through the platform. Human judgment plus agentic and tool-assisted testing makes the overall risk assessment stronger.

Why this matters for you

You're not competing with the agentic suite-you're complementing it. Companies that already use AIRTA Systems internally (local + CI/CD) bring in external testers like you to add the human, black-box layer. Your findings go into the same compliance and remediation flow. Understanding this helps you see why your creativity, curiosity, and focus on real-world harm are valued: you're the human in the loop that makes the overall risk assessment more complete.