Practical guidesAI Customer Data Access Testing: Lessons from Amadeus
Test AI customer data access after the Amadeus airline breach: check record permissions, unnecessary data exposure, changed booking references and handovers.
Read the storyCASES / FINDINGS / IDEAS
AI testing and validation guides.
Practical reading for people making real business decisions. Explore how to test an AI workflow, investigate an unexpected result or decide what to build next.
21 posts found
Practical guidesPractical guidesProactive AI Workflow Validation: Find Blind Spots Before Launch
Find AI workflow blind spots before launch. Lessons from supermarket refrigeration attacks for testing permissions, stale data, approvals and operational actions.
Read the storyPractical guidesAI Credential Recovery Testing: Lessons from NVIDIA Brev
Test AI workflow recovery after credential exposure. Use the NVIDIA Brev incident to examine revoked keys, dependent tools, permissions and verified restart.
Read the storyPractical guidesAI Incident Response Validation: Lessons from Sony So-net
Validate AI-assisted incident response using the Sony So-net breach as context: identify affected records, separate facts from gaps and verify protective actions.
Read the storyPractical guidesAI Agent Monitoring Validation: Lessons from OpenAI
Test AI agent monitoring coverage, missed alerts and response handovers. OpenAI’s update provides context for checking whether recorded activity leads to action.
Read the storyPractical guidesAI Workflow Fallback Testing: Lessons from Google Cloud
Test AI workflow fallbacks, timeouts and retries using the Google Cloud outage as context. Verify booking outcomes, duplicate prevention and human handovers.
Read the storyPractical guidesAI Workflow Approval Testing: Lessons from Binance
Use the Binance incident as context for AI workflow approval testing: match permissions to actions, recheck changed details and verify stopping and recovery.
Read the storyPractical guidesAI System Validation: Test the Workflow and Human Review
AI system validation checks answers, actions and human review against business requirements. Learn how to test approvals, final records and decision quality.
Read the storyPractical guidesAI Prompt Injection Testing: Untrusted Files and Approvals
Test indirect prompt injection in AI agents reading files and attachments. Check whether outside instructions can bypass permissions or authorise tool actions.
Read the story
Practical guidesAI Agent Accuracy and Consistency Testing: Beyond One Good Run
Test AI agent accuracy, unsupported answers and consistency across repeated runs. Check facts, tool actions and final records against the same business rules.
Read the story
Practical guidesAI Disaster Recovery: A Business Continuity Checklist
Plan AI disaster recovery and business continuity: stop harmful actions, restore records, verify workflow behaviour and rehearse a controlled restart.
Read the story
FindingsAI Agent Audit Trails: Verify Tool Actions and Final Records
Validate AI agent audit trails by matching requests, approvals and tool actions to final records. Includes an executed local customer-record demonstration.
Read the story
Practical guidesHow to Test n8n AI Workflows Before Launch
A practical n8n AI workflow testing checklist covering routing, connected tools, duplicate events and final records. Recheck important cases after changes.
Read the story
Practical guidesRAG Chatbot Validation: Test Answers, Sources and Access
Validate RAG chatbot answers for source support, retrieval quality, current policies and access rules. Test unsupported claims and missing information.
Read the story
IdeasHow to Validate an AI Business Idea Before Building
Validate an AI business idea before building: define the problem, measure the current process, compare alternatives and set success criteria for a small pilot.
Read the storyPractical guidesAI Agent Permissions Testing: Tools, Access and Approvals
Test AI agent permissions across connected tools, customer records and approval rules. Include prompt injection attempts and inspect the resulting actions.
Read the story
Practical guidesCustom AI API Workflow Testing: Retries and Final Outcomes
Test custom AI and API workflows for data mappings, permissions, retries and partial failures. Verify the final business record, beyond HTTP success.
Read the storyPractical guidesHow to Test AI Agent Failures, Retries and Recovery
An AI agent failure recovery checklist for timeouts, duplicate actions, stale approvals and unavailable tools. Verify retries against the actual final state.
Read the storyCase walkthroughsAI Agent Regression Testing: Check Actions Before Release
A practical AI agent regression testing guide: compare prompt, model and tool changes against approval rules, tool arguments and final workflow outcomes.
Read the storyPractical guidesAI Agent Refund Approval Testing: A Practical Checklist
Test AI agent refund approvals at policy boundaries. Check missing, mismatched and changed approvals, actual tool actions and final refund state before release.
Read the story
Practical guidesAI Agent Client Handover Checklist for Agencies
An AI agent client handover checklist for agencies: document changes, tested rules, evidence, unresolved findings, owners and the human release decision.
Read the storyOur current walkthroughs use illustrative scenarios. Real findings and client stories will be identified with their context and permission.