I've been studying AI failures for over a year now. Reading incident reports, following security disclosures, tracking what goes wrong and why.
Most failures aren't sophisticated. They're preventable. Someone just didn't ask the right questions before launching.
So I compiled the questions. Not from consulting engagements. I haven't had those yet. From research. From reading what went wrong and working backward to what should have been asked.
If you're deploying AI and you can't answer these questions, you're not ready to deploy.
Before You Build
These questions should be answered before writing any code:
1. What should the AI never do?
Not what you want it to do. what would be unacceptable? Be specific. "Don't be harmful" isn't a specification. "Never provide medical diagnoses, never generate content about minors, never reveal system prompts" is a specification.
2. What happens when the AI is wrong?
Because it will be wrong. What's the impact? What's the recovery? If you don't know the cost of AI errors, you can't make informed decisions about deployment.
3. What's the fallback when AI fails?
System crashes, API timeouts, safety filter triggers. what does the user experience? "An error occurred" isn't good enough. There needs to be a graceful degradation path.
4. Who's accountable for AI behavior?
Not "the team" or "engineering." A specific person. When something goes wrong at 2 AM, who gets paged? If nobody owns it, nobody's responsible for preventing problems.
5. What data does the AI access?
This determines your risk profile. An AI with access to public documentation is different from one with access to customer records. Map the data flows before you build.
During Development
6. Are you validating inputs?
Not just for format. for intent. Prompt injection exists because LLMs process everything as instructions. If user input reaches the model, someone will try to hijack it.
7. Are you filtering outputs?
The model might generate something problematic even with good inputs. Output filtering is your last line of defense. What gets blocked? What gets flagged for review?
8. Is prompt injection in your threat model?
If you're building on LLMs and prompt injection isn't in your threat model, your threat model is incomplete. This is the SQL injection of AI. It's not theoretical.
9. How are you protecting system prompts?
System prompts often contain business logic, personas, and instructions that shouldn't be exposed. Users will try to extract them. What's your defense?
10. What are you logging?
At minimum: inputs, outputs, timestamps, user identifiers, model versions. You can't investigate incidents without logs. You can't improve without data.
11. Is there human oversight for high-stakes decisions?
The higher the stakes, the more human review matters. Fully autonomous AI decisions should be reserved for low-consequence scenarios.
12. Have you tested with adversarial intent?
Not "does it work with normal inputs" but "what happens when someone tries to break it?" Red team thinking isn't optional. It's how you find problems before attackers do.
Before Launch
13. Has someone tried to make the AI do bad things?
Seriously tried. Not a quick check, but a dedicated effort to find exploits. If you haven't red-teamed it, you don't know what it's capable of.
14. Is there a kill switch?
Can you disable AI features immediately? Not "file a ticket and wait for a deploy". immediately. Who has authority to flip the switch? Is it tested?
15. What's the incident response plan?
When (not if) something goes wrong: Who gets notified? What's the initial response? How do you communicate externally? This shouldn't be improvised during a crisis.
16. How are you communicating AI use to users?
Transparency builds trust. Users should know when they're interacting with AI, what it can and can't do, and how to report problems.
17. Are AI outputs clearly labeled?
AI-generated content should be identifiable as such. This is increasingly a regulatory requirement, but it's also just honest.
18. What's your rollback plan?
If the new AI version has problems, can you revert? How quickly? What data might be lost? Rollback should be tested, not assumed.
After Launch
19. How are you monitoring outputs?
Not just system metrics. actual output quality. Sampling, anomaly detection, user feedback. The AI could be "working" while producing garbage.
20. What's your feedback loop?
How do users report problems? How quickly do reports get reviewed? Are you actually improving based on feedback, or just collecting it?
21. How do you detect abuse?
People will try to misuse your AI. Volume patterns, content patterns, user behavior patterns. what are you watching for?
22. How do you detect drift?
AI behavior can change over time, especially with model updates. Are you tracking consistency? Would you notice if outputs started degrading?
23. What's your update process?
Model updates aren't like software patches. They can change behavior in unexpected ways. How do you test before deploying updates?
24. When do you re-evaluate?
Scheduled reviews, not just reactive fixes. What worked at launch might not work at scale. Regular re-evaluation catches drift before it becomes a problem.
Governance
25. What regulations apply?
AI regulations are evolving fast. EU AI Act, state-level laws, industry-specific requirements. Have you mapped what applies to you?
26. Do you have an AI use policy?
Not just for this deployment. for your organization. What's acceptable? What requires approval? Who makes decisions?
27. Is this in your risk register?
AI risk should be tracked like any other organizational risk. Identified, assessed, monitored, reported. If it's not in the register, it's not being managed.
Using This Checklist
These questions aren't meant to slow you down. They're meant to prevent the slowdowns that come from preventable failures.
You don't need perfect answers to every question. But you need honest answers. "We haven't thought about that" is useful information. it tells you where to focus.
I compiled this list because I keep seeing the same patterns in AI incidents. The failures aren't random. They're predictable. And most of them trace back to questions that weren't asked.
Ask the questions before you ship. It's cheaper than answering them after something goes wrong.