Artificial intelligence can support a range of organizational tasks, from searching internal documents to classifying requests and drafting summaries. But the relevant business question is not whether AI is powerful. It is whether a specific system is useful, appropriate, controllable, and worth maintaining in a particular environment.
Before commissioning an AI integration, leadership should be able to answer seven questions in plain language.
- 1ScopeWhat exact decision or activity are we improving?
- 2InformationIs the underlying information suitable?
- 3AccountabilityWho remains accountable for decisions?
- 4ReliabilityHow will we test reliability?
- 5SecurityWhat security risks does the design create?
- 6OperationsWho will operate and review the system after launch?
- 7Business caseWhat makes the business case worthwhile?
Each is a legitimate outcome.
If the team cannot define success, it cannot credibly claim improvement.
Key insight
1. What exact decision or activity are we improving?
Start with one real workflow. Describe the input, desired output, responsible person, and what a successful outcome looks like. Use AI to make operations smarter is not a project scope. Help managers find relevant policy sections when answering routine internal questions is much closer.
Ask whether a simpler rule, improved search function, form, or conventional software integration could solve the same problem. NIST's AI Risk Management Framework encourages understanding the system's intended context and determining whether AI is appropriate for the use case.[1]
2. Is the underlying information suitable?
Outputs depend on the completeness, relevance, accuracy, and permissions attached to the information they use. If documents contain conflicting policies or obsolete procedures, AI may reproduce that inconsistency in polished language.
Identify data owners, sensitivity, lawful uses, retention requirements, and the source documents that users should trust. Document what information may not be sent to third-party services. Canadian privacy obligations vary by organization, sector, province, and activity; the federal Privacy Commissioner provides general business guidance, but project-specific legal review may be necessary.[2] Québec's private-sector privacy law, for example, requires enterprises subject to it to conduct a privacy impact assessment for projects to acquire, develop, or overhaul an information system involving personal information, and requires organizations to inform people when a decision about them is based exclusively on automated processing.[4]
3. Who remains accountable for decisions?
A person should be able to identify when a system is wrong, correct its output, and stop or override the workflow. Set clear boundaries: can AI suggest, classify, summarize, or trigger an action? Can it modify a record? Can it send messages externally? The more consequential the action, the stronger the human approval and audit controls should be.
4. How will we test reliability?
Define representative real-world cases, known difficult cases, and what constitutes an unacceptable error. Measure more than whether an answer appears fluent. Depending on the use case, relevant checks include factual accuracy, unsupported claims, completeness, correct retrieval of authorized information, and whether escalation occurs when confidence is insufficient.
A pilot should include both routine and exceptional cases. If the team cannot define success, it cannot credibly claim improvement.
5. What security risks does the design create?
AI systems may introduce novel failure paths on top of ordinary application-security risks. OWASP's Top 10 for LLM and generative AI applications identifies issues including prompt injection, sensitive-information disclosure, excessive agency, and improper output handling; its 2026 edition updates the rankings and expands the threat coverage of the 2025 list.[3]
Controls may include limiting tool permissions, validating outputs, protecting secrets, segregating data, recording actions, and restricting system authority. These protections must be implemented and tested for the proposed design; they are not properties of AI technology in general.
6. Who will operate and review the system after launch?
Identify the accountable owner for changes, model or vendor updates, incident response, feedback, monitoring, and policy revisions. If the workflow changes, who revises prompts, rules, evaluation cases, or connectors? What is the process for disabling the integration when needed?
A prototype is not an operational service until ownership, support, and monitoring are defined.
7. What makes the business case worthwhile?
Set a baseline and evaluate benefits alongside implementation effort, data preparation, exception-handling cost, supervision, risk, and vendor fees. Possible measures include turnaround time, manual touches, percentage of outputs accepted without correction, escalation rate, and user satisfaction.
Use conservative estimates. An AI-generated summary that saves two minutes but requires extensive review may not produce meaningful net value.
Turn the answers into a small, controlled pilot
A readiness discussion should result in one of three decisions: proceed to a limited pilot, complete prerequisite data/process work, or defer AI in favour of a simpler solution. Each is a legitimate outcome.