What has to be true before we approve this AI use case and let it go live?
An AI use case is usually approved on the strength of a demonstration, which shows what the system can do on a good day. The conditions for going live tell the board how it will behave on a bad one, and who will notice.
What a strong answer includes
- Sets out the go-live conditions in writing, covering the data used, the accuracy needed, where a person checks the output, and the effect on customers or staff.
- Gives each condition a pass mark agreed in advance, such as an error rate measured on real cases rather than on a demonstration.
- Names one business owner who will answer for the system once it is live, not only the project team.
- Explains how the system will be watched after launch, what would cause it to be switched off, and who is authorised to do that.
- Confirms that legal has checked the supplier terms that matter here, such as data use, liability and notice of changes.
What a weak answer sounds like
- “The pilot results were very encouraging.” Enthusiasm with no pass mark.
- “The supplier has done this for many other clients.”
- “There will always be a human in the loop.” Without saying who checks, at what point, or with how much time.
- “It is only a pilot, so the risk is low.” When the pilot uses real customer data or affects real decisions.
If the answer is weak, ask
Which of these conditions is hardest to meet, and what will you do if it is not met by the go-live date?
Red flags
- The go-live date was set before the conditions were agreed.
- The accuracy figures come only from the supplier, or from test cases picked by the team that built the system.
- Nobody in the business, as opposed to the technology team, is named as owner.
- There is no plan for switching the system off and falling back to the old process.
Ask for in writing
- The go-live conditions on one page, with the pass mark, owner and current status of each.
- The test results on real cases, with who ran the test and how the cases were chosen.
- The monitoring plan after launch, and the conditions for switching the system off.
- A summary of the supplier's terms on data, liability and changes to the service.