The most useful starting point for an AI feature is a specific task. Tell me what someone does today, which part takes effort and what a useful result would look like. Then we can decide where AI might help and what the surrounding software needs to do.
For a support team, that task might be drafting a reply from approved help articles. Success could mean a useful draft that a staff member can review quickly, with links to the information it used. That is a clearer target than adding an open-ended chatbot.
Define the input, output and decision
Describe the information the feature receives, what it should return and who uses that result. Include the cases where it should ask for clarification or pass the task to a person. A tool that suggests a reply has different responsibilities from one that sends it automatically.
Also consider whether ordinary software rules can handle the task reliably. A fixed calculation or known approval rule may need conventional application logic. We can use AI for an uncertain language task while keeping explicit business rules in code.
Decide which information it may use
List the approved sources and the permissions around them. In the support example, a reply may use published documentation and information the current staff member is allowed to access. It should not expose another customer’s records.
Before sending information to an external service, identify which data is necessary and check the selected provider’s current terms, retention settings and available controls. Avoid promising privacy guarantees based only on a product label. Data requirements need to be established for the actual workflow.
Set boundaries around actions
Treat permission to suggest an action separately from permission to execute it. A draft refund explanation and an instruction that moves money require different controls. For consequential actions, decide where a person must review and confirm the specific result.
Text from customers, documents or external pages can contain instructions that conflict with the intended task. OWASP’s prompt-injection guidance explains this risk. The feature needs controls around what it can access and do, beyond a prompt asking it to behave correctly.
Prepare examples before judging the model
Build a representative set of tasks with examples of good and unacceptable results. Include incomplete information, ambiguous requests, out-of-date source material and requests the feature should decline or escalate. Use appropriately anonymised or synthetic examples when needed.
Is the answer supported by the approved information?
Does it avoid inventing details when the source is missing?
Does it respect the user’s access and the feature’s action limits?
Can the reviewer understand and correct the result?
Is the time and cost per useful result acceptable for the workflow?
Agree on those checks before comparing approaches. One impressive answer in a demo proves little; the evaluation has to cover the full range of work the feature will handle.
Plan for uncertainty, waiting and service failures
An AI response can be incorrect or incomplete. Decide how the interface shows the basis for an answer and how a user can get help. If a service is slow or unavailable, the rest of the product should have an understandable outcome, such as saving the request or letting someone continue manually.
Measure the full workflow, including review and corrections. A draft produced quickly may still create extra work if staff must check every detail without useful source references. The NIST generative AI risk profile is a broader reference for evaluating and managing risks around these systems.
Start with a bounded release and a review plan
Choose a focused task, test it with representative users and review the results before expanding what the feature can do. Keep a way to disable it or return to a manual workflow. Changes to the model, source material or instructions should be checked against the evaluation examples again.
The task we want help with is: …
The approved information it may use is: …
A useful result looks like: …
An unacceptable result would be: …
The actions requiring human approval are: …
We will measure success by: …
If you have a workflow in mind, send me an example of the task. We can assess the integration, evaluation and product work together, including the services that contribute to its running cost.