Speakers:
Building Trust in Agentic AI: From Pilot to Production
Date:
Wednesday, May 19, 2027
Time:
2:35 pm
Track:
Summary:
AI pilots are easy; production is not. This session gives business analysts, architects, and transformation leaders a practical playbook for taking AI agents from demos to trusted enterprise systems. Drawing on an enterprise-scale deployment with two agent personas, one customer-facing and one for infrastructure operations, we walk through the four capability gaps that decide whether AI output can be trusted: capture (complete, well-structured context into the model), validation (checking AI output before it acts, including automated review by a second model), guardrails (tool permissions and blast-radius limits), and observability (measuring quality at scale so drift is caught before customers are). We also cover human-in-the-loop approvals for high-risk actions, evaluation at volume instead of demo-day anecdotes, and lessons of redesigning work around AI that is right most, but not all, of the time. Attendees leave with a capability checklist they can apply the week they return.
Learning Objectives:
- Describe the four capabilities (capture, validation, guardrails, observability) that separate production-grade AI from demos
- Apply tool-permission and human-in-the-loop approval patterns to limit AI agent risk in core business processes
- Use volume-based evaluation to measure AI output quality and catch drift before it reaches customers