It is easy to build an AI prototype. Wire up an SDK, write a clever prompt, demo it on a clean dataset, and the room nods. The hard part is the rest: the input handling, the failure modes, the audit trail, the cost ceiling, the rollback plan.
What we focus on after the prompt works
- Inputs. Validate, normalize, and redact before the model sees them.
- Outputs. Parse defensively. The model will eventually return
malformed JSON, and the system needs to handle that without dropping the request.
- Cost. Set a per-tenant budget and a circuit breaker. A runaway loop
can spend a month of revenue in an afternoon.
- Observability. Log the prompt, the output, and the cost of every
call. You cannot debug what you cannot see.
- Fallbacks. When the model is unavailable, the feature should
degrade, not fail closed.
The honest takeaway
The interesting part of an AI feature is rarely the model. It is everything around the model. That is also where the work gets done.