Software Engineering for AI Platforms: The Non-Negotiable Baseline
Software engineering for AI platforms is the discipline of applying ordinary, rigorous software engineering to systems whose core component is nondeterministic. An AI platform is still software: it has interfaces, dependencies, state, configuration, builds, releases, tests and failure modes. The novelty is a model that can return a different answer to the same input, call tools, and fail in ways a type checker will never catch. This article covers the engineering baseline an AI architect must insist on — code structure, interface contracts, configuration and secrets, dependency and artifact management, testing a probabilistic system, CI/CD with evaluation gates, reproducibility, and the review discipline that keeps a fast-moving AI codebase from rotting.
The whole domain reduces to one uncomfortable sentence:
Most AI incidents are not model failures. They are ordinary software-engineering failures — an unpinned dependency, an untested edge case, a secret in a log, a config change with no rollback — wearing an AI costume.
A model you cannot test, a prompt you cannot version, a pipeline you cannot reproduce and a deployment you cannot roll back are engineering problems, not research problems. This is the domain that decides whether everything else in the handbook survives contact with production.
17.0 Architect-level mental model
The mental model is simple: an AI platform is a software system with a probabilistic subsystem inside it. Everything that is true of good software systems is still true. What changes is that one dependency — the model — is not deterministic, not version-stable, not fully testable, and not owned by you.
So the architect's job is to wrap that probabilistic core in deterministic engineering:
| Concern | Deterministic part | Probabilistic part |
|---|---|---|
| Interfaces | API contracts, schemas, types | Model input/output content |
| Testing | Unit, integration, contract tests | Evaluation suites, statistical gates |
| Config | Versioned, environment-separated | Prompts, model choice, routing |
| Releases | Immutable builds, canary, rollback | Prompt/model promotion, shadow runs |
| Reliability | Timeouts, retries, circuit breakers | Verification, fallbacks, refusal paths |
| Reproducibility | Pinned dependencies, lockfiles, SBOM | Pinned model version + decoding params |
17.1 Why does AI code rot faster than normal code?
AI code rots faster because three things change underneath it at once, and none of them are compiled.
In a normal service, behaviour changes when someone changes code. In an AI platform, behaviour can change when:
someone edits a prompt string
the provider silently updates the model
a retrieved document changes
a dependency upgrades a tokenizer
a temperature default changes
an environment variable drifts between environmentsNone of those trip a failing build. The code still compiles, the service still returns HTTP 200, and the behaviour is now different. This is why the baseline matters more for AI, not less: the compiler and the type system protect you from a smaller fraction of the real failure surface, so explicit engineering discipline has to cover the rest.
The practical consequence: every behaviour-changing input — prompt, model id, decoding parameters, retrieval config — must be treated as versioned code, not as runtime trivia.
17.2 What does "AI systems are software first" actually require?
It requires that the AI parts live inside a normal, boring, well-structured application — not that a notebook gets a REST wrapper.
A production AI platform should have the same structural properties you would demand of any serious service:
clear module boundaries
a dependency-inversion boundary around the model
typed interfaces between layers
no business logic inside prompts
no prompts inlined inside business logic
configuration separated from code
tests that run in CI
observable, structured logs
a documented build and release processThe single most common architectural smell is the model call reaching everywhere. If model.generate() is called from twelve files, you cannot swap providers, add caching, enforce policy, or instrument calls consistently. The fix is a single boundary.
17.3 The model gateway as a code boundary
Put every model call behind one interface. This is the same model gateway that handles routing and fallbacks, viewed from the code side.
Application code
│
▼
LLMClient (interface)
│ generate(request) -> response
▼
Gateway implementation
├── provider adapters
├── retries / timeouts
├── token + cost accounting
├── prompt/version resolution
├── policy hooks
└── telemetryWhat this boundary buys you, as pure engineering:
swap providers without touching business logic
mock the model in tests deterministically
enforce timeouts and budgets in one place
inject tracing consistently
add caching transparently
version the contract independently of callersIf an architect inherits an AI codebase and can find no such boundary, that is the first thing to fix, before any model or prompt work.
17.4 How do you design interfaces for AI components?
Design them so the nondeterministic content is the only thing that varies — everything around it is a stable, typed contract.
A good model-call contract separates four things:
1. The request envelope (typed, stable)
2. The prompt/template (versioned artifact)
3. The model config (versioned: model id, params)
4. The response envelope (typed, validated)Concretely, the response should always be validated against a schema before it enters the rest of the system:
model returns text
↓
parse into declared schema (JSON schema / typed object)
↓
validate (required fields, enums, ranges)
↓
on failure: repair, retry, or route to fallback
↓
only valid, typed objects cross into business logicThe rule is: never let raw model text flow directly into business logic. The boundary where free text becomes a validated typed object is one of the most important lines in the system — it is also where structured-output and prompt-contract design pays off.
17.5 Configuration: prompts and model choice are config, not code
Treat prompts, model ids, decoding parameters, retrieval settings and feature flags as versioned configuration that is deployed, promoted and rolled back — not as literals buried in source.
The anti-pattern:
response = client.generate("gpt-4o", 0.7,
"You are a helpful assistant. Summarise: " + text)Three behaviour-changing decisions (model, temperature, prompt) are now welded into a code path, invisible to anyone reviewing behaviour, and impossible to change without a code deploy.
The baseline:
config/
prompts/summarise.v17.txt
models/summariser.yaml # model id, params, fallback
routing/summariser.yaml # which prompt+model alias is "production"This connects directly to LLMOps versioning: the registry and aliases described there are only enforceable if the application reads prompts and model choice from versioned configuration in the first place.
The twelve-factor principle still applies
The twelve-factor principle of strict separation of config from code predates LLMs and applies cleanly: config varies between deploys, code does not. For AI platforms, extend "config" to include every behaviour-changing artifact. (The Twelve-Factor App)
17.6 Secrets and credentials
The fastest way to turn an AI feature into a security incident is to let the model, the logs or the prompt touch a secret.
Three rules an architect should enforce without exception:
1. No secret in code, prompt, or config committed to git.
2. No secret in a prompt sent to a model provider.
3. No secret in a trace, log, or captured completion.Secrets live in a vault or secret manager and are injected at runtime with least privilege. The agent or application authenticates to tools; it does not carry raw production credentials in its context. This is the engineering side of the AI security control map — the control only works if the code never puts the secret where the model can see it.
Vault / secret manager
│ runtime injection, scoped, short-lived
▼
Application / tool adapter ← holds credential briefly
│
▼
Model context ← NEVER contains the credential17.7 Dependency management and the AI supply chain
AI platforms pull in unusually heavy, fast-moving dependencies — SDKs, tokenizers, vector clients, orchestration frameworks, model weights — and each one is attack surface and drift surface.
Minimum discipline:
pin exact versions (lockfiles), no floating ranges in production
reproducible installs from the lockfile
generate an SBOM (software bill of materials)
scan dependencies for known vulnerabilities in CI
review transitive dependencies for orchestration frameworks
treat downloaded model weights as untrusted artifacts with provenanceTwo AI-specific hazards deserve naming. First, a tokenizer or SDK minor-version bump can silently change tokenization or defaults, which changes behaviour and cost. Second, model weights and datasets are supply-chain artifacts — they should have a known source, an integrity check, and a provenance record, exactly like any binary dependency. Frameworks such as SLSA exist to describe exactly this kind of build and artifact provenance. (SLSA)
17.8 How do you test a system that is nondeterministic?
You test it in layers, and you stop expecting any single layer to prove the whole system is correct. The deterministic scaffolding is tested like any software; the model output is evaluated statistically, not asserted exactly.
The test pyramid for an AI platform:
| Layer | What it tests | Deterministic? |
|---|---|---|
| Unit tests | Parsing, validation, routing, tool adapters, config | Yes |
| Contract tests | Request/response schemas, tool I/O schemas | Yes |
| Integration tests | Wiring, retries, timeouts, fallbacks (model mocked) | Yes |
| Evaluation suites | Model/prompt/agent output quality on a dataset | No (scored) |
| Adversarial/red-team | Injection, jailbreaks, unsafe outputs, leakage | No (scored) |
| Canary / shadow | Real traffic behaviour before full rollout | No (observed) |
The non-deterministic layers belong to evaluation, and the key engineering requirement is that evaluation runs in CI like a test suite, not as an occasional manual exercise.
17.9 Deterministic testing: mock the model
The deterministic layers are only cheap and fast if the model is mocked. The model gateway boundary (17.3) is what makes this possible.
test → LLMClient (fake) → returns canned/recorded response
↓
exercise parsing, validation, routing,
retry logic, fallback logic, tool dispatch,
error handling, cost accountingThis lets you test the parts that must never break — "what happens when the model returns malformed JSON?", "what happens when the tool times out?", "what happens when the model asks to call a tool that is not allowed?" — deterministically, in milliseconds, on every commit.
A useful heuristic: every failure mode that does not depend on model intelligence should have a deterministic test. Malformed output, empty output, oversized output, disallowed tool, timeout, rate limit, provider error, schema drift — all of these are ordinary tests.
17.10 Evaluation as a test gate
Evaluation is the AI-specific test layer, and the engineering requirement is that it behaves like a gate, not a report.
PR opened
↓
unit + contract + integration tests → must pass
↓
evaluation suite on golden dataset → must meet thresholds
↓
adversarial/safety suite → must meet thresholds
↓
merge allowed
↓
canary with online evaluation → promote or roll backThe thresholds must be explicit and versioned, because — as the LLMOps domain stresses — a score is meaningless without the version of the evaluator that produced it. An evaluation gate that nobody can fail is theatre; the architect's job is to make sure a genuine quality regression actually blocks a merge or a promotion.
17.11 What belongs in CI/CD for an AI platform?
CI/CD for an AI platform is ordinary CI/CD plus two AI-specific additions: evaluation gates and independent promotion of behaviour-changing artifacts.
A reference pipeline:
COMMIT
├── lint / type check
├── unit tests
├── contract tests
├── dependency + secret scan
├── build immutable artifact
└── SBOM
PRE-MERGE
├── integration tests (model mocked)
├── evaluation suite (golden dataset)
└── safety / adversarial suite
DEPLOY
├── deploy to staging
├── smoke + shadow traffic
├── canary (small % real traffic)
├── online evaluation on canary
└── promote or auto-rollback
POST-DEPLOY
├── monitor quality + cost + latency
└── feedback loop into eval datasetThe deployable unit should be immutable and completely described — application version plus the full AI configuration bundle (prompt versions, model ids, retrieval config, policy pack, eval gate). This is the "AI deployment manifest" idea from LLMOps, enforced at the pipeline level.
17.12 Independent deployability of behaviour-changing artifacts
A prompt change should not require a full application redeploy, and a model swap should not require a code change. Each behaviour-changing artifact gets its own promotion path and its own rollback.
App code → normal deploy pipeline
Prompt → prompt registry, alias promotion, instant rollback
Model choice → config/routing change, canary, instant rollback
Retrieval config→ versioned, canary
Policy pack → versioned, canaryWhy this matters as engineering: the smaller and more independent the unit of change, the smaller the blast radius and the faster the rollback. If the only unit of change is "redeploy the whole platform," every prompt tweak carries the risk of a full deployment, and rollback is slow precisely when you need it fast.
17.13 Reproducibility: can you rebuild exactly what ran?
Reproducibility means that for any production response, you can reconstruct the exact code, dependencies, prompt, model, parameters and data that produced it. Without it, root-cause analysis becomes guesswork.
The reproducibility checklist:
code commit SHA
dependency lockfile hash / SBOM
container image digest (not a mutable tag)
prompt version
model id + provider + region
decoding parameters (temperature, top_p, max tokens, seed if available)
retrieval config + index version
policy version
feature-flag stateTwo AI-specific caveats the architect must state honestly:
- Provider-hosted models are not fully reproducible. Even with a pinned model id, the provider can change behaviour, and most APIs do not guarantee bit-identical output. A
seedparameter, where offered, reduces but does not eliminate variance. Reproducibility here means "same configuration," not "same bytes." - Mutable image tags break reproducibility.
:latestis not a version. Deploy by digest.
17.14 Error handling for probabilistic systems
AI platforms fail in more ways than normal services, and the engineering must treat each as a first-class, handled path rather than an exception that bubbles up as a 500.
A non-exhaustive taxonomy the code must handle:
malformed / unparseable output
schema-valid but wrong output
empty or truncated output (hit max tokens)
model refusal
disallowed tool request
tool failure / timeout
provider rate limit (429)
provider outage
latency SLO breach
cost budget breach
injection / unsafe content detectedEach needs a defined response: repair, retry with backoff, route to fallback model, degrade gracefully, refuse safely, or escalate to a human. A retry policy for AI calls must be especially careful, because retries amplify latency, token cost and provider throttling — a retry storm against a rate-limited model is self-amplifying. Bound retries, use jittered backoff, and make tool calls idempotent so a retry cannot double-execute a side effect.
17.15 Idempotency and side effects
Because models can be retried and agents can loop, any action with a side effect must be idempotent or guarded.
agent proposes: "create purchase order PO-5567"
↓
idempotency key = hash(intent + business key)
↓
tool checks: has this key already executed?
├── yes → return prior result, do not re-execute
└── no → execute, record key, return resultThis is ordinary distributed-systems hygiene, but it is non-optional for agents, because the combination of retries, loops and nondeterminism makes accidental double-execution far more likely than in a deterministic workflow.
17.16 Code structure: where do prompts live?
Prompts are versioned assets with owners and tests — so they live in their own place in the repository (or a registry), not scattered as string literals.
A workable layout:
src/
domain/ # business logic, no model calls
ai/
client/ # the LLMClient boundary + adapters
prompts/ # versioned prompt templates
schemas/ # request/response/tool schemas
evaluation/ # eval datasets + suites
policies/ # guardrails, allowlists
api/ # transport layer
config/
models/ # model ids, params, fallbacks
routing/ # prompt+model aliases per environmentThe architectural rule behind the layout: business logic should not know which model it is using, and prompts should not contain business logic. When those two leak into each other, the system becomes impossible to test, review or evolve.
17.17 Review discipline for AI changes
A prompt change is a behaviour change and deserves the same review rigour as a code change — arguably more, because its effects are harder to predict and invisible to the compiler.
What a reviewer of an AI change should see in the PR:
the diff (prompt / model / config / code)
the evaluation results before and after
the cost and latency impact
the safety / adversarial results
the rollback plan (what alias reverts to what)"It looked better in my testing" is not a reviewable claim. "Groundedness went from 91% to 94% on the golden set, cost per task unchanged, safety suite still passing, rollback is prompt alias → v16" is. The engineering goal is to make AI changes reviewable on evidence, not on impression.
17.18 Documentation and architecture decision records
Fast-moving AI codebases accumulate undocumented decisions — why this model, why this chunk size, why this retry policy — and lose the reasoning within weeks. Architecture Decision Records (ADRs) are the cheap fix.
Record, at minimum:
decision (what was chosen)
context (what problem, what constraints)
alternatives considered
consequences / trade-offs accepted
how to revisit (what would change this decision)For AI platforms the highest-value ADRs cover model strategy, build-vs-buy, autonomy boundaries, data and retrieval design, and evaluation thresholds — exactly the decisions that are expensive to reverse and easy to forget.
17.19 Environments and promotion
An AI platform needs the same environment separation as any serious system — and the model configuration must be part of what differs between environments, not an afterthought.
dev → cheap models allowed, verbose capture, synthetic data
staging → production-like config, shadow traffic, full eval
prod → pinned models, strict capture/redaction, canary + rollbackA frequent, expensive mistake: testing against one model configuration and deploying another. If staging uses a different model, prompt or temperature than production, staging proves very little. Promote configuration, not just code, and keep the difference between environments explicit and reviewed.
17.20 The engineering baseline checklist
An architect can assess an AI platform's engineering maturity with this checklist. It maps one-to-one to the sections above.
[ ] All model calls go through one gateway boundary
[ ] Prompts, model ids and params are versioned config, not literals
[ ] Raw model output is validated against a schema before use
[ ] No secrets in code, prompts, config, or logs
[ ] Dependencies pinned; SBOM generated; vulnerabilities scanned
[ ] Model weights/datasets have provenance and integrity checks
[ ] Deterministic failure modes have deterministic tests (model mocked)
[ ] Evaluation and safety suites run in CI as gates with versioned thresholds
[ ] Builds are immutable and deployed by digest, not by mutable tag
[ ] Every behaviour-changing artifact is independently deployable and rollbackable
[ ] Any production response can be reproduced from recorded versions
[ ] Side-effecting actions are idempotent; retries are bounded with backoff
[ ] AI changes are reviewed on evaluation evidence, not impression
[ ] Environments differ in a reviewed, explicit way (config promoted, not just code)
[ ] Key AI decisions are captured in ADRsIf a platform fails half of these, the model quality is almost irrelevant — the system will fail operationally before the model gets a chance to.
17.21 Common mistakes
| Mistake | Consequence |
|---|---|
| Prompts as inline string literals | Behaviour changes are invisible and unreviewable |
| Model called from everywhere | Cannot swap, mock, cache, or instrument |
| Raw model text into business logic | Silent corruption, injection, type errors |
| No deterministic tests ("we'll just eval") | Ordinary bugs ship; eval is slow and flaky as a net |
| Floating dependency versions | Silent behaviour and cost drift |
:latest image tags | Non-reproducible, un-rollbackable deploys |
| Secrets in prompts or logs | Credential leakage to provider or log store |
| Unbounded retries / non-idempotent tools | Retry storms, double-executed side effects |
| Staging config ≠ production config | Staging proves nothing |
| Evaluation as an occasional manual check | Regressions reach production undetected |
17.22 FAQ: Are AI systems really just software?
Mostly, yes — and the parts that are not are smaller than teams assume. An AI platform is a normal software system with one nondeterministic subsystem. The interfaces, dependencies, configuration, builds, tests, deployments and failure handling are ordinary software engineering, and that is where the majority of real incidents originate. The genuinely novel part is that the model can return different outputs for the same input, is version-unstable, is not fully testable, and is often operated by a third party. The architect's job is to apply standard engineering rigour to everything around the model and reserve the special techniques — evaluation, verification, guardrails — for the model itself.
17.23 FAQ: How do you unit-test an LLM feature?
You unit-test everything around the model deterministically by mocking the model at the gateway boundary. Tests cover parsing, schema validation, routing, tool dispatch, retry and fallback logic, error handling, and cost accounting — using canned or recorded model responses so the tests are fast and repeatable. The model's actual output quality is not unit-tested; it is measured by evaluation suites on a dataset, with statistical thresholds, and those suites run in CI as a separate gate. The division is: deterministic logic gets deterministic tests, probabilistic output gets scored evaluation.
17.24 FAQ: What is an AI deployment manifest?
It is an immutable, complete description of everything that defines a deployment's behaviour, so the deployment is reproducible and rollbackable as a unit. It records the application version, the prompt versions, the primary and fallback model ids and parameters, the retrieval and reranker configuration, the policy pack version, and the evaluation gate that approved it. Because each of those is also independently versioned, you can roll back a single component — for example moving the production prompt alias from v17 to v16 — rather than redeploying the whole platform when one artifact causes an incident.
17.25 FAQ: Can AI systems be fully reproducible?
The software around the model can be fully reproducible; the model itself usually cannot. You can pin the code commit, the dependency lockfile, the container image digest, the prompt version, the model id, the decoding parameters, the retrieval index version and the policy version — which lets you reconstruct the exact configuration that ran. But provider-hosted models can change behaviour over time and rarely guarantee bit-identical output even with a fixed id and seed. So reproducibility for AI means "same configuration and same inputs," which is enough for root-cause analysis and regression testing, rather than "identical bytes," which is generally not achievable with external models.
17.26 FAQ: Why not just put prompts in the code?
Because a prompt is a behaviour-changing artifact, and burying it in code makes behaviour changes invisible, unversioned, unreviewable and un-rollbackable without a full deploy. When prompts live as versioned configuration or in a registry, you get independent promotion and rollback, evaluation tied to each version, clear ownership, and the ability to change behaviour without shipping code. The same argument applies to model ids and decoding parameters: they decide behaviour, so they are configuration, and configuration is separated from code.
17.27 What you should know cold
- An AI platform is software with a probabilistic subsystem; apply full engineering rigour everywhere except the model's output, and use evaluation for that.
- Put every model call behind one gateway boundary so you can swap, mock, cache, instrument and govern in one place.
- Treat prompts, model ids and decoding parameters as versioned config, never as code literals; validate all model output against a schema before it enters business logic.
- Test deterministic logic deterministically by mocking the model; run evaluation and safety suites in CI as gates with versioned thresholds.
- Make builds immutable, deploy by digest, keep every behaviour-changing artifact independently rollbackable, and keep enough version records that any production response can be reproduced in configuration.
The one sentence to remember
"Most AI incidents are ordinary engineering failures — unpinned dependencies, untested edge cases, leaked secrets, un-rollbackable config — so I apply standard software discipline to everything around the model, and reserve evaluation and verification for the model itself."
That is the engineering baseline a platform has to meet before any of the model work is worth doing.
Related reading
- Cloud Architecture for AI Workloads: Isolation, Identity, Networking and Serving, where this deployable, reproducible software actually runs.
- LLMOps and Observability: Tracing, Metrics, Drift and Feedback Loops, the versioning, rollback and evaluation-gate machinery this baseline makes possible.
- Evaluating LLM, RAG and Agent Systems, the test layer that replaces exact assertions for model output.
- The Enterprise AI Architect's Handbook, the full series this domain belongs to.
Part of the series
The Enterprise AI Architect's Handbook- 1.The Enterprise AI Architect Roadmap: The 29 Domains the Role Actually Owns
- 2.The AI Architect Operating Model: Turning a Business Objective into an Architecture
- 3.LLM Fundamentals for Architects: Tokens, Context, Latency, Throughput and Cost
- 4.Prompt and Context Engineering as an Architectural Concern
- 5.RAG Architecture: The Full Pipeline and Where Each Stage Fails
- 6.Knowledge Architecture: Ontologies, Entity Resolution and Graph Retrieval
- 7.Agent Architecture: Loops, Planning, Verification and Termination
- 8.Agent State and Memory Architecture: Scoping, Retention and Provenance
- 9.Multi-Agent Systems: When They Help, and How They Fail
- 10.Agent Orchestration: Frameworks, Durable Execution and Framework-Independent Design
- 11.MCP Architecture and the Enterprise Tool Gateway
- 12.Model Strategy: Selection, Gateways, Routing and Fallbacks
- 13.Fine-Tuning, RAG or Prompting: How an Architect Decides
- 14.Evaluating LLM, RAG and Agent Systems: Metrics, Judges and Quality Gates
- 15.LLMOps and Observability: Tracing, Metrics, Drift and Feedback Loops
- 16.AI Security: The Full Threat and Control Map for Architects
- 17.Responsible AI, Privacy and Governance as Architecture, Not Paperwork
- 18.Software Engineering for AI Platforms: The Non-Negotiable Baseline← you are here
- 19.Cloud Architecture for AI Workloads: Isolation, Identity, Networking and Serving
- 20.Containers, Infrastructure as Code and Delivery for AI Systemscoming soon
- 21.Cost and Performance Architecture: Designing for Cost per Successful Taskcoming soon
- 22.Reliability and Resilience: The Twenty Failure Modes of AI Systemscoming soon
- 23.Enterprise AI Platform Architecture: Control Plane and Runtime Planecoming soon
- 24.Production and Launch Readiness for AI Systemscoming soon
- 25.Domain Architecture: Applying the Model to a Real Business Functioncoming soon
- 26.AI System Design Practice: Fifteen Problems and How to Approach Themcoming soon
- 27.Architecture Artefacts: The Diagrams an AI Architect Must Be Able to Drawcoming soon
- 28.Structured Answers: System Design, Trade-offs, Incidents and Reviewscoming soon
- 29.Experience Narratives: The Stories an Architect Must Be Able to Tellcoming soon
- 30.Architecture Leadership and Technical Strategycoming soon

Aakash Ahuja
Enterprise AI, Cybersecurity & Platform Engineering
Aakash writes about secure AI agents, microservices architecture, enterprise platforms, and production engineering. He has 20+ years of experience building and operating software systems across banking, cloud, cybersecurity, AI, and enterprise workflow automation. He is Director of Technology at itmtb Technologies and teaches AI, Big Data, and Reinforcement Learning at top institutes in India.