Software Engineering for AI Platforms: The Non-Negotiable Baseline

By Aakash Ahuja··22 min read

Software engineering for AI platforms is the discipline of applying ordinary, rigorous software engineering to systems whose core component is nondeterministic. An AI platform is still software: it has interfaces, dependencies, state, configuration, builds, releases, tests and failure modes. The novelty is a model that can return a different answer to the same input, call tools, and fail in ways a type checker will never catch. This article covers the engineering baseline an AI architect must insist on — code structure, interface contracts, configuration and secrets, dependency and artifact management, testing a probabilistic system, CI/CD with evaluation gates, reproducibility, and the review discipline that keeps a fast-moving AI codebase from rotting.

The whole domain reduces to one uncomfortable sentence:

Most AI incidents are not model failures. They are ordinary software-engineering failures — an unpinned dependency, an untested edge case, a secret in a log, a config change with no rollback — wearing an AI costume.

A model you cannot test, a prompt you cannot version, a pipeline you cannot reproduce and a deployment you cannot roll back are engineering problems, not research problems. This is the domain that decides whether everything else in the handbook survives contact with production.


17.0 Architect-level mental model

The mental model is simple: an AI platform is a software system with a probabilistic subsystem inside it. Everything that is true of good software systems is still true. What changes is that one dependency — the model — is not deterministic, not version-stable, not fully testable, and not owned by you.

So the architect's job is to wrap that probabilistic core in deterministic engineering:

ConcernDeterministic partProbabilistic part
InterfacesAPI contracts, schemas, typesModel input/output content
TestingUnit, integration, contract testsEvaluation suites, statistical gates
ConfigVersioned, environment-separatedPrompts, model choice, routing
ReleasesImmutable builds, canary, rollbackPrompt/model promotion, shadow runs
ReliabilityTimeouts, retries, circuit breakersVerification, fallbacks, refusal paths
ReproducibilityPinned dependencies, lockfiles, SBOMPinned model version + decoding params
Everything in the left column is textbook software engineering that AI teams routinely skip because "it's just a prototype." Everything in the right column is the subject of other domains in this series. This domain is about refusing to let the left column lapse.


17.1 Why does AI code rot faster than normal code?

AI code rots faster because three things change underneath it at once, and none of them are compiled.

In a normal service, behaviour changes when someone changes code. In an AI platform, behaviour can change when:

someone edits a prompt string
the provider silently updates the model
a retrieved document changes
a dependency upgrades a tokenizer
a temperature default changes
an environment variable drifts between environments

None of those trip a failing build. The code still compiles, the service still returns HTTP 200, and the behaviour is now different. This is why the baseline matters more for AI, not less: the compiler and the type system protect you from a smaller fraction of the real failure surface, so explicit engineering discipline has to cover the rest.

The practical consequence: every behaviour-changing input — prompt, model id, decoding parameters, retrieval config — must be treated as versioned code, not as runtime trivia.


17.2 What does "AI systems are software first" actually require?

It requires that the AI parts live inside a normal, boring, well-structured application — not that a notebook gets a REST wrapper.

A production AI platform should have the same structural properties you would demand of any serious service:

clear module boundaries
a dependency-inversion boundary around the model
typed interfaces between layers
no business logic inside prompts
no prompts inlined inside business logic
configuration separated from code
tests that run in CI
observable, structured logs
a documented build and release process

The single most common architectural smell is the model call reaching everywhere. If model.generate() is called from twelve files, you cannot swap providers, add caching, enforce policy, or instrument calls consistently. The fix is a single boundary.


17.3 The model gateway as a code boundary

Put every model call behind one interface. This is the same model gateway that handles routing and fallbacks, viewed from the code side.

Application code
      │
      ▼
  LLMClient (interface)
      │  generate(request) -> response
      ▼
  Gateway implementation
      ├── provider adapters
      ├── retries / timeouts
      ├── token + cost accounting
      ├── prompt/version resolution
      ├── policy hooks
      └── telemetry

What this boundary buys you, as pure engineering:

swap providers without touching business logic
mock the model in tests deterministically
enforce timeouts and budgets in one place
inject tracing consistently
add caching transparently
version the contract independently of callers

If an architect inherits an AI codebase and can find no such boundary, that is the first thing to fix, before any model or prompt work.


17.4 How do you design interfaces for AI components?

Design them so the nondeterministic content is the only thing that varies — everything around it is a stable, typed contract.

A good model-call contract separates four things:

1. The request envelope   (typed, stable)
2. The prompt/template     (versioned artifact)
3. The model config        (versioned: model id, params)
4. The response envelope   (typed, validated)

Concretely, the response should always be validated against a schema before it enters the rest of the system:

model returns text
      ↓
parse into declared schema (JSON schema / typed object)
      ↓
validate (required fields, enums, ranges)
      ↓
on failure: repair, retry, or route to fallback
      ↓
only valid, typed objects cross into business logic

The rule is: never let raw model text flow directly into business logic. The boundary where free text becomes a validated typed object is one of the most important lines in the system — it is also where structured-output and prompt-contract design pays off.


17.5 Configuration: prompts and model choice are config, not code

Treat prompts, model ids, decoding parameters, retrieval settings and feature flags as versioned configuration that is deployed, promoted and rolled back — not as literals buried in source.

The anti-pattern:

response = client.generate("gpt-4o", 0.7,
    "You are a helpful assistant. Summarise: " + text)

Three behaviour-changing decisions (model, temperature, prompt) are now welded into a code path, invisible to anyone reviewing behaviour, and impossible to change without a code deploy.

The baseline:

config/
  prompts/summarise.v17.txt
  models/summariser.yaml      # model id, params, fallback
  routing/summariser.yaml     # which prompt+model alias is "production"

This connects directly to LLMOps versioning: the registry and aliases described there are only enforceable if the application reads prompts and model choice from versioned configuration in the first place.

The twelve-factor principle still applies

The twelve-factor principle of strict separation of config from code predates LLMs and applies cleanly: config varies between deploys, code does not. For AI platforms, extend "config" to include every behaviour-changing artifact. (The Twelve-Factor App)


17.6 Secrets and credentials

The fastest way to turn an AI feature into a security incident is to let the model, the logs or the prompt touch a secret.

Three rules an architect should enforce without exception:

1. No secret in code, prompt, or config committed to git.
2. No secret in a prompt sent to a model provider.
3. No secret in a trace, log, or captured completion.

Secrets live in a vault or secret manager and are injected at runtime with least privilege. The agent or application authenticates to tools; it does not carry raw production credentials in its context. This is the engineering side of the AI security control map — the control only works if the code never puts the secret where the model can see it.

Vault / secret manager
      │  runtime injection, scoped, short-lived
      ▼
Application / tool adapter   ← holds credential briefly
      │
      ▼
Model context                ← NEVER contains the credential

17.7 Dependency management and the AI supply chain

AI platforms pull in unusually heavy, fast-moving dependencies — SDKs, tokenizers, vector clients, orchestration frameworks, model weights — and each one is attack surface and drift surface.

Minimum discipline:

pin exact versions (lockfiles), no floating ranges in production
reproducible installs from the lockfile
generate an SBOM (software bill of materials)
scan dependencies for known vulnerabilities in CI
review transitive dependencies for orchestration frameworks
treat downloaded model weights as untrusted artifacts with provenance

Two AI-specific hazards deserve naming. First, a tokenizer or SDK minor-version bump can silently change tokenization or defaults, which changes behaviour and cost. Second, model weights and datasets are supply-chain artifacts — they should have a known source, an integrity check, and a provenance record, exactly like any binary dependency. Frameworks such as SLSA exist to describe exactly this kind of build and artifact provenance. (SLSA)


17.8 How do you test a system that is nondeterministic?

You test it in layers, and you stop expecting any single layer to prove the whole system is correct. The deterministic scaffolding is tested like any software; the model output is evaluated statistically, not asserted exactly.

The test pyramid for an AI platform:

LayerWhat it testsDeterministic?
Unit testsParsing, validation, routing, tool adapters, configYes
Contract testsRequest/response schemas, tool I/O schemasYes
Integration testsWiring, retries, timeouts, fallbacks (model mocked)Yes
Evaluation suitesModel/prompt/agent output quality on a datasetNo (scored)
Adversarial/red-teamInjection, jailbreaks, unsafe outputs, leakageNo (scored)
Canary / shadowReal traffic behaviour before full rolloutNo (observed)
The mistake to avoid is collapsing the pyramid into "we ran the demo and it looked good." The deterministic layers (the bottom three) should be as rigorous as in any backend service, because they are fully testable and they are where most real bugs live.

The non-deterministic layers belong to evaluation, and the key engineering requirement is that evaluation runs in CI like a test suite, not as an occasional manual exercise.


17.9 Deterministic testing: mock the model

The deterministic layers are only cheap and fast if the model is mocked. The model gateway boundary (17.3) is what makes this possible.

test → LLMClient (fake) → returns canned/recorded response
                           ↓
            exercise parsing, validation, routing,
            retry logic, fallback logic, tool dispatch,
            error handling, cost accounting

This lets you test the parts that must never break — "what happens when the model returns malformed JSON?", "what happens when the tool times out?", "what happens when the model asks to call a tool that is not allowed?" — deterministically, in milliseconds, on every commit.

A useful heuristic: every failure mode that does not depend on model intelligence should have a deterministic test. Malformed output, empty output, oversized output, disallowed tool, timeout, rate limit, provider error, schema drift — all of these are ordinary tests.


17.10 Evaluation as a test gate

Evaluation is the AI-specific test layer, and the engineering requirement is that it behaves like a gate, not a report.

PR opened
   ↓
unit + contract + integration tests  → must pass
   ↓
evaluation suite on golden dataset   → must meet thresholds
   ↓
adversarial/safety suite             → must meet thresholds
   ↓
merge allowed
   ↓
canary with online evaluation        → promote or roll back

The thresholds must be explicit and versioned, because — as the LLMOps domain stresses — a score is meaningless without the version of the evaluator that produced it. An evaluation gate that nobody can fail is theatre; the architect's job is to make sure a genuine quality regression actually blocks a merge or a promotion.


17.11 What belongs in CI/CD for an AI platform?

CI/CD for an AI platform is ordinary CI/CD plus two AI-specific additions: evaluation gates and independent promotion of behaviour-changing artifacts.

A reference pipeline:

COMMIT
  ├── lint / type check
  ├── unit tests
  ├── contract tests
  ├── dependency + secret scan
  ├── build immutable artifact
  └── SBOM

PRE-MERGE
  ├── integration tests (model mocked)
  ├── evaluation suite (golden dataset)
  └── safety / adversarial suite

DEPLOY
  ├── deploy to staging
  ├── smoke + shadow traffic
  ├── canary (small % real traffic)
  ├── online evaluation on canary
  └── promote or auto-rollback

POST-DEPLOY
  ├── monitor quality + cost + latency
  └── feedback loop into eval dataset

The deployable unit should be immutable and completely described — application version plus the full AI configuration bundle (prompt versions, model ids, retrieval config, policy pack, eval gate). This is the "AI deployment manifest" idea from LLMOps, enforced at the pipeline level.


17.12 Independent deployability of behaviour-changing artifacts

A prompt change should not require a full application redeploy, and a model swap should not require a code change. Each behaviour-changing artifact gets its own promotion path and its own rollback.

App code        → normal deploy pipeline
Prompt          → prompt registry, alias promotion, instant rollback
Model choice    → config/routing change, canary, instant rollback
Retrieval config→ versioned, canary
Policy pack     → versioned, canary

Why this matters as engineering: the smaller and more independent the unit of change, the smaller the blast radius and the faster the rollback. If the only unit of change is "redeploy the whole platform," every prompt tweak carries the risk of a full deployment, and rollback is slow precisely when you need it fast.


17.13 Reproducibility: can you rebuild exactly what ran?

Reproducibility means that for any production response, you can reconstruct the exact code, dependencies, prompt, model, parameters and data that produced it. Without it, root-cause analysis becomes guesswork.

The reproducibility checklist:

code commit SHA
dependency lockfile hash / SBOM
container image digest (not a mutable tag)
prompt version
model id + provider + region
decoding parameters (temperature, top_p, max tokens, seed if available)
retrieval config + index version
policy version
feature-flag state

Two AI-specific caveats the architect must state honestly:

  1. Provider-hosted models are not fully reproducible. Even with a pinned model id, the provider can change behaviour, and most APIs do not guarantee bit-identical output. A seed parameter, where offered, reduces but does not eliminate variance. Reproducibility here means "same configuration," not "same bytes."
  2. Mutable image tags break reproducibility. :latest is not a version. Deploy by digest.

17.14 Error handling for probabilistic systems

AI platforms fail in more ways than normal services, and the engineering must treat each as a first-class, handled path rather than an exception that bubbles up as a 500.

A non-exhaustive taxonomy the code must handle:

malformed / unparseable output
schema-valid but wrong output
empty or truncated output (hit max tokens)
model refusal
disallowed tool request
tool failure / timeout
provider rate limit (429)
provider outage
latency SLO breach
cost budget breach
injection / unsafe content detected

Each needs a defined response: repair, retry with backoff, route to fallback model, degrade gracefully, refuse safely, or escalate to a human. A retry policy for AI calls must be especially careful, because retries amplify latency, token cost and provider throttling — a retry storm against a rate-limited model is self-amplifying. Bound retries, use jittered backoff, and make tool calls idempotent so a retry cannot double-execute a side effect.


17.15 Idempotency and side effects

Because models can be retried and agents can loop, any action with a side effect must be idempotent or guarded.

agent proposes: "create purchase order PO-5567"
      ↓
idempotency key = hash(intent + business key)
      ↓
tool checks: has this key already executed?
      ├── yes → return prior result, do not re-execute
      └── no  → execute, record key, return result

This is ordinary distributed-systems hygiene, but it is non-optional for agents, because the combination of retries, loops and nondeterminism makes accidental double-execution far more likely than in a deterministic workflow.


17.16 Code structure: where do prompts live?

Prompts are versioned assets with owners and tests — so they live in their own place in the repository (or a registry), not scattered as string literals.

A workable layout:

src/
  domain/            # business logic, no model calls
  ai/
    client/          # the LLMClient boundary + adapters
    prompts/         # versioned prompt templates
    schemas/         # request/response/tool schemas
    evaluation/      # eval datasets + suites
    policies/        # guardrails, allowlists
  api/               # transport layer
config/
  models/            # model ids, params, fallbacks
  routing/           # prompt+model aliases per environment

The architectural rule behind the layout: business logic should not know which model it is using, and prompts should not contain business logic. When those two leak into each other, the system becomes impossible to test, review or evolve.


17.17 Review discipline for AI changes

A prompt change is a behaviour change and deserves the same review rigour as a code change — arguably more, because its effects are harder to predict and invisible to the compiler.

What a reviewer of an AI change should see in the PR:

the diff (prompt / model / config / code)
the evaluation results before and after
the cost and latency impact
the safety / adversarial results
the rollback plan (what alias reverts to what)

"It looked better in my testing" is not a reviewable claim. "Groundedness went from 91% to 94% on the golden set, cost per task unchanged, safety suite still passing, rollback is prompt alias → v16" is. The engineering goal is to make AI changes reviewable on evidence, not on impression.


17.18 Documentation and architecture decision records

Fast-moving AI codebases accumulate undocumented decisions — why this model, why this chunk size, why this retry policy — and lose the reasoning within weeks. Architecture Decision Records (ADRs) are the cheap fix.

Record, at minimum:

decision (what was chosen)
context (what problem, what constraints)
alternatives considered
consequences / trade-offs accepted
how to revisit (what would change this decision)

For AI platforms the highest-value ADRs cover model strategy, build-vs-buy, autonomy boundaries, data and retrieval design, and evaluation thresholds — exactly the decisions that are expensive to reverse and easy to forget.


17.19 Environments and promotion

An AI platform needs the same environment separation as any serious system — and the model configuration must be part of what differs between environments, not an afterthought.

dev      → cheap models allowed, verbose capture, synthetic data
staging  → production-like config, shadow traffic, full eval
prod     → pinned models, strict capture/redaction, canary + rollback

A frequent, expensive mistake: testing against one model configuration and deploying another. If staging uses a different model, prompt or temperature than production, staging proves very little. Promote configuration, not just code, and keep the difference between environments explicit and reviewed.


17.20 The engineering baseline checklist

An architect can assess an AI platform's engineering maturity with this checklist. It maps one-to-one to the sections above.

[ ] All model calls go through one gateway boundary
[ ] Prompts, model ids and params are versioned config, not literals
[ ] Raw model output is validated against a schema before use
[ ] No secrets in code, prompts, config, or logs
[ ] Dependencies pinned; SBOM generated; vulnerabilities scanned
[ ] Model weights/datasets have provenance and integrity checks
[ ] Deterministic failure modes have deterministic tests (model mocked)
[ ] Evaluation and safety suites run in CI as gates with versioned thresholds
[ ] Builds are immutable and deployed by digest, not by mutable tag
[ ] Every behaviour-changing artifact is independently deployable and rollbackable
[ ] Any production response can be reproduced from recorded versions
[ ] Side-effecting actions are idempotent; retries are bounded with backoff
[ ] AI changes are reviewed on evaluation evidence, not impression
[ ] Environments differ in a reviewed, explicit way (config promoted, not just code)
[ ] Key AI decisions are captured in ADRs

If a platform fails half of these, the model quality is almost irrelevant — the system will fail operationally before the model gets a chance to.


17.21 Common mistakes

MistakeConsequence
Prompts as inline string literalsBehaviour changes are invisible and unreviewable
Model called from everywhereCannot swap, mock, cache, or instrument
Raw model text into business logicSilent corruption, injection, type errors
No deterministic tests ("we'll just eval")Ordinary bugs ship; eval is slow and flaky as a net
Floating dependency versionsSilent behaviour and cost drift
:latest image tagsNon-reproducible, un-rollbackable deploys
Secrets in prompts or logsCredential leakage to provider or log store
Unbounded retries / non-idempotent toolsRetry storms, double-executed side effects
Staging config ≠ production configStaging proves nothing
Evaluation as an occasional manual checkRegressions reach production undetected
---

17.22 FAQ: Are AI systems really just software?

Mostly, yes — and the parts that are not are smaller than teams assume. An AI platform is a normal software system with one nondeterministic subsystem. The interfaces, dependencies, configuration, builds, tests, deployments and failure handling are ordinary software engineering, and that is where the majority of real incidents originate. The genuinely novel part is that the model can return different outputs for the same input, is version-unstable, is not fully testable, and is often operated by a third party. The architect's job is to apply standard engineering rigour to everything around the model and reserve the special techniques — evaluation, verification, guardrails — for the model itself.


17.23 FAQ: How do you unit-test an LLM feature?

You unit-test everything around the model deterministically by mocking the model at the gateway boundary. Tests cover parsing, schema validation, routing, tool dispatch, retry and fallback logic, error handling, and cost accounting — using canned or recorded model responses so the tests are fast and repeatable. The model's actual output quality is not unit-tested; it is measured by evaluation suites on a dataset, with statistical thresholds, and those suites run in CI as a separate gate. The division is: deterministic logic gets deterministic tests, probabilistic output gets scored evaluation.


17.24 FAQ: What is an AI deployment manifest?

It is an immutable, complete description of everything that defines a deployment's behaviour, so the deployment is reproducible and rollbackable as a unit. It records the application version, the prompt versions, the primary and fallback model ids and parameters, the retrieval and reranker configuration, the policy pack version, and the evaluation gate that approved it. Because each of those is also independently versioned, you can roll back a single component — for example moving the production prompt alias from v17 to v16 — rather than redeploying the whole platform when one artifact causes an incident.


17.25 FAQ: Can AI systems be fully reproducible?

The software around the model can be fully reproducible; the model itself usually cannot. You can pin the code commit, the dependency lockfile, the container image digest, the prompt version, the model id, the decoding parameters, the retrieval index version and the policy version — which lets you reconstruct the exact configuration that ran. But provider-hosted models can change behaviour over time and rarely guarantee bit-identical output even with a fixed id and seed. So reproducibility for AI means "same configuration and same inputs," which is enough for root-cause analysis and regression testing, rather than "identical bytes," which is generally not achievable with external models.


17.26 FAQ: Why not just put prompts in the code?

Because a prompt is a behaviour-changing artifact, and burying it in code makes behaviour changes invisible, unversioned, unreviewable and un-rollbackable without a full deploy. When prompts live as versioned configuration or in a registry, you get independent promotion and rollback, evaluation tied to each version, clear ownership, and the ability to change behaviour without shipping code. The same argument applies to model ids and decoding parameters: they decide behaviour, so they are configuration, and configuration is separated from code.


17.27 What you should know cold

  • An AI platform is software with a probabilistic subsystem; apply full engineering rigour everywhere except the model's output, and use evaluation for that.
  • Put every model call behind one gateway boundary so you can swap, mock, cache, instrument and govern in one place.
  • Treat prompts, model ids and decoding parameters as versioned config, never as code literals; validate all model output against a schema before it enters business logic.
  • Test deterministic logic deterministically by mocking the model; run evaluation and safety suites in CI as gates with versioned thresholds.
  • Make builds immutable, deploy by digest, keep every behaviour-changing artifact independently rollbackable, and keep enough version records that any production response can be reproduced in configuration.

The one sentence to remember

"Most AI incidents are ordinary engineering failures — unpinned dependencies, untested edge cases, leaked secrets, un-rollbackable config — so I apply standard software discipline to everything around the model, and reserve evaluation and verification for the model itself."

That is the engineering baseline a platform has to meet before any of the model work is worth doing.


Part of the series

The Enterprise AI Architect's Handbook
  1. 1.The Enterprise AI Architect Roadmap: The 29 Domains the Role Actually Owns
  2. 2.The AI Architect Operating Model: Turning a Business Objective into an Architecture
  3. 3.LLM Fundamentals for Architects: Tokens, Context, Latency, Throughput and Cost
  4. 4.Prompt and Context Engineering as an Architectural Concern
  5. 5.RAG Architecture: The Full Pipeline and Where Each Stage Fails
  6. 6.Knowledge Architecture: Ontologies, Entity Resolution and Graph Retrieval
  7. 7.Agent Architecture: Loops, Planning, Verification and Termination
  8. 8.Agent State and Memory Architecture: Scoping, Retention and Provenance
  9. 9.Multi-Agent Systems: When They Help, and How They Fail
  10. 10.Agent Orchestration: Frameworks, Durable Execution and Framework-Independent Design
  11. 11.MCP Architecture and the Enterprise Tool Gateway
  12. 12.Model Strategy: Selection, Gateways, Routing and Fallbacks
  13. 13.Fine-Tuning, RAG or Prompting: How an Architect Decides
  14. 14.Evaluating LLM, RAG and Agent Systems: Metrics, Judges and Quality Gates
  15. 15.LLMOps and Observability: Tracing, Metrics, Drift and Feedback Loops
  16. 16.AI Security: The Full Threat and Control Map for Architects
  17. 17.Responsible AI, Privacy and Governance as Architecture, Not Paperwork
  18. 18.Software Engineering for AI Platforms: The Non-Negotiable Baseline← you are here
  19. 19.Cloud Architecture for AI Workloads: Isolation, Identity, Networking and Serving
  20. 20.Containers, Infrastructure as Code and Delivery for AI Systemscoming soon
  21. 21.Cost and Performance Architecture: Designing for Cost per Successful Taskcoming soon
  22. 22.Reliability and Resilience: The Twenty Failure Modes of AI Systemscoming soon
  23. 23.Enterprise AI Platform Architecture: Control Plane and Runtime Planecoming soon
  24. 24.Production and Launch Readiness for AI Systemscoming soon
  25. 25.Domain Architecture: Applying the Model to a Real Business Functioncoming soon
  26. 26.AI System Design Practice: Fifteen Problems and How to Approach Themcoming soon
  27. 27.Architecture Artefacts: The Diagrams an AI Architect Must Be Able to Drawcoming soon
  28. 28.Structured Answers: System Design, Trade-offs, Incidents and Reviewscoming soon
  29. 29.Experience Narratives: The Stories an Architect Must Be Able to Tellcoming soon
  30. 30.Architecture Leadership and Technical Strategycoming soon
View full series →
AISeriesNovember 6, 2026
Share
Aakash Ahuja

Aakash Ahuja

Enterprise AI, Cybersecurity & Platform Engineering

Aakash writes about secure AI agents, microservices architecture, enterprise platforms, and production engineering. He has 20+ years of experience building and operating software systems across banking, cloud, cybersecurity, AI, and enterprise workflow automation. He is Director of Technology at itmtb Technologies and teaches AI, Big Data, and Reinforcement Learning at top institutes in India.