LLM.co

Building an EU AI Act conformity file for a self-hosted LLM is harder than most security teams expect — and the clock is already running. This episode breaks down exactly what your evidence file must prove before December 2027.

Show Notes

The EU AI Act's December 2027 deadline for high-risk AI systems looks distant on a roadmap — but with notified body queues already growing and harmonised standards still being finalised, the organisations that will be ready are the ones assembling their evidence files now. This episode of LLM.co walks through the full conformity assessment requirements for self-hosted LLMs, cutting through the policy noise to focus on what a market surveillance authority would actually demand to see.

The episode covers the full arc of Article 43 conformity for enterprise LLM deployments — from the legal triggers that push a system into Annex III high-risk territory, to the structural compliance advantages of owning your own inference stack. Key topics include:

  • Why the internal control lane is not the easy lane: Self-assessment under Annex VI eliminates the need for a notified body, but it transfers the entire evidentiary burden to the deploying organisation — including a quality management system, risk management file, and a signed declaration of conformity available on demand.
  • Annex IV, section by section: Each of the nine documentation sections maps to concrete artefacts for a self-hosted model — build manifests with cryptographic hashes, base model provenance, fine-tuning dataset lineage, RAG index composition, and retrieval policy, among others.
  • Where evidence files most often fall apart: Auditors cross-reference the design section, the changelog, and the monitoring logs — and gaps between those three sources are where assessments become adversarial.
  • Self-hosting as a structural compliance advantage: Unlike public API deployments — where provider and deployer can end up pointing at each other under Article 3 — a self-hosted stack gives one accountable provider, immutable WORM logs, version-pinned model artefacts, and a single tenant of record for data residency.
  • Article 72 post-market monitoring done right: A quarterly review or an unwatched dashboard does not satisfy the requirement. The episode details the closed-loop signals regulators expect — including embedding drift, refusal-rate deviation, calibration drift on labeled probes, and jailbreak-signature detection — all tied to documented corrective action.
  • Penalty context for board-level conversations: Fines for high-risk violations are material, and prohibited-practice breaches can reach €35 million or seven percent of global annual turnover — making the cost of a defensible evidence file straightforward to justify.

More from the show: if your organisation relies on third-party model providers, the earlier episode Why DeepSeek's China Data Storage Policy Is an Enterprise Red Flag covers exactly the kind of supply-chain accountability gaps that make self-hosting a compliance imperative, not just an architectural preference.

LLM.co

cstm.ai

What is LLM.co?

Private and custom large language models — the build, the boundaries and the bill. Fine-tuning versus retrieval, running models in your own environment, evaluation you can actually trust, data governance, and the questions to ask before a vendor answers them for you.

Each episode takes one decision a team is facing — whether your problem needs a custom model at all, how to evaluate output without fooling yourself, what "private" has to mean contractually — and works it through concretely. Written for engineering and data leaders putting a model into production. Five or six minutes, one idea, no demos.

Topics include fine-tuning versus retrieval, self-hosted and private deployment, evaluation you can trust, prompt and context design, data governance and retention, cost and latency tradeoffs, and what "private" has to mean contractually.

Produced by LLM.co, private and custom large language models. Full details, services and further reading at https://llm.co