LLM.co

Before you sign a private LLM contract, your RFP needs to do far more than check feature boxes. This episode breaks down the six vendor requirements that separate a deal you can enforce from one you'll spend years regretting.

Show Notes

Signing a private LLM vendor agreement without the right RFP language is how companies end up locked into data topologies, logging schemas, and termination terms they never agreed to — they simply never required anything different. This episode of LLM.co walks procurement leaders, CISOs, and heads of AI through the six contractual line items that actually determine whether a private LLM deal is enforceable — drawing directly from the private LLM RFP requirements checklist published on LLM.co.

Here's what the episode covers:

  • Data and weight ownership: Five questions every RFP must force in writing — who owns training data, fine-tuned weights, and derived telemetry — with any hedge treated as disqualifying. Vendors who retain rights to derived weights will leverage that claim at renewal.
  • Air-gap and update mechanics: "Air-gapped deployment" is a marketing phrase until a vendor submits a network diagram covering every ingress and egress across install, inference, and patching. The episode details what a genuine offline deployment commitment looks like in writing, relevant to teams evaluating offline AI agents and air-gapped LLMs.
  • Audit log schemas: Logging "all activity" means nothing without a field-level specification. The episode walks through required fields at the request, agent, and administrative levels — all in structured JSON, streamed to the buyer's SIEM, with retention controlled by the buyer, not the vendor.
  • Red-team acceptance tests: Final payment should be gated on adversarial testing performed on the buyer's infrastructure against the buyer's data — anchored to the OWASP Top 10 for LLM Applications 2025 and the NIST Generative AI Profile (AI 600-1), with defined pass thresholds and re-test rights after every model or RAG index update. Teams managing ongoing risk may also want to look at LLM security audit and AI model review services.
  • Incident notification windows: Vendor default timelines routinely consume half a regulated buyer's compliance clock before the buyer even learns of an incident. The episode lays out the specific 24-hour discovery-to-notification requirement and why named breach coordinators and customer-led disclosure rights belong in the RFP.
  • Exit, escrow, and the kill clause: Assume the relationship ends. Source and weight escrow with a neutral third party, data portability across all fine-tuned artifacts and RAG indexes, and a defined kill clause triggered by acquisition, insolvency, or unpatched critical vulnerabilities are the three provisions every exit section needs.

For more on structuring the contract type itself — fixed-scope, managed appliance, or co-build — the related episode Fixed-Scope, Managed Appliance, or Co-Build: Picking the Right Private LLM Contract covers the trade-offs in depth.

LLM.co

cstm.ai

What is LLM.co?

Private and custom large language models — the build, the boundaries and the bill. Fine-tuning versus retrieval, running models in your own environment, evaluation you can actually trust, data governance, and the questions to ask before a vendor answers them for you.

Each episode takes one decision a team is facing — whether your problem needs a custom model at all, how to evaluate output without fooling yourself, what "private" has to mean contractually — and works it through concretely. Written for engineering and data leaders putting a model into production. Five or six minutes, one idea, no demos.

Topics include fine-tuning versus retrieval, self-hosted and private deployment, evaluation you can trust, prompt and context design, data governance and retention, cost and latency tradeoffs, and what "private" has to mean contractually.

Produced by LLM.co, private and custom large language models. Full details, services and further reading at https://llm.co