DEV

Building a private RAG system over Microsoft 365 is solvable — but only if you treat permissions, delta sync, and ACL-trimmed retrieval as first-class engineering problems, not afterthoughts. This episode breaks down what production-ready SharePoint and OneDrive grounding actually requires.

Show Notes

Enterprise teams keep hitting the same wall when grounding LLMs on SharePoint and OneDrive: a working prototype that fails the moment security asks who actually owns access to the returned content. This episode of DEV.co examines the architecture behind a permission-aware, production-grade RAG system over Microsoft 365 — drawing on the team's deep-dive article on private RAG over Microsoft 365 — and maps out exactly where complexity lives and how to handle it correctly from the start.

The episode covers the full stack of decisions engineering teams face when moving beyond the demo, including:

  • Delta queries and change tracking: How to use the Microsoft Graph delta endpoint to incrementally sync content — additions, modifications, and deletions — without expensive nightly full recrawls.
  • Scoping Graph permissions correctly: Why Sites.Read.All is a zero-trust red flag and how Sites.Selected with explicit site allow-listing is what actually clears a security review.
  • Entra ID object IDs over UPNs: User principal names break when accounts are renamed or deprovisioned; stable object IDs are the right identity anchor for downstream permission filtering.
  • ACL-trimmed retrieval: Materializing effective ACLs as fields on each index record and filtering every query against the caller's resolved group memberships before results are returned — the step most prototypes skip, and the reason overshared files become overshared vectors. Teams building production RAG pipelines need this baked in, not bolted on.
  • Format-aware parsing: The M365 corpus is not just Word docs — spreadsheets, OneNote sections, PowerPoint files, and SharePoint page markup all require distinct parsing logic, and retrieval format support varies significantly across surfaces.
  • Choosing a retrieval surface: A practical comparison of three real options — the Copilot Retrieval API (native ACLs, zero egress), Azure AI Search with security filters (full chunking control), and a fully custom pgvector or Qdrant index (maximum control, maximum ownership) — with guidance on which fits which stage of maturity. Cybersecurity considerations are woven throughout, including the OWASP 2025 LLM Top 10's new vector and embedding weakness category covering data poisoning, embedding inversion, and cross-user context leakage.

For more on how ingestion pipeline decisions shape the security boundary of an AI system, the episode Why Your LLM Ingestion Pipeline Is Your Real Security Perimeter covers closely related ground and pairs well with this one.

DEV.co

RFP.co

What is DEV?

Software and web development from the side that has to ship it and then live with it. Architecture decisions with a cost attached, scoping, technical debt, hiring and vendor selection, and the AI tooling question every engineering team is now answering whether they planned to or not.

Each episode takes one decision — rewrite or refactor, framework choice, build versus buy, how to scope a fixed-bid project honestly — and works through the tradeoffs, including the ones that only show up in year two. Written for engineering leads, technical founders and the people who fund them. Five or six minutes, no hand-waving.

Topics include rewrite versus refactor, build versus buy, scoping fixed-bid work honestly, technical debt you should keep, framework and platform choices, hiring and vendor selection, code review culture, and where AI tooling actually helps.

Produced by DEV.co, web and software development. Full details, services and further reading at https://dev.co