Subscribe
Share
Share
Embed
Databricks and Fabric have five integration paths, not one. Each moves data. None moves permissions. We pull apart the one-credential model, the hour-long revoke delay, and the ADLS shortcut that silently bypasses Unity Catalog's row-level security.
Episode 41 • 2026-09-18 Duration: 8:45
Databricks and Fabric have five integration paths, not one. Each moves data. None moves permissions. We pull apart the one-credential model, the hour-long revoke delay, and the ADLS shortcut that silently bypasses Unity Catalog's row-level security.
The Fabric Friday video on this topic: https://youtu.be/RAqhXwSj_HE
AI-generated voices. Matthias — cloned voice. Fabia — designed AI co-host. See Matthias live on YouTube (Fabric Friday), at his meetups, and at conferences like FabCon.
Hosted by Matthias Falland — Microsoft Data Platform MVP and community architect behind the Fabric Periodic Table. New episodes every Friday.
Have an architecture decision you are wrestling with? DM Matthias on LinkedIn — find him as Matthias Falland. Three to five sentences about the decision, your team size, and your current stack. We anonymize before airing.
This podcast was generated by AI. Both voices are synthetic: Matthias is a cloned voice, Fabia is a designed AI co-host.
Brand design based on fabricperiodictable.com.
Architecture decisions for Microsoft Fabric. Anonymized real customer scenarios, cost realism, counter-arguments included. Weekly episodes aligned with Fabric Friday recordings.
Five Doors and One Credential
Episode 41 | 2026-09-18
Fabric Architecture Podcast | Hosted by Matthias Falland with AI co-host Fabia
---
[00:08] Fabia: Five integration paths between Databricks and Fabric. People talk about them like they're one feature.
[00:14] Matthias: And they're... five different answers to who can read the data?
[00:19] Fabia: Five different answers to who can read it, who can write it, and what happens to your grants along the way.
[00:26] Matthias: So we're pulling apart what actually crosses that boundary.
[00:30] Fabia: What crosses, what gets quietly dropped, and why revoking a grant in Databricks doesn't mean what you'd expect in Fabric.
[00:38] Fabia: The mirrored Azure Databricks catalog is the path people reach for first. GA since July 2025. Fabric reads what Unity Catalog knows, creates one shortcut per table, and copies zero bytes. The data stays in the ADLS Gen2 storage Databricks already owns.
[00:54] Matthias: So Fabric points at the files through the shortcut.
[00:58] Fabia: And it's read-only. In the video, an INSERT INTO the mirrored table got refused on the spot. The second path, publishing from the Databricks UI, lands in the same Fabric item. Different starting point, same result.
[01:12] Matthias: What about the other direction — Databricks reading Fabric?
[01:16] Fabia: Catalog federation. Databricks sees a Fabric lakehouse or warehouse as a foreign catalog. Three-part names, SELECT only.
[01:24] Matthias: Three doors, none of them write.
[01:26] Fabia: Door four does. Spark over the OneLake ABFS endpoint. Reads and writes.
[01:31] Matthias: And the fifth?
[01:32] Fabia: OneLake becomes the managed storage for a Unity Catalog catalog. Still in beta. That's where it gets consequential.
[01:40] Fabia: Here's the sentence that matters from the security page. The credential used to create the connection to Unity Catalog of this catalog mirroring is used for all data queries.
[01:51] Matthias: Every Fabric user reads through one credential?
[01:54] Fabia: Unity Catalog sees one principal. Fabric sees everyone who holds a workspace role. Nothing maps between them.
[02:02] Matthias: So Unity Catalog has no idea who's actually asking.
[02:05] Fabia: It can't. Fabric reads the Delta files through the shortcut. There's no Databricks query engine in that path to enforce a grant.
[02:13] Matthias: That means the tables I select when I create the mirror—
[02:17] Fabia: —are the access boundary. And in the video, the default was the whole catalog ticked.
[02:23] Matthias: Of course it was.
[02:24] Fabia: The REST API lets you choose Full or Partial at catalog, schema, and table level. Partial mirrors nothing unless you select it.
[02:32] Matthias: I'd start every first mirror on Partial. The cost of selecting too little is a support ticket. The cost of selecting too much is a data exposure that sits there for months. That's not a symmetric risk.
[02:46] Fabia: Someone asked this, and it opens the real question. "Our row-level security is in Unity Catalog. Does it still apply in Fabric?"
[02:55] Matthias: No.
[02:55] Fabia: The platform handles that by refusing to mirror those tables entirely. Any table with row-level or column-level security policies gets skipped.
[03:05] Matthias: So the refusal is the safety mechanism.
[03:07] Fabia: Because the filters live in the Databricks query engine. The Delta files carry no filter. If Fabric read them through a shortcut, it'd read every row.
[03:17] Matthias: But what if... someone in my org didn't go through the mirror? What if they made a plain ADLS Gen2 shortcut straight to the same storage account?
[03:27] Fabia: That's the question nobody asks out loud. The docs say it plainly — Unity Catalog policies are not enforced at the storage layer. A direct connection to storage bypasses every RLS filter, every column mask.
[03:41] Matthias: And the ADLS shortcut's faster to set up than the mirror. So the path of least resistance is the one that bypasses everything. If that shortcut exists in your tenant, go find it. That's the real homework from this episode.
[03:56] Matthias: In the video, you revoked a SELECT grant in Databricks and it took about an hour to land in Fabric. What's going on there?
[04:04] Fabia: Two different clocks. The docs say data changes can take a few seconds to several minutes to propagate. But that's data changes reaching the mirror. A credential revoke is different — the connection uses short-lived tokens refreshed on an hourly cycle. That's the delay the demo showed.
[04:23] Matthias: An hour's a long time when you're responding to a security event.
[04:27] Fabia: During that hour, every query in Fabric still reads through the old credential state. The grant's gone in Databricks. Fabric doesn't know yet.
[04:37] Matthias: So revoking a grant feels like you've closed the door—
[04:40] Fabia: —and Fabric's still reading through it. The architecture answer is to mirror less in the first place.
[04:47] Matthias: Every time someone shows me a mirrored catalog, I ask what they selected. It's the whole catalog. Nobody goes back to narrow it once it's running.
[04:57] Fabia: OneLake storage is pay-as-you-go per gigabyte and consumes no capacity units at all. Transactions — reads, writes, listing files — those draw from your Fabric capacity. There's no separate transaction charge.
[05:11] Matthias: Who pays when there are two capacities involved?
[05:14] Fabia: The reader. If capacity two has a shortcut pointing at data in capacity one, reads consume CUs from capacity two. Capacity one keeps the storage bill.
[05:24] Matthias: That's... actually clean for chargeback. The team running the query pays for the query.
[05:30] Fabia: Until someone pauses that capacity to save money. Every shortcut inside it goes dark.
[05:36] Matthias: Cost control and availability on the same lever. Make sure finance knows that before they schedule capacity pauses.
[05:43] Fabia: For a mirrored Databricks catalog specifically, the files sit in ADLS Gen2 outside OneLake. Shortcut reads to a source outside OneLake don't consume OneLake CUs — the storage service bills you directly.
[05:57] Matthias: Let me steel-man the direct ADLS shortcut. If your Databricks environment has no RLS, no column masks, no lineage tracking — nothing in Unity Catalog worth protecting — then the mirror is overhead. You're mirroring governance that was never built.
[06:13] Fabia: That's fair. If there's nothing in the catalog to respect, a shortcut to the storage is simpler and it works today.
[06:21] Matthias: The condition's narrow, though. The moment anyone adds a row filter on the Databricks side, that shortcut becomes a bypass nobody authorized. And nobody's going to notify you when it happens.
[06:34] Fabia: The fifth door carries the strongest warning. Microsoft's own docs say any principal with a non-viewer role in the Fabric workspace can read, write, and delete data in the Azure Databricks Storage item — regardless of Unity Catalog grants.
[06:50] Matthias: Regardless?
[06:51] Fabia: Their mitigation is a dedicated restricted workspace used for nothing else.
[06:56] Matthias: So the beta door has a second door right next to it. And it's open.
[07:00] Fabia: If Databricks is the system of record and analysts need Power BI, the mirrored catalog with Direct Lake is the cleanest path. No data copy, no Databricks cluster awake for reports.
[07:12] Matthias: And if a Databricks job needs to write into a Fabric lakehouse, that's door four — but only on a standard cluster. Serverless rejects the OAuth config with CONFIG_NOT_AVAILABLE.
[07:24] Fabia: The workaround is MSAL for the token and the Python deltalake library. And regardless of which door writes — one writer per table path. Delta's transaction log can't arbitrate between engines on different runtimes.
[07:38] Matthias: Here's what I'd tell a team walking into this. Start the mirror on Partial. Mirror only the schemas analysts need. Rebuild access controls on the Fabric side with the same Entra groups you use in Unity Catalog. And go audit for ADLS shortcuts that skip the mirror. That's the work.
[07:57] Fabia: And accept that "keep access configurations synchronized" is documentation for there is no sync. Two grant systems, one manual reconciliation. Simplicity on the slide is not simplicity at runtime.
[08:10] Matthias: Every path between these platforms moves data. None of them moves authority.
[08:15] Fabia: Somewhere right now, a mirrored catalog is serving a workspace that was supposed to see two schemas. It's been ticked to the whole catalog since the day someone set it up.
[08:27] Matthias: If that sounds familiar, come talk to me. DM me on LinkedIn — find me as Matthias Falland.
[08:33] Fabia: Bring your table picker screenshot.
---
End of episode. ~08:35 estimated.