Subscribe
Share
Share
Embed
Direct Lake promises no refresh and VertiPaq speed at lake scale. But its failure modes are silent — it degrades instead of breaking. We pull apart framing, transcoding, and the fallback tax nobody talks about.
Episode 27 • 2026-07-03 Duration: 9:40
Direct Lake promises no refresh and VertiPaq speed at lake scale. But its failure modes are silent — it degrades instead of breaking. We pull apart framing, transcoding, and the fallback tax nobody talks about.
AI-generated voices. Matthias — cloned voice. Fabia — designed AI co-host. See Matthias live on YouTube (Fabric Friday), at his meetups, and at conferences like FabCon.
Hosted by Matthias Falland — Microsoft Data Platform MVP and community architect behind the Fabric Periodic Table. New episodes every Friday.
Have an architecture decision you are wrestling with? DM Matthias on LinkedIn — find him as Matthias Falland. Three to five sentences about the decision, your team size, and your current stack. We anonymize before airing.
This podcast was generated by AI.
Brand design based on fabricperiodictable.com.
Architecture decisions for Microsoft Fabric. Anonymized real customer scenarios, cost realism, counter-arguments included. Weekly episodes aligned with Fabric Friday recordings.
Speaker 1
404. Blob Not Found. A semantic model pointing at a parquet file that existed five minutes ago. And now it's gone.
Speaker 2
And the report just keeps running?
Speaker 1
Sometimes it errors, sometimes it serves yesterday's numbers, depends on when the frame last landed
Speaker 2
Architecture decisions for Microsoft Fabric. This is the Fabric Architecture Podcast. Today we're pulling apart Direct Lake semantic models. What's actually happening underneath the no refresh needed promise and where it quietly falls apart.
Speaker 1
Import mode copies your entire dataset into Vertipack. The in-memory columnar engine that makes Power BI fast. Direct query skips the copy and hits the source database on every single visual. Direct Lake does something in between. It loads parquet column chunks from OneLake directly into Vertipak on demand with no ETL copy step. Same fast engine, no scheduled refresh window.
Speaker 2
So what's the framing operation?
Speaker 1
Framing is when the model reads the delta transaction log and repoints itself at the latest parquet files. It takes seconds. It's metadata only. But until framing happens, your model serves data from the last successful frame The docs say it directly. Subsequent data modifications aren't visible until the next framing operation occurs. And the automatic update setting, which triggers framing for you, is on by default. Turn it off, and that refresh button becomes load-bearing.
Speaker 2
So no refresh needed really means the refresh is so fast you didn't notice it.
Speaker 1
The other half is transcoding. When a DAX query needs a column that isn't resident in memory, the engine loads that entire column from one lake. The whole column. Because both ParK and Vertipak are columnar stores. The unit of work is the column segment.
Speaker 2
Wait, cold start performance scales with columns, not rows?
Speaker 1
It does. The number of columns your query actually touches is usually far fewer than what's defined in the model. Every column you leave out is one that never gets transcoded. And the model moves through four memory states cold, semi-warm, warm, hot. A warm model performs on par with import. The gap between cold and warm is where the complaints live. Someone posted this exact problem on the fabric community. Thread title: Semantic Model, Direct Lake Not Refreshing. Their setup. A pipeline overwrites a lakehouse table every five minutes. Direct lake model on top with automatic updates enabled They expected near real-time freshness. Instead, stale numbers and intermittent blob not found errors
Speaker 2
The blobnot found tells you everything. If you're overwriting the whole table, you're deleting the park files that the model's current frame still references. The models holding a pointer to a file that no longer exists.
Speaker 1
And this is where the two direct lake types diverge. Direct Lake on One Lake, the recommended default for new models, doesn't fall back to Direct Query at all. It runs in Direct Lake Only mode, period. If those files are gone, the query fails. Direct lake on SQL would silently fall back to Direct Query and return results, just slower.
Speaker 2
So the fix is the right pattern. Append or merge instead of overwrite, so incremental framing can drop just the affected column segments and keep the dictionaries warm If you genuinely need subminute freshness on a mutating table, Direct Lake isn't the right tool. That's a direct query problem, and there's nothing wrong with saying so.
Speaker 1
There's something the docs bury in the middle of the performance page. Fallback relies on hybrid query plans, and those plans carry performance trade-offs even when DirectLake doesn't actually need to fall back. Merely allowing fallback taxes your query execution.
Speaker 2
Even on queries that stay in direct leg mode?
Speaker 1
Even then, the engine prepares for both paths, and the guardrail rule is worse. A single table that exceeds any guardrail limit prevents direct lake mode for the entire model Your carefully tuned dimensions don't save you if one fact table drifts past the row group limit.
Speaker 2
One bad table poisons the whole model.
Speaker 1
Which is why the diagnostic matters. Run Evaluate Table Traits in DAX. The Direct Lake Fallback Info column shows the fallback reason per table. None means you're healthy. Anything else names your problem. In development, set direct lake behavior to direct lake only. Convert a silent slowdown into a loud error, where errors are cheap.
Speaker 2
What I keep running into is teams that treat the storage mode as a modeling decision and move on. Pick import or direct query. Done. With DirectLake, your upstream pipeline became a semantic model concern. Whether you append or overwrite determines whether the model stays warm. The data engineer who didn't know their right job affects report performance. That's the conversation nobody's having.
Speaker 1
The capacity guardrails make this concrete. There's a cliff at F64. Below it, you're capped at 300 million rows per table, a thousand parquet files. And a model-sized ceiling that tops out at 40GB on an F32 and drops to 10 on the smallest SKUs At F-64, Rose jump to 1. 5 billion, files to 5,000, and the model size cap disappears. That's a 32-fold jump in capacity units from an F2. And the first tier where lakescale data is realistic.
Speaker 2
And VORDER, the thing that gives you 40 to 60% faster cold cache queries, is off by default in new workspaces.
Speaker 1
VORDER applies VertiPak compatible encoding at right time, so the transcoding step becomes a near-direct remap instead of a full re-encode. But it costs Spark 15 to 33% on writes, with zero read benefit for Spark itself. V-order your gold layer tables that feed Direct Lake. Skip it on bronze. It's a targeted cost.
Speaker 2
So you're paying for F64, and the biggest performance lever it offers is sitting there switched off. I want to be fair to import here, because every direct lay conversation treats it like the thing you're leaving behind.
Speaker 1
The docs are explicit. Import works well for a self-service analyst who needs to move fast without IT dependencies. You keep Power Query transformations inside the model, which means the analyst owns the data prep end-to-end. That capability doesn't exist in DirectLake. And here's what most DirectLake content omits. OneLake integration automatically writes import table data to Delta in OneLake. Choosing import doesn't lock you out of the lake.
Speaker 2
So when someone shows me a model that leans heavily on Power Query and asks about going direct lake. I tell them to budget for a replatforming of their data preparation, not a storage mode change. Moving the transformations into Spark or TSQL or data flows That's the actual project. If there's no data engineering team to hand it to, if the analyst owns the prep, import is genuinely the right answer. Don't migrate for the sake of migrating.
Speaker 1
The security architecture has to travel with it too. Direct Lake on OneLake reads files directly. And file access doesn't observe SQL-based row-level security. If your security model lives at the SQL Analytics endpoint and you build an on-one-lake model, queries succeed. An RLS simply isn't applied. That's documented behavior. Your security layer has to move when your storage mode does.
Speaker 2
So the lesson. Direct Lake moved where the discipline lives. With import the hard part was the refresh window. With Direct Lake, the model itself takes seconds. It's everything upstream that carries the weight.
Speaker 1
Simplicity on the slide is not simplicity at runtime.
Speaker 2
If you're working through this decision, DM me on LinkedIn. Find me as Matthias Falland.
Speaker 1
Run table traits before you ship. One line of DAX. It tells you whether your model is actually in direct lake mode or quietly serving something else.
Speaker 2
Somewhere right now a semantic model is serving yesterday's numbers and nobody's noticed.
Speaker 3
This was the Fabric Architecture Podcast. This podcast was generated by AI New episode every Friday.