Cache-Mesh vs Iceberg Metadata Cache for Hot Data Access

Cache-mesh and Iceberg metadata cache solve different halves of hot-data access. Metadata caching helps a query find the files it needs. Cache-mesh helps it read those files without making object-storage round-trips over and over. Consistently low latency depends on knowing which phase is slow, then keeping the relevant cache and the data beneath it healthy.

Why hot-data latency is a two-part problem, not one

A query returns slowly even though it runs on a supposedly fast architecture. The natural suspicion is that reading the data took too long. But did the engine spend that time reading rows, or deciding where to find them? In a directory-listing approach, discovering relevant files means enumerating potentially enormous numbers of them, which can consume many seconds before a byte of table data moves.

Execution introduces a separate source of delay. Once the engine knows which files it needs, fetching them still requires network round-trips to object storage. Those delays on each call slow execution when queries make many reads, even when the reads run in parallel. Faster discovery does not make those fetches faster, and faster reads do not remove the work of discovery. This split defines the two cache layers, with metadata caching serving planning and data-block caching serving execution.

What Iceberg's metadata tree does and does not fix

Iceberg replaces directory discovery with a metadata tree. Instead of scanning directories, the engine follows a catalog pointer through metadata files, manifest lists, and manifest files to identify relevant data files. The Iceberg specification defines manifest entries that include partition values, file sizes, and column-level statistics such as minimums, maximums, and null counts. These let the engine prune files without making directory LIST calls.

The tree and its cache serve related but different purposes. Iceberg provides the structure that makes metadata-based planning possible, while a metadata cache keeps planning information available without repeatedly fetching it from object storage. That is a genuine performance gain. On tables where file discovery once dominated query time, removing that work can change how quickly useful execution begins. Caching the metadata then reduces the storage access needed to navigate that structure.

Once the file selection is complete, an uncached Parquet read still requires a network call to object storage. Pruning can reduce the files selected, but it does not shorten the round-trip for a file that execution must fetch. Meanwhile, frequent commits accumulate snapshots, manifest lists, and manifest files. Without compaction and snapshot expiration, the growing tree can itself slow planning and leave the metadata cache with more information to hold and invalidate.

What a cache-mesh actually is and which latency it targets

A cache-mesh stores the columnar data blocks themselves across executor nodes in distributed local storage or memory. The cached content is data in formats the engine reads natively. That distinction defines its place in the architecture. Metadata tells the engine which files to read, while the mesh provides a faster place from which to read them. When the cache holds the data, execution can read it without a fresh object-storage fetch.

Object-storage fetches incur network latency, whereas blocks resident on local NVMe or in memory can be read much faster. The qualification is important, because the mesh accelerates the working set it actually holds. It does not turn every file in the lake into a local file. Metadata caching and cache-mesh therefore stack, with one reducing the wait before execution and the other reducing the wait during it.

Their operation also needs to remain distinct. The Clock2Q+ research on VMware vSAN finds that metadata references can arrive in correlated bursts that cache replacement algorithms mistakenly treat as evidence of hot blocks. Its correlation window prevents short-term bursts from causing premature promotion. That finding shows why metadata and data caches should not be tuned on the same assumptions by default, since repeated references can mean different things at each layer, without prescribing a particular replacement algorithm for Iceberg.

How the two layers stack in production: the four-cache architecture

Dremio's documented stack combines metadata caching and data caching with results caching and Reflections. Its query-plane metadata cache holds schemas, partition statistics, and planning information on coordinator nodes, so plans can be built in sub-second time without object-storage calls. That gives execution an earlier start. The results cache can then skip execution when the underlying data has not changed and return earlier results stored as Arrow files written by executors. Coordinators manage expiration, and results are accessible through JDBC, ODBC, REST, and Arrow Flight.

When execution is required, Columnar Cloud Cache, or C3, provides the data-cache layer. It stores frequently accessed files, including Parquet and ORC, on local NVMe at executor nodes. Subsequent queries against those files read from local SSD. Reflections address a different cost by storing pre-computed query results, including materialized aggregations, back in the data lake. As a Reflection is queried repeatedly, C3 caches its files, combining less query work with faster reads.

These layers have dependencies as well as separate jobs. A results cache needs the metadata layer to check whether an earlier result is still valid. For an Iceberg query, C3 accelerates reads after metadata pruning has identified the relevant files. Local storage cannot make a slow planning stage faster. Nor can a fast plan substitute for a reusable result or a cached data file when the workload needs those benefits.

The stack consequently creates four places to tune, invalidate, and diagnose. Teams operating it need to understand what each layer stores and how invalidation propagates before changing any one cache in isolation. Metadata maintenance belongs in that work. Compaction through OPTIMIZE and snapshot expiration through VACUUM constrain the size and freshness of the information the coordinator must hold. If that information grows unchecked, downstream execution begins later regardless of how fast its data cache is.

Where the cache economics actually break down

Faster storage has a substantially higher cost than raw object storage, so residency needs to earn its place. The CIDR 2025 research by Duwe et al. establishes an access-frequency threshold above which caching becomes cost-effective, with different thresholds for latency-sensitive and non-latency-sensitive workloads. When reads fall below that threshold, it costs less to read straight from object storage. Hotness is therefore an economic property as well as a description of recent activity. A file being read once does not by itself justify keeping it on expensive storage.

The practical response is to separate hot and cold storage rather than mount the entire lake through a cache layer. A tightly scoped working bucket limits cache costs to active data whose access frequency justifies them. Cold data stays on raw object storage. This makes working-set selection part of the architecture. An unbounded cache can consume more resources without delivering enough reuse to pay for them.

Metadata caching has its own less visible cost. As an uncompacted table ages, a growing metadata footprint puts more pressure on coordinator memory and can slow plan construction despite the cache. The data cache cannot correct that pressure. More broadly, cache-mesh amplifies a well-partitioned, well-compacted table, while only temporarily and expensively masking poor organization. The table and its access pattern must make the cache’s faster reads worth retaining.

How agentic query patterns stress both cache layers differently

AI agents complicate that economic test because their access patterns can differ sharply from repeated dashboard queries. Human analysts often return to similar queries over stable data, giving caches repeated opportunities to reuse work. Agents may explore dynamically and never issue the same query again. Raw text-to-SQL agents derive joins and filters from prompt context, producing novel plans that a plan cache may not serve. Each new plan takes fresh planning work, which may include metadata reads and data fetches.

Does a new plan always make every cache useless? No, because plan reuse and data-block reuse are separate questions. An agent can change its queries enough to miss the plan and results caches while still reading some files that are already cached. But exploratory access also makes the data working set less predictable, weakening the assumptions that justify cache-mesh residency. When agents share compute with production traffic, probing in parallel adds pressure because bursts can cause noisy-neighbor contention.

A semantic layer offers a response at the query level. Governed metric selection can produce deterministic SQL for a given metric regardless of how an agent phrases the question. That gives plan and results caches something repeatable to serve. Stabilizing query construction addresses needless variation above the data cache, while leaving the separate problem of which data blocks deserve residency.

Compute isolation addresses that second problem. A separate agent node pool lets its cache warm to the agent working set without having to share the same NVMe capacity with production traffic. Logging every query then helps show which access patterns lead to misses, too many metadata reads, and compaction falling behind. Dedicated read-only compute with every query recorded gives operators a bounded, observable workload, which is necessary for managing the economics.

Choosing the right layer given your latency target and data shape

For engineers choosing an investment, the first diagnostic question is whether queries are slow to start or slow to finish after starting. Planning-dominated latency points toward metadata health and metadata caching. Execution-dominated latency points toward the data-read path. This distinction should precede a decision about adding storage or expanding a cache. Otherwise, that spending may speed up a phase that accounts for little of the delay the user sees.

Metadata caching alone can be sufficient when tables are compact and well-partitioned, manifests are small, and planning dominates total latency. It is also a reasonable fit when data volumes are modest enough that uncached object-storage execution is tolerable once planning improves. Reuse may be scarce in this setting. Ad hoc queries that do not repeat give a results cache little to reuse, and the data they read may not be reused enough to pay for warming a mesh.

Cache-mesh becomes necessary when queries start promptly but data fetching still prevents the required interactive response. Stable working sets and repeated access provide the conditions that amortize its cost premium. The full stack is warranted when both phases remain constraints, particularly on large, complex tables with hot working sets. For parallel agent workloads, the semantic layer can stabilize plans, the results cache can serve repeated queries, and an isolated data cache can serve agent reads without displacing production data.

The investment order follows those dependencies. Metadata health comes first, metadata caching second, and data-block caching third. At real-time analytics scale, isolated agent compute, logged queries, and NVMe-backed caching support consistently low latency as data volumes grow, provided the working set stays managed. Compute-hour pricing that does not penalize query frequency aligns with repeated agent probing without adding a charge for every query. Sustaining that architecture is an operational discipline, with compaction, snapshot expiration, working-set scoping, and access-pattern observation keeping each layer useful over time.