Hook
It was 3 AM in a Dublin co-working space, and I was staring at a Grafana dashboard that told a story I’d seen before—but never with such clarity. A decentralized AI inference cluster, built on a mesh of volunteers and a token-incentivized storage layer, was hitting a wall. The KV cache—the memory that allows LLMs to remember context across conversations—was ballooning. The node operators were swapping out HDDs for NVMe SSDs, but the latency was still killing throughput. Then I read the SanDisk report: by 2030, KV caches will drive 35% of all NAND workloads in AI data centers. The numbers hit me not as a hardware forecast, but as a philosophical challenge to the decentralized AI movement. The code is open, but the vision is ours to build—and we need to build for this storage reality.
Context
SanDisk’s prediction isn’t just about flash memory. It’s a statement about the architecture of intelligence. KV caches are the forgotten middlemen of LLM inference: they store the key-value pairs that allow models to maintain context without recomputing every token. As models grow to 1M+ token contexts, the cache size explodes. Today, most of it lives in expensive HBM or DRAM. SanDisk argues that by 2030, economic pressure will force a shift: a significant portion of that cache will be offloaded to NAND flash—specifically, high-capacity QLC SSDs with optimized latency and endurance. This is a seismic shift in the storage hierarchy, and it’s happening entirely within the centralized data center model. But what does it mean for the decentralized web? I’ve spent years auditing protocols—from Filecoin’s proof-of-spacetime to Arweave’s blockweave—and the gap between centralized and decentralized storage performance is not just a tech gap; it’s a values gap. Volatility is the tax we pay for freedom, but if we can’t match the performance of a SanDisk enterprise SSD, the freedom to run AI without gatekeepers remains a pipe dream.
Core
Let’s start with the numbers. SanDisk’s 35% is a workload share, not a capacity share. That means nearly one out of every three I/O operations in an AI data center will be KV cache reads and writes. This is a dramatic shift from today, where the bulk of NAND work is model weight loading, checkpointing, and logging. The implication is that storage vendors will need to design SSDs with a very specific profile: high random read/write IOPS, low tail latency, and high endurance—especially for QLC, which traditionally suffers from write fatigue. SanDisk’s own BiCS 8 technology (200+ layers) and its QLC roadmap are clearly aimed at this. But here’s the kicker: the economics only work if the storage is cheap enough to replace DRAM. SanDisk is betting that NAND cost-per-bit will continue to drop faster than DRAM’s, making the trade-off attractive. In my 2020 DeFi Summer analysis of on-chain data storage, I noticed a similar pattern: protocols that stored data on Arweave or Filecoin were cheaper than Ethereum calldata, but the latency made them unusable for hot data. KV cache offloading is the same problem—it’s a hot data problem. The centralized solution is a fast, close-to-compute SSD. The decentralized solution? It doesn’t exist yet. Not at scale. Based on my experience auditing the economic models of storage networks, I see a fundamental tension: decentralized storage networks prioritize verifiability and durability over latency. Filecoin’s retrieval market is improving, but it’s nowhere near the sub-millisecond latency required for KV cache. The core insight here is that the 35% prediction is a wake-up call for the decentralized storage community. We do not follow trends; we architect ecosystems. If we want decentralized AI infrastructure to be credible, we need to design storage layers that can handle this workload. That means hardware-aware protocol design—perhaps using ZNS SSDs, or even building custom nodes with NVMe-oF and CXL interfaces. The code is open, but the vision is ours to build, and that vision now includes a storage spec that looks nothing like today’s Filecoin miners.
Contrarian
Here’s the counter-intuitive angle: the very success of SanDisk’s prediction could accelerate the adoption of decentralized storage. Why? Because the centralized data center model, by offloading KV caches to NAND, will dramatically lower the cost of AI inference. This will democratize access to AI, but it also creates a new bottleneck: the hyperscalers will own the storage layer. AWS, Azure, and Google Cloud will become even more entrenched as the gatekeepers of AI memory. And when the cost of inference drops, the demand for AI agents, autonomous systems, and edge devices will explode. Those edge devices won’t be able to connect to a centralized KV cache over the public internet with acceptable latency. They will need local, distributed, or federated storage. That’s where decentralized networks can step in—not as a replacement for SanDisk’s SSDs, but as a complementary layer for edge AI and privacy-preserving inference. The blind spot in SanDisk’s vision is the assumption that all AI compute will happen in monolithic data centers. In reality, the future of AI is multi-modal, multi-location, and increasingly sovereign. Trust is not given; it is compiled, line by line, and that compilation must happen on open infrastructure. The contrarian take is that SanDisk’s 35% number is actually too low, because it only counts data center workloads, ignoring the explosion of edge AI and the need for decentralized storage to serve those caches. The real opportunity is in building a decentralized KV cache layer that bridges the gap between local inference and global persistence.
Takeaway
Volatility is the tax we pay for freedom, and SanDisk’s prediction is a reminder that the storage landscape is volatile. The next three years will see a race between centralized giants and decentralized communities to own the memory layer of AI. The question is not whether 35% of NAND workloads will be KV caches—it’s whether that workload will be served by closed, proprietary hardware or by open, community-owned protocols. From the ashes of FUD, we forge true adoption. The code is open, but the vision is ours to build. Let’s build a storage layer that doesn’t just store the past, but remembers the context of every conversation—without asking for permission.
