Memory Finally Goes Multi-Tier

In the realm of data storage, multi-tier architectures have become standard practice, enabling optimized performance and costs. System memory, on the other hand, has been constrained to a single price / performance tier—DRAM. As data demands evolve and the need for greater computing efficiency intensifies, this single-tier approach to memory must be reevaluated. 

Storage: A Mature Multi-Tier Model

Storage systems have long utilized tiered architectures to balance performance and cost. Hierarchical Storage Management (HSM) automatically moves data between high-cost and low-cost storage media, keeping frequently accessed / “hot” data on high-speed devices like SSDs and infrequently accessed / "cold" data on slower, cost-effective devices like HDDs or tape. This strategy enhances efficiency and reduces costs.¹

Memory: The Emerging Frontier of Tiering

Unlike storage, system memory has been predominantly homogeneous, primarily using DRAM.² Historically, implementing a multi-tiered memory system has presented significant challenges, including:

• Complexity: Managing data placement across different memory tiers requires sophisticated algorithms to ensure optimal performance.
• Compatibility: Ensuring that existing applications can leverage a tiered memory system without significant modifications is crucial for adoption.
• Cost-Benefit Analysis: Organizations must evaluate whether the performance gains justify the investment in new memory technologies and the complexity of implementation.

Intel Optane was an earnest attempt at hierarchical memory tiering. It promised to bring a lower-cost technology into the memory tier. For many reasons, however—and particularly because implementation wasn’t simple (it required new hardware and software changes)—adoption was slow and the business was eventually shut down. 

Similarly, CXL-attached memory and CXL memory pooling devices promise to increase accessible memory and utilization, but both approaches fail to change the economics enough to truly create a new price / performance tier (as is highlighted in Google’s paper A Case Against CXL Memory Pooling). 

There has never been a greater appetite for a new, intelligent memory tiering technology that can truly reduce complexity, seamlessly integrate into existing environments, and yield actual cost savings. 

MEXT AI

MEXT is the answer to this need. MEXT begins by determining which memory pages in DRAM are hot vs. cold. It then offloads the cold pages to a Flash device (20x cheaper than DRAM). From there, MEXT leverages AI to continually predict which pages from Flash will be requested by the application, and preemptively pushes those pages back to DRAM. As a result, from the application’s perspective, relevant memory pages are generally findable in DRAM. 

This keeps performance intact within a smaller DRAM footprint, yielding substantial efficiencies (up to 40% lower operational costs). Alternatively, MEXT can be deployed to effectively double the memory capacity of a system with virtually no additional costs (by enabling the system’s Flash to function as though it were an extension of its memory).

Memory Finally Goes Multi-Tier

MEXT has instantiated a new tier in the memory pyramid (MEXT + Flash) that is far lower-cost than the tier above it (DRAM) yet far lower latency than the tier below (traditional Flash-as-storage). In this way, we’re finally pushing memory more towards a multi-tier model, enabling the kinds of efficiencies that have been seen in the storage world for years. 

As the scale of data-intensive applications continues to grow, the demand for efficient memory solutions accelerates. Learning from the successes in storage tiering, the industry is poised to adopt a multi-tier approach to memory. MEXT is delivering on this imperative, enabling memory systems that intelligently balance performance and cost to usher in a new era of computing efficiency. 

Sources: 1: https://en.wikipedia.org/wiki/Hierarchical_storage_management 2: https://ieeexplore.ieee.org/document/1005381