Unlocking the Power of Prediction

Algorithms to Live By

At a high level, the book explores how various phenomena found in computer science can be applied to human cognition and behavior to lead us to optimal outcomes—from selecting the right parking spot, to knowing how to organize our closets, to picking the right spouse, and beyond.

In the book, Christian and Griffith illustrate why memory tiering is critical, and explain how computing strategies that support intelligent tiering can also be leveraged to effectively organize your closet, as well as understand how memories work as we age.¹

A Brief History of Memory Hierarchies

In 1946, Arthur Burks, Herman Goldstine, and John van Neumann came up with the idea of “memory hierarchies”—creating distinct tiers of memories with some fast and physically small, and others slow and physically large. They theorized that this approach would help improve system performance in a significant and tenable way (while systems with ALL of their memory being “fast and small” would be ideal, they’d be too costly and unfeasible in real-world scenarios).

Their ideas remained purely theoretical until 1962, when the Atlas computer—designed by Tom Kilburn of the University of Manchester—put a version of the concept into practice. It was called “paged virtual memory”. Paging allowed the computer to use more memory space than was actually in the computer by mapping blocks of memory addresses in the limited primary memory, and then allowing such blocks called “pages” to be swapped between the primary memory tier and a slower, less expensive, secondary memory tier.²

The LRU Method: For Closets and More

There’s a whole variety of mechanisms that can guide efficient memory tiering in computing systems—and some of them have interesting applications even outside of the digital sphere. One such method is the Least Recently Used (LRU) method, where you choose to keep the more recently-used items in primary memory and push items that haven’t been used for a long time into the secondary tier.

If we apply this idea to the realm of closet organization, we find that we should place what we have worn most recently in the front part of our closets, while relegating our less-frequently worn clothing to the back of our closets.

If we apply this idea to the realm of human memory as we age, we find that older individuals who have trouble remembering things might not be facing these issues due to cognitive decline; instead, they simply have a greater volume of information, so sifting through what was “recent” is a longer, more laborious process. Information not used for a long time has likely been evicted from the forefront of the brain. As Christian and Griffith humorously put it, “the old can mock the young for their speed: ‘It’s because you don’t know anything yet!’”

MEXT AI

MEXT’s software solution leverages the value of heuristics like the LRU idea, and enhances them through advanced AI techniques. MEXT’s AI continually observes an application’s memory page access patterns, and uses that historical knowledge to predict what pages are likely to be requested next. It then transparently pushes those pages from lower tiers (Flash) back up to higher tiers (DRAM), before they are even requested by the application. This keeps performance intact within a smaller DRAM footprint, yielding substantial cost efficiencies by requiring less of the higher-priced DRAM tier.

MEXT’s AI engine consists of a family of prediction models that work together. For any given workload, it automatically adjusts to use the model or group of models that performs best. From there, observing which predicted pages were actually used by the application provides real-time feedback regarding model accuracy, supporting ongoing improvements to the prediction model’s accuracy.

We founded MEXT with the goal to bring Flash, traditionally constrained to the storage tier, into the memory domain. Now, with our AI-powered predictive memory software, Flash is finally there. If we go back to our closet example, it’s like having a closet that can predict what you’ll wear next and then continually reorganize itself to place that outfit right in front of you when you open the door.