Most of you by now have seen the announcement that Micron has joined with Samsung to create the "Hybrid Memory Cube (HMC) Consortium" with fellow founding members Altera, Open Silicon, and Xilinx. [
link 1,
link 2]
The consortium is built around Micron’s hybrid (previously referred to as "hyper") memory cube technology. The initial goal of the consortium is to define specifications for HMC. The HMC interface is totally different, having nothing in common with current DDR implementations, so it is felt that standardization and adoption by major producers and users is the only way that HMC will become a standard memory product for the industry.
We have previously addressed the fact that the memory bandwidth required by high-performance computers and next-generation networking equipment has increased beyond what conventional memory architectures can provide. The term "memory wall" has been used to describe the problem. A solution to the memory wall problem requires an architecture that can deliver increased density and bandwidth at significantly reduced power consumption.
Micron initially announced their memory breakthrough earlier this year [see IFTLE 38: "Of Memory Cubes and Ivy Bridges — More 3D and TSV"], and began releasing information at conferences this summer [see J.T. Pawlowski, "Hybrid Memory Cube: Breakthrough DRAM performance with a fundamentally re-architected DRAM subsystem", Proc. 23rd Hot Chips Symposium, 2011].
While DDR DRAMs have gotten bigger through the years by increasing the parallel arrays of DRAM cells on chip, they remain limited to the bandwidth supported by package I/O. DDR3-1333 and DDR3-1600 devices currently offer bandwidths of 10.66 Gbps and 12.8 Gbps respectfully. The HMC is a stack of multiple memory die sitting atop a logic controller chip bonded together using TSV. This greatly increases available DRAM bandwidth by leveraging the large number of I/O pins available through TSVs. Both the number of contacts and their shorter lengths enable dramatically higher data transfer rates than today’s memory other memory architectures — Micron has shown prototypes rated at 128 Gbps.
Current DRAM burns a huge amount of the power in laptops and phones. Brian M. Shirley, VP of DRAM solutions at Micron, claims that the company’s hybrid memory cube technology "offers a 20-fold performance increase while reducing the size of the chip and consuming about one-tenth of the power," while occupying 10% of the volume of a DDR3 memory module.
[Performance & Power consumption Paradigm shift due to HMC (left); TSV stacked memory layers on logic layer (right). Source: Micron]
Micron reports that the HMC module achieves and exceeds 128 Gbps by using parallel channels. An image of the first-generation Micron HMC memory die showing the large number of I/O coming off each die:
[HMC Memory Device showing large Number of I/O. Source: Steve Liebson, Cadence EDA360 Insider blog]
Joe Jeddeloh, whose Micron team developed the logic portion of the HMC has described [link] the key "themes" of their technology as follows: "Instead of a DRAM die being one large device that has one set of I/Os on it, we break it into, say, 16 separate DRAMs, in essence much like a multicore processor. Each of those DRAMs has its own interface so when you go to access data, you go to a very local area of DRAM […]It’s a more directed access." Then, "we move that down the Z direction on a TSV."
When asked about the impact of 3D stacking on memory performance, Jeddeloh responded:
"When you think of a DIMM, maybe it has 4, 8, 16 [memory cell] banks in it, …once you go to a memory cube where you have these tiles and partitions, each of those has its own bank structure. So instead of 8 banks, you have 128 banks, 256 banks and each of these are put into parallel DRAM structures so you have a tremendous amount of concurrency available. You can think of a many-core processor coming at a many tiled memory system that marries up and can handle a lot of concurrent transactions."
In terms of mating this memory to today’s and future microprocessors, Jeddeloh commented that "As we go to more and more cores on processors [they become] more and more bandwidth-hungry. In this generation, you can’t stack the DRAM on top of the processor because the processor is too hot. That means the processor has to go off-chip to get that bandwidth [and] you need to connect a pipe to that processor that can bring in as much bandwidth at the lowest amount of power." Micron’s HMC technology, he explained, "can put more density in a very local area and put that right next to the processor." He also characterized power as the No. 1 theme going forward: "Once we reduce that power, we can create a smaller, more efficient I/O structure when the processor and the memory system are right next to each other. If you have, say, eight cubes hooked up to a processor, there’s a tremendous amount of bandwidth and concurrency that can happen in a very small area."
Concerning heat issues in the HMC, Jeddeloh noted that "DRAM doesn’t like heat; it messes up the refresh. If we are not on top of the processor, the heat is manageable. Once you create that low-power I/O […] and you’re not creating as much power within the cube itself, then you stack it up and pull the heat out the top."
[160 Gbps = 1 HMC or 15 DDR3-1333 DIMMS. Source: Micron Technology]
Scott Graham, general manager for Micron’s Hybrid Memory Cube (HMC), predicts that HMC impact will be seen in multiple markets such as high-performance computing, networking, video, medical, energy, wireless communications, transportation, security — basically any applications that will require the transfer of tremendous amounts of data. When asked about the Samsung partnership, he answered: "We need multiple sources for a broad adoption," since the industry is not comfortable with any sole source products. He indicated that the plan of record is to begin production in the second half of 2013.
At the recent Intel designer forum (IDF 2011) we found out that Micron teamed up with Intel to create the technology [link]. The company highlighted that a big impediment to scaling the performance of servers and data centers is the available bandwidth to memory:
"As the number of cores on a microprocessor increases, the need to feed the cores with more memory data expands proportionally. There [are] severe limitations to achieving high-speed and low-power using commodity DRAM […] We came to the conclusion that mating DRAM and a logic process based I/O buffer using 3D stacking could be the way to solve the dilemma. We found out that once we placed a multi-layer DRAM stack on top of a logic layer, we could solve another memory problem which limits the ability to efficiently transfer data from the DRAM memory cells to the corresponding I/O circuits."
Intel CTO Justin Rattner demonstrated the Hybrid Memory Cube toward the end of his keynote lecture which can be seen here [link]. Rattner noted that the HMC was "the world’s highest-bandwidth DRAM device with sustained transfer rates of 1 terabit per second (trillion bits per second). It is also the most energy efficient DRAM ever built."
It is currently unclear whether Intel holds any of this HMC IP — and it is equally unclear why Intel was not a founding member of the HMC consortium. IFTLE will follow this evolving story closely.
For all the latest in 3D IC and advanced packaging stay linked to IFTLE……….
