This is an old revision of the document!
Table of Contents
Optimising the memory performance of AMD EPYC 7002 Rome and 7003 Milan
Processors of the AMD EPYC 7002 Rome and 7003 Milan series expose more than eight memory controller channels that can be connected to a memory channel. Up to two DIMMs are supported per channel. To achieve the best possible overall performance, it is helpful to ensure a balanced allocation across all channels. This article presents frequently used DIMM configurations for these CPUs.
Basics
The following properties characterise processors of the AMD EPYC 7002/7003 series in terms of memory:
- 8 memory controllers per CPU
- 1 memory channel per memory controller
- 2 DIMMs per channel, with a maximum of 16 DIMMs per CPU and 32 DIMMs in a dual‑CPU system
Memory bandwidth and use cases
The highest possible memory bandwidth is particularly relevant in HPC environments. In other application areas, memory bandwidth has a lower impact on overall performance and depends on the specific workload. For example, tests show that applications can still achieve up to 90% of their potential performance with a memory bandwidth of about 35%. Configurations with fewer DIMMs reduce the maximum bandwidth but consume less energy and offer good expansion potential for the future.
Support references:
1. Single-CPU systems with 8 DIMM slots
In single‑CPU systems with 8 DIMM slots, the following configurations are commonly used:
| module size | 2 DIMMs | 4 DIMMs | 8 DIMMs |
|---|---|---|---|
| 8 GB | 16 GB | 32 GB | 64 GB |
| 16 GB | 32 GB | 64 GB | 128 GB |
| 32 GB | 64 GB | 128 GB | 256 GB |
| 64 GB | 128 GB | 256 GB | 512 GB |
| Configuration | 2 DIMMs | 4 DIMMs | 8 DIMMs |
|---|---|---|---|
| Number of DIMMs | 2 | 4 | 8 |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s |
| Maximum memory bandwidth | ~28% | ~54% | ~100% |
2. Single-CPU systems with 16 DIMM slots
The following table shows possible DIMM configurations for a single‑CPU system with a mainboard featuring 16 DIMM slots (for example ASUS KRPA‑U16 mainboards / ASUS RS500A‑E11‑RS12U systems):
| module size | 2 DIMMs | 4 DIMMs | 8 DIMMs | 16 DIMMs |
|---|---|---|---|---|
| 8 GB | 16 GB | 32 GB | 64 GB | 128 GB |
| 16 GB | 32 GB | 64 GB | 128 GB | 256 GB |
| 32 GB | 64 GB | 128 GB | 256 GB | 512 GB |
| 64 GB | 128 GB | 256 GB | 512 GB | 1.024 GB |
| Configuration | 2 DIMMs | 4 DIMMs | 8 DIMMs | 16 DIMMs |
|---|---|---|---|---|
| Number of DIMMs | 2 | 4 | 8 | 16 |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s | 2.933 MT/s (2 DPC) |
| Maximum memory bandwidth | ~28% | ~54% | ~100% | ~92% (*) |
- (*) In the referenced test report, 100% is quoted, but the tests were performed with 2.933 MT/s modules. Compared to 8 × 3.200 MT/s modules, populating all 16 slots reduces the maximum transfer rate to about 92%, because the CPU supports only 2.933 MT/s at 2 DIMMs per channel (2 DPC).
Officially supported speeds are DDR4‑3200 at 1 DPC and DDR4‑2933 at 2 DPC.
(See: [[https://www.tomshardware.com/news/amd-epyc-rome-sku-list-block-diagram,40115.html|AMD EPYC Rome SKU List and Block Diagram]] (tomshardware.com, 07.08.2019).)
3. Dual-CPU systems with 16 DIMM slots
In dual‑CPU systems with 16 DIMM slots (8 per CPU), typical balanced configurations are:
| module size | 4 DIMMs\\ (2 DIMMs per CPU) | 8 DIMMs\\ (4 DIMMs per CPU) | 16 DIMMs\\ (8 DIMMs per CPU) |
|---|---|---|---|
| 8 GB | 32 GB | 64 GB | 128 GB |
| 16 GB | 64 GB | 128 GB | 256 GB |
| 32 GB | 128 GB | 256 GB | 512 GB |
| 64 GB | 256 GB | 512 GB | 1.024 GB |
| Configuration | 4 DIMMs | 8 DIMMs | 16 DIMMs |
|---|---|---|---|
| Number of DIMMs per CPU | 2 | 4 | 8 |
| Total DIMMs in system | 4 | 8 | 16 |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s |
| Maximum memory bandwidth | ~28% | ~54% | ~100% |
4. Dual-CPU systems with 32 DIMM slots
In dual‑CPU systems with 32 DIMM slots (16 per CPU), the following configurations are common:
| module size | 4 DIMMs\\ (2 DIMMs per CPU) | 8 DIMMs\\ (4 DIMMs per CPU) | 16 DIMMs\\ (8 DIMMs per CPU) | 32 DIMMs\\ (16 DIMMs per CPU) |
|---|---|---|---|---|
| 8 GB | 32 GB | 64 GB | 128 GB | 256 GB |
| 16 GB | 64 GB | 128 GB | 256 GB | 512 GB |
| 32 GB | 128 GB | 256 GB | 512 GB | 1.024 GB |
| 64 GB | 256 GB | 512 GB | 1.024 GB | 2.048 GB |
| Configuration | 4 DIMMs | 8 DIMMs | 16 DIMMs | 32 DIMMs |
|---|---|---|---|---|
| Number of DIMMs per CPU | 2 | 4 | 8 | 16 |
| Total DIMMs in system | 4 | 8 | 16 | 32 |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s | 2.933 MT/s |
| Maximum memory bandwidth | ~28% | ~54% | ~100% | ~92% (*) |
- (*) Same reasoning as in the 16‑DIMM configuration: 16 DIMMs per CPU run at 2.933 MT/s (2 DPC), slightly reducing the achievable bandwidth versus 8 × 3.200 MT/s DIMMs.