This is an old revision of the document!
Table of Contents
Optimising the memory performance of AMD EPYC 7002 Rome and 7003 Milan
The processors in the AMD EPYC 7002 Rome and AMD EPYC 7003 Milan series have more than eight memory controllers that can be connected to a memory channel. Up to two DIMMs are possible per channel. To achieve the best possible overall performance, it is helpful to ensure a balanced allocation. This article presents frequently used DIMM configurations.
Basics
The following properties characterise processors of the AMD EPYC 7002/7003 series in terms of memory:
- 8 memory controllers per CPU
- 1 memory channel per memory controller
- 2 DIMMs per channel, with a maximum of 16 DIMMs per CPU and 32 DIMMs in a dual-CPU system
Configuration
The highest possible memory bandwidth is particularly relevant in the HPC environment. In other application areas, memory bandwidth has a lower influence on overall performance and depends on the specific application. For example, tests show that performance of up to 90% can be achieved with a memory bandwidth of just 35%. RAM assemblies with fewer modules reduce the maximum bandwidth. However, they consume less energy and offer good expansion potential for the future.
The following chart illustrates the correct configuration of the RAM slots.
Single-CPU systems with 8 DIMM slots
| module size | 2 DIMMs | 4 DIMMs | 8 DIMMs |
|---|---|---|---|
| 8 GB | 16 GB | 32 GB | 64 GB |
| 16 GB | 32 GB | 64 GB | 128 GB |
| 32 GB | 64 GB | 128 GB | 256 GB |
| 64 GB | 128 GB | 256 GB | 512 GB |
| Assembly | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-02-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-04-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-08-DIMMs.png |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s |
| Maximum memory bandwidth 1)) | ~28% | ~ 54 % | ~ 100 % |
Single CPU systems with 16 DIMM slots
The following table shows possible DIMM-configurations of a Single-CPU system with a mainboard with 16 DIMM slots (for example ASUS KRPA-U16 mainboards/the ASUS RS500A-E11-RS12U systems respectively):
| module size | 2 DIMMs | 4 DIMMs | 8 DIMMs | 16 DIMMs |
|---|---|---|---|---|
| 8 GB | 16 GB | 32 GB | 64 GB | 128 GB |
| 16 GB | 32 GB | 64 GB | 128 GB | 256 GB |
| 32 GB | 64 GB | 128 GB | 256 GB | 512 GB |
| 64 GB | 128 GB | 256 GB | 512 GB | 1.024 GB |
| Assembly | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-02-of-16-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-04-of-16-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-08-of-16-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Single-16-DIMMs.png |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s | 2.933 MT/s (2 DPC) |
| Maximum memory bandwidth 2)) | ~28% | ~ 54% | ~ 100 % | ~ 92% (*) |
- (*) In the listed test report, 100 per cent are listed. However, the tests have been performed with 2.933 MT/s modules. Compared to 8 x 3,200 MT/s modules, equipping the system with 16 modules results in approximately 92% of the maximum transfer rate, as when using 2 DPC (Dimms per channel) maximum 2.933 MT/s are supported by the CPU. 3)
Dual-CPU systems with 16 DIMM slots
| module size | 4 DIMMs (2 DIMMs per CPU) | 8 DIMMs (4 DIMMs per CPU) | 16 DIMMs (8 DIMMs pero CPU) |
|---|---|---|---|
| 8 GB | 32 GB | 64 GB | 128 GB |
| 16 GB | 64 GB | 128 GB | 256 GB |
| 32 GB | 128 GB | 256 GB | 512 GB |
| 64 GB | 256 GB | 512 GB | 1.024 GB |
| Assembly | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-04-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-08-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-16-DIMMs.png |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s |
| Maximum memory bandwidth 4)) | ~28% | ~ 54 % | ~ 100 % |
Dual-CPU systems with 32 DIMM slots
| module size | 4 DIMMs (2 DIMMs per CPU) | 8 DIMMs (4 DIMMs per CPU) | 16 DIMMs (8 DIMMs pro CPU) | 32 DIMMs (16 DIMMs per CPU) |
|---|---|---|---|---|
| 8 GB | 32 GB | 64 GB | 128 GB | 256 GB |
| 16 GB | 64 GB | 128 GB | 256 GB | 512 GB |
| 32 GB | 128 GB | 256 GB | 512 GB | 1.024 GB |
| 64 GB | 256 GB | 512 GB | 1.024 GB | 2.048 GB |
| Assembly | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-04-of-32-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-08-of-32-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-16-of-32-DIMMs.png | ![]() AMD-EPYC-Rome-DIMM-Performance-Dual-32-DIMMs.png |
| Maximum transfer rate | 3.200 MT/s | 3.200 MT/s | 3.200 MT/s | 2.933 MT/s |
| Maximum memory bandwidth 5)) | ~28% | ~ 54 % | ~ 100 % | ~ 92 % (*) |














