The Need for Precise and Efficient Memory Capacity Budgeting

Modern high performance computing (HPC) systems pack hundreds of CPU cores to enable extreme parallelism. However, with increasing core counts, the effective per-core memory capacity is reducing. Reducing performance bottlenecks require precise monitoring and budgeting of application memory capacity requirements for attaining high performance, maximum resource efficiency, and low performance variability. Unfortunately, current operating systems (OS) and their toolsets are inaccurate, lack the capability to precisely measure the memory requirements of applications, forcing system administrators to either underestimate or over-provision memory, consequently compromising performance or resource efficiency, respectively. In this paper, we decipher the memory budgeting limitations in current OSes and their impact on both homogeneous and heterogeneous memory systems (e.g., nonvolatile memory). The limitations mainly stem from the mismatch between application-level and global memory accounting in the OS memory manager, fixing which can be prohibitively expensive at runtime. Our analysis of popular HPC workloads using widely-used memory budgeting strategies and deep instrumentation of the memory management layer reveals that imprecise budgeting can reduce performance by more than 1.65x and 2.05x in homogeneous and heterogeneous memory systems respectively. The program's memory requirement increases by up to 25x without significant performance gains. We also briefly describe our ongoing research approach to redesign the budgeting mechanisms in the OS.