HeteroOS: OS Design for Heterogeneous Memory Management in Datacenters

Heterogeneous memory management combined with server virtualization in datacenters is expected to increase the software and OS management complexity. State-of-the-art solutions rely exclusively on the hypervisor (VMM) for expensive page hotness tracking and migrations, limiting the benefits from heterogeneity. To address this, we design HeteroOS, a novel application-transparent OS-level solution for managing memory heterogeneity in virtualized systems. The HeteroOS design first makes the guest-OSes heterogeneityaware, and then extracts rich OS-level information about applications' memory usage to place data in the 'right' memory, avoiding page migrations. When such proactive placements are not possible, HeteroOS combines the power of the guest-OSes' information about applications with the VMM's hardware control to track for hotness and migrate only performance-critical pages. Finally, HeteroOS also designs an efficient heterogeneous memory sharing across multiple guest-VMs. Evaluation of HeteroOS with memory, storage, and network-intensive datacenter applications show up to 2x performance improvement compared to the state-of-the-art VMMexclusive approach.