It is enabled as default for glibc.malloc.hugetlb set to 2 or higher. It also uses a non configurable minimum value and maximum value, currently set respectively to 1 and 4 selected huge page size.
The arena allocation with huge pages does not use MAP_NORESERVE. As indicate by kernel internal documentation , the flag might trigger a SIGBUS on soft page faults if at memory access there is no left pages in the pool.
On systems without a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB) allocation failure. To improve test coverage it is required to create a pool with some allocated pages.
Checked on x86_64-linux-gnu with no reserved pages, 10 reserved pages (which trigger mmap(MAP_HUGETBL) failures) and with 256 reserved pages (which does not trigger mmap(MAP_HUGETLB) failures).
c1beb51d08 malloc: Add Huge Page support to arenas
malloc/Makefile | 7 ++-
malloc/arena.c | 134 ++++++++++++++++++++++++++++++++++++++------------------
malloc/malloc.c | 2 +-
3 files changed, 99 insertions(+), 44 deletions(-)