ちょっと検証してみたメモ。
検証結果
- vm.nr_hugepages を設定する
[root@yazekats-linux ~]# grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@yazekats-linux ~]# sysctl -w vm.nr_hugepages=200 vm.nr_hugepages = 200 [root@yazekats-linux ~]# grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 200 HugePages_Free: 200 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
- Huge Page を System V 共有メモリとして使うプログラムを実行する
[root@yazekats-linux hugepage_test]# gdb ./hugepage-shm GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/yazekats/Documents/github/yoheia/misc/hugepage_test/hugepage-shm...done. (gdb) b 68 Breakpoint 1 at 0x40076c: file hugepage-shm.c, line 68. (gdb) r Starting program: /home/yazekats/Documents/github/yoheia/misc/hugepage_test/hugepage-shm shmid: 0x25002c Breakpoint 1, main () at hugepage-shm.c:68 68 if (shmaddr == (char *)-1) { Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64 (gdb)
- Huge Page が割り当てられている
[yazekats@yazekats-linux ~]$ grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 200 HugePages_Free: 200 HugePages_Rsvd: 128 ★ HugePages_Surp: 0 Hugepagesize: 2048 kB
HugePages_Total is the size of the pool of huge pages.
HugePages_Free is the number of huge pages in the pool that are not yet
allocated.
HugePages_Rsvd★ is short for "reserved," and is the number of huge pages for
which a commitment to allocate from the pool has been made,
but no allocation has yet been made. Reserved huge pages
guarantee that an application will be able to allocate a
huge page from the pool of huge pages at fault time.
HugePages_Surp is short for "surplus," and is the number of huge pages in
the pool above the value in /proc/sys/vm/nr_hugepages. The
maximum number of surplus huge pages is controlled by
/proc/sys/vm/nr_overcommit_hugepages.
- 実行したプログラムのPIDを調べる
[yazekats@yazekats-linux ~]$ ps auxw|grep [h]ugepage-shm root 2509 0.0 0.0 187884 12928 pts/0 S+ 12:09 0:00 gdb ./hugepage-shm root 2524★ 0.0 0.0 3972 284 pts/0 t 12:10 0:00 /home/yazekats/Documents/github/yoheia/misc/hugepage_test/hugepage-shm
- PID:2524 が作成した共有メモリセグメントの shmid を調べる。
[root@yazekats-linux ~]# ipcs -p|grep 2524 2424876★ root 2524 2524
- shmid:2424876 の共有メモリセグメントを確認する。
[root@yazekats-linux ~]# ipcs -m |grep 2424876 0x00000002 2424876 root 600 268435456★ 1
268435456 bytes = 256MB使っている
- プログラムを終了する
(gdb) quit A debugging session is active. Inferior 1 [process 2524] will be killed. Quit anyway? (y or n) y
- プロセスがないことを確認する。
[yazekats@yazekats-linux ~]$ ps auxw|grep [h]ugepage-shm [yazekats@yazekats-linux ~]$
- Huge Page は解放されていない。
[yazekats@yazekats-linux ~]$ grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 200 HugePages_Free: 200 HugePages_Rsvd: 128 HugePages_Surp: 0 Hugepagesize: 2048 kB
- 共有メモリも解放されない。
[root@yazekats-linux ~]# ipcs -p|grep 2524 2424876 root 2524 2524 [root@yazekats-linux ~]# ipcs -m |grep 2424876 0x00000002 2424876 root 600 268435456 0
- ipcrm コマンドで共有メモリを解放する
[root@yazekats-linux ~]# ipcrm -m 2424876
- Huge Page も共有メモリも解放されている
[root@yazekats-linux ~]# grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 200 HugePages_Free: 200 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@yazekats-linux ~]# ipcs -p|grep 2524 [root@yazekats-linux ~]# ipcs -m |grep 2424876
参考
hugepage-shm.c のソースコード
- hugepage-shm.c\vm\selftests\testing\tools - kernel/git/torvalds/linux.git - Linux kernel source tree
/* * hugepage-shm: * * Example of using huge page memory in a user application using Sys V shared * memory system calls. In this example the app is requesting 256MB of * memory that is backed by huge pages. The application uses the flag * SHM_HUGETLB in the shmget system call to inform the kernel that it is * requesting huge pages. * * For the ia64 architecture, the Linux kernel reserves Region number 4 for * huge pages. That means that if one requires a fixed address, a huge page * aligned address starting with 0x800000... will be required. If a fixed * address is not required, the kernel will select an address in the proper * range. * Other architectures, such as ppc64, i386 or x86_64 are not so constrained. * * Note: The default shared memory limit is quite low on many kernels, * you may need to increase it via: * * echo 268435456 > /proc/sys/kernel/shmmax * * This will increase the maximum size per shared memory segment to 256MB. * The other limit that you will hit eventually is shmall which is the * total amount of shared memory in pages. To set it to 16GB on a system * with a 4kB pagesize do: * * echo 4194304 > /proc/sys/kernel/shmall */ #include <stdlib.h> #include <stdio.h> #include <sys/types.h> #include <sys/ipc.h> #include <sys/shm.h> #include <sys/mman.h> #ifndef SHM_HUGETLB #define SHM_HUGETLB 04000 #endif #define LENGTH (256UL*1024*1024) #define dprintf(x) printf(x) /* Only ia64 requires this */ #ifdef __ia64__ #define ADDR (void *)(0x8000000000000000UL) #define SHMAT_FLAGS (SHM_RND) #else #define ADDR (void *)(0x0UL) #define SHMAT_FLAGS (0) #endif int main(void) { int shmid; unsigned long i; char *shmaddr; shmid = shmget(2, LENGTH, SHM_HUGETLB | IPC_CREAT | SHM_R | SHM_W); if (shmid < 0) { perror("shmget"); exit(1); } printf("shmid: 0x%x\n", shmid); shmaddr = shmat(shmid, ADDR, SHMAT_FLAGS); if (shmaddr == (char *)-1) { perror("Shared memory attach failure"); shmctl(shmid, IPC_RMID, NULL); exit(2); } printf("shmaddr: %p\n", shmaddr); dprintf("Starting the writes:\n"); for (i = 0; i < LENGTH; i++) { shmaddr[i] = (char)(i); if (!(i % (1024 * 1024))) dprintf("."); } dprintf("\n"); dprintf("Starting the Check..."); for (i = 0; i < LENGTH; i++) if (shmaddr[i] != (char)i) { printf("\nIndex %lu mismatched\n", i); exit(3); } dprintf("Done.\n"); if (shmdt((const void *)shmaddr) != 0) { perror("Detach failure"); shmctl(shmid, IPC_RMID, NULL); exit(4); } shmctl(shmid, IPC_RMID, NULL); return 0; }
hugepage-shm.c のコンパイル手順
[yazekats@yazekats-linux hugepage_test]$ gcc -g -o hugepage-shm hugepage-shm.c [yazekats@yazekats-linux hugepage_test]$ ls -1 hugepage-shm ★←実行可能ファイルができる hugepage-shm.c
hugepage-shm.c の実行手順
[root@yazekats-linux hugepage_test]# ./hugepage-shm shmid: 0x22802c shmaddr: 0x7f8aa5600000 Starting the writes: ................................................................................................................................................................................................................................................................ Starting the Check...Done.
環境
[root@yazekats-linux ~]# cat /etc/issue Oracle Linux Server release 6.4 Kernel \r on an \m [root@yazekats-linux ~]# uname -r 2.6.39-400.17.1.el6uek.x86_64
補足（2014/12/14）：
vm.nr_hugepages を 0 にしても解放されません。
[root@yazekats-linux ~]# sysctl -w vm.nr_hugepages=0 vm.nr_hugepages = 0 [root@yazekats-linux ~]# grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 128 HugePages_Free: 128 HugePages_Rsvd: 128 HugePages_Surp: 128 Hugepagesize: 2048 kB [root@yazekats-linux ~]# tail /etc/sysctl.conf kernel.msgmax = 65536 # Controls the maximum shared segment size, in bytes kernel.shmmax = 68719476736 # Controls the maximum number of shared memory segments, in pages kernel.shmall = 4294967296 vm.nr_hugepages=0 [root@yazekats-linux ~]# sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 vm.nr_hugepages = 0 [root@yazekats-linux ~]# grep -i hugepage /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 128 HugePages_Free: 128 HugePages_Rsvd: 128 HugePages_Surp: 128 Hugepagesize: 2048 kB
