impdp: ORA-04030: out of process memory when trying to allocate 4088 bytes (PLS CGA hp,pdzgM64_New_Link)

Even on Oracle 11.2.0.4.x there seems to be problem with “ORA-04030: out of process memory when trying to allocate 4088 bytes (PLS CGA hp,pdzgM64_New_Link)”. I’ve just hit it during DataPump import. Alert log states something like this:

ORA-04030: out of process memory when trying to allocate 1052696 bytes (pga heap,log read buffer)
ORA-04030: out of process memory when trying to allocate 7438704 bytes (pga heap,kgh stack)
ORA-04030: out of process memory when trying to allocate 4088 bytes (PLS CGA hp,pdzgM64_New_Link)
Dumping diagnostic data in directory=[cdmp_20141106193106], requested by (instance=2, osid=4893 (DW00)), summary=[incident=240603]

Detailed incident trace shows:

mmap(offset=207282176, len=8192) failed with errno=12 for the file ora_dw00_DBSIDNAME3
[..]
*** 2014-11-06 19:48:31.300
OPIRIP: Uncaught error 27102. Error stack:
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 108
Additional information: 4521984
ORA-00448: normal completion of background process
ORA-31671: Worker process DW00 had an unhandled exception.
ORA-04030: out of process memory when trying to allocate 16048 bytes (session heap,kuxLpxAlloc)
ORA-06512: at "SYS.KUPW$WORKER", line 1887

/usr/include/asm-generic/errno-base.h states that 12 is exactly ENOMEM error code. Investigation for mmap() syscall shows that ENOMEM stands for “No memory is available, or the process’s maximum number of mappings would have been exceeded.”
In Linux apparently there seems to be max_map_count limit defined as follows:

The max_map_count file allows for the restriction of the number of VMAs (Virtual Memory Areas) that a particular process can own. A Virtual Memory Area is a contiguous area of virtual address space. These areas are created during the life of the process when the program attempts to memory map a file, links to a shared memory segment, or allocates heap space. Tuning this value limits the amount of these VMAs that a process can own. Limiting the amount of VMAs a process can own can lead to problematic application behavior because the system will return out of memory errors when a process reaches its VMA limit but can free up lowmem for other kernel uses. If your system is running low on memory in the NORMAL zone, then lowering this value will help free up memory for kernel use.

This seems to be simple check in http://lxr.free-electrons.com/source/mm/mmap.c and seemed to be switched to dynamic sysctl value at times of 2.6.5 kernels (https://www.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.5-rc3/2.6.5-rc3-mm3/broken-out/max_map_count-sysctl.patch) in do_mmap_pgoff() function.

Simple workaround of increasing this value via sysctl works like a harm. The interesting fact is that oracle-validated RPM does not change this value, but it could.

Comments are closed.