1. During a recent migration exercise at a customer site the following error was encountered whilst migrating one of the databases:
2009-04-08-14.21.06.625141+060 E61438A838 LEVEL: Error (OS)
PID : 18892 TID : 164 PROC : db2sysc 0
INSTANCE: db2inst1 NODE : 000
EDUID : 164 EDUNAME: db2pclnr (MQTRACK) 0
FUNCTION: DB2 UDB, oper system services, sqloLioAIOCollect, probe:100
MESSAGE : ZRC=0×870F0024=-2029060060=SQLO_MEM “out of memory”
DIA8533C The system memory limit was reached.
CALLED : OS, -, aio_return
OSERR : ENOMEM (12) “Not enough space”
DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
0xFFFFFFFF643FD328 : 0000 0005 0000 0000 ……..
DATA #2 : unsigned integer, 8 bytes
8192
DATA #3 : signed integer, 8 bytes
6684672
DATA #4 : String, 105 bytes
Search for ossError*Analysis probe point after this log entry for further
self-diagnosis of this problem.
2009-04-08-14.21.06.628605+060 I62277A2206 LEVEL: Error (OS)
PID : 18892 TID : 164 PROC : db2sysc 0
INSTANCE: db2inst1 NODE : 000
EDUID : 164 EDUNAME: db2pclnr (MQTRACK) 0
FUNCTION: DB2 Common, OSSe, ossErrorIOAnalysis, probe:100
CALLED : OS, -, aio_return
OSERR : ENOMEM (12) “Not enough space”
DATA #1 : String, 110 bytes
A total of 5 analysis will be performed :
- User info
- ulimit info
- Memory info
Target file handle = 5
DATA #2 : String, 190 bytes
Real user ID of current process = 10002
Effective user ID of current process = 10002
Real group ID of current process = 1116
Effective group ID of current process = 1116
DATA #3 : String, 353 bytes
Current process limits (unit in bytes except for nofiles) :
mem (S/H) = unlimited / unlimited
core (S/H) = unlimited / unlimited
cpu (S/H) = unlimited / unlimited
data (S/H) = unlimited / unlimited
fsize (S/H) = unlimited / unlimited
nofiles (S/H) = 65536 / 65536
stack (S/H) = 8388608 / unlimited
rss (S/H) = 0 / 0
DATA #4 : String, 119 bytes
System RAM information (in megabytes) :
Total = 8064
Free = 4572
Available = -1
Addressable = -1
DATA #5 : String, 69 bytes
Swap space information (in megabytes) :
Total = 8194
Free = 8194
DATA #6 : String, 117 bytes
Virtual Memory Information (in megabytes) :
Total = 16258
Reserved = -1
Available = -1
Free = 12766
CALLSTCK:
[0] 0xFFFFFFFF781947B4 ossLogSysRC + 0×3A0
[1] 0xFFFFFFFF78186F60 ossErrorNameMapSystem + 0×1AC0
[2] 0xFFFFFFFF7C45F250 sqloSystemErrorHandler + 0×860
[3] 0xFFFFFFFF7C4B5A94 __1cUSQdDLO_LIO_HANDLE_DATARsqloLioAIOCollect6MLpnXSQdDLO_LIO_COLLECT_STATUS_ppnLSQdDLO_IO_REQdD__i_ + 0×6E4
[4] 0xFFFFFFFF7C4B646C sqloLioCollectNBlocks + 0×51C
[5] 0xFFFFFFFF7ADD789C __1cWsqlbClnrCollectSomeAIO6FpnMSQdDLB_CLNR_CB_L_v_ + 0×74
[6] 0xFFFFFFFF7ADD82D4 __1cVsqlbClnrCollectAllAIO6FpnMSQdDLB_CLNR_CB__v_ + 0×74
[7] 0xFFFFFFFF7ADDAAFC __1cQsqlbClnrFindWork6FpnMSQdDLB_CLNR_CB__i_ + 0×11F4
[8] 0xFFFFFFFF7ADDBAE8 __1cSsqlbClnrEntryPoint6FpCI_v_ + 0xD0
[9] 0xFFFFFFFF7C4CE414 sqloEDUEntry + 0×3A4
DB2 subsequently went into a panic and the instance crashed. This happened for another database as well. We resolved the error by changing the stack size value in unlimit from 8K to “unlimited”.