Kann der Speicherdruck auf Solaris zu SIGSEGVs in JVM führen?

Question

Ich führe mehrere speicherhungrige Java-Serverprozesse auf Sun JVMs in einem Solaris-Container aus. Kürzlich habe ich mehr Prozesse hinzugefügt und möglicherweise den gesamten verfügbaren Speicher erschöpft. Java-Prozesse stürzten unter Last zufällig ab. In jedem Fall war es ein SIGSEGV in der JVM im Thread GCTaskThread. An einem Punkt starb sogar top mit einem SIGSEGV.

Tötet Solaris Prozesse mit SIGSEGV, wenn der Speicher knapp wird?

Anstelle von SIGSEGV hätte ich OutOfMemoryErrors an zufälligen Orten erwartet. Ich dachte auch immer, dass segfaults in der JVM Fehler anzeigen (außer JNI ist beteiligt). Ich muss wissen, ob ich einen zusätzlichen JVM-Fehler gefunden habe.

Dies ist Solaris 10 und 32-Bit JRE 1.6u20.

---

# 
# A fatal error has been detected by the Java Runtime Environment: 
# 
# SIGSEGV (0xb) at pc=0xfeb6297c, pid=26468, tid=6 
# 
# JRE version: 6.0_20-b02 
# Java VM: Java HotSpot(TM) Server VM (16.3-b01 mixed mode solaris-sparc) 
# Problematic frame: 
# V [libjvm.so+0x76297c] 
# 
# If you would like to submit a bug report, please visit: 
# http://java.sun.com/webapps/bugreport/crash.jsp 
# 

--------------- T H R E A D --------------- 

Current thread (0x0003bc00): GCTaskThread [stack: 0x8b300000,0x8b380000] [id=6] 

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), si_addr=0x8b828bfb 

Registers: 
O0=0xf824d1d0 O1=0xdcdc43e8 O2=0xf824d1d3 O3=0x00000001 
O4=0xfee2d800 O5=0x00000003 O6=0x8b37f718 O7=0xfe546d18 
G1=0xfe55a854 G2=0x90d7f7b4 G3=0xfffffff4 G4=0x00000000 
G5=0xfee2dd5f G6=0x00000000 G7=0xfeed2200 Y=0x00000000 
PC=0xfeb6297c nPC=0xfeb62980 


Top of Stack: (sp=0x8b37f718) 
0x8b37f718: 8bc00000 00033158 fee3f000 fbc00000 
0x8b37f728: 70000000 1c000000 8b3a2000 00033200 
0x8b37f738: fee3b800 00486bfb 00000011 00000003 
0x8b37f748: f824d1d3 dbc00000 8b37f778 fe55a854 
0x8b37f758: ed204000 00001000 00000000 00000032 
0x8b37f768: 8737f7f0 0003c3a8 8b37f7ec 0191c568 
0x8b37f778: 0003c3a8 00000010 00000003 00000003 
0x8b37f788: 00000009 8b37f84c 00000004 8b37f84c 

Instructions: (pc=0xfeb6297c) 
0xfeb6296c: ee 06 21 b8 b3 36 60 09 b4 10 20 11 ec 05 e0 4c 
0xfeb6297c: f4 2d 80 19 81 c7 e0 08 81 e8 20 00 9d e3 bf a0 

Stack: [0x8b300000,0x8b380000], sp=0x8b37f718, free space=1fdfeb6297ck 
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) 
V [libjvm.so+0x76297c] 
V [libjvm.so+0x15a85c] 
V [libjvm.so+0x486acc] 
V [libjvm.so+0x169314] 
V [libjvm.so+0x16bda0] 
V [libjvm.so+0x24c2a4] 
V [libjvm.so+0x72a1a0] 


--------------- P R O C E S S --------------- 

Java Threads: (=> current thread) 
    0x027bf800 JavaThread "pool-3-thread-31" [_thread_blocked, id=1171, stack(0x90d00000,0x90d80000)] 
    0x028df000 JavaThread "AWT-EventQueue-0" [_thread_blocked, id=1169, stack(0x91400000,0x91480000)] 
    0x028dec00 JavaThread "AWT-Shutdown" [_thread_blocked, id=1168, stack(0x91700000,0x91780000)] 
    0x00dd4800 JavaThread "Product-producer-thread-7" [_thread_blocked, id=1165, stack(0x90b00000,0x90b80000)] 
    0x03a18400 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" [_thread_in_native, id=580, stack(0x85100000,0x85180000)] 
    0x00f83c00 JavaThread "BaseComTimerThread-9" daemon [_thread_blocked, id=287, stack(0x88d80000,0x88e00000)] 
    0x014d5000 JavaThread "PACCommand_7" [_thread_blocked, id=105, stack(0x86500000,0x86580000)] 
    0x0109b800 JavaThread "PACCommand_6" [_thread_blocked, id=104, stack(0x86600000,0x86680000)] 
    0x0109b000 JavaThread "PACCommand_5" [_thread_blocked, id=103, stack(0x86700000,0x86780000)] 
    0x00d75800 JavaThread "PACCommand_4" [_thread_blocked, id=102, stack(0x86800000,0x86880000)] 
    0x0106ac00 JavaThread "PACCommand_3" [_thread_blocked, id=101, stack(0x86900000,0x86980000)] 
    0x01529000 JavaThread "PACCommand_2" [_thread_blocked, id=100, stack(0x86a00000,0x86a80000)] 
    0x0131e800 JavaThread "PACCommand_1" [_thread_blocked, id=99, stack(0x86b00000,0x86b80000)] 
    0x01163400 JavaThread "BaseComSingleThread-1" daemon [_thread_blocked, id=91, stack(0x86d00000,0x86d80000)] 
    0x002f6800 JavaThread "FailoverExecutorThread-0" daemon [_thread_blocked, id=90, stack(0x87500000,0x87580000)] 
    0x00dd6800 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=89, stack(0x86f00000,0x86f80000)] 
    0x00b8fc00 JavaThread "NonGUI_5" [_thread_blocked, id=87, stack(0x86e00000,0x86e80000)] 
    0x0091d000 JavaThread "NonGUI_4" [_thread_blocked, id=86, stack(0x87000000,0x87080000)] 
    0x0191e800 JavaThread "NonGUI_3" [_thread_blocked, id=82, stack(0x87100000,0x87180000)] 
    0x01fa0400 JavaThread "NonGUI_2" [_thread_blocked, id=81, stack(0x87200000,0x87280000)] 
    0x018f6800 JavaThread "NonGUI_1" [_thread_blocked, id=80, stack(0x87300000,0x87380000)] 
    0x00ef0400 JavaThread "Timer legends" [_thread_blocked, id=79, stack(0x87400000,0x87480000)] 
    0x01179400 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=75, stack(0x87600000,0x87680000)] 
    0x009bd800 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=73, stack(0x87700000,0x87780000)] 
    0x00931800 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=71, stack(0x88980000,0x88a00000)] 
    0x00a6e400 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=69, stack(0x88a80000,0x88b00000)] 
    0x0113b400 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=67, stack(0x88b80000,0x88c00000)] 
    0x00c72000 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=65, stack(0x87900000,0x87980000)] 
    0x001bbc00 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=63, stack(0x87a00000,0x87a80000)] 
    0x00f45c00 JavaThread "Access Layer MDT Update Thread #0" daemon [_thread_blocked, id=60, stack(0x89900000,0x89980000)] 
    0x00a70c00 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=59, stack(0x87800000,0x87880000)] 
    0x00a68800 JavaThread "AWT-XAWT" daemon [_thread_in_native, id=53, stack(0x87b00000,0x87b80000)] 
    0x008ecc00 JavaThread "ProductTimer" [_thread_blocked, id=52, stack(0x87d00000,0x87d80000)] 
    0x00030400 JavaThread "DestroyJavaVM" [_thread_blocked, id=2, stack(0xfe300000,0xfe380000)] 
    0x00e34c00 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" [_thread_in_native, id=48, stack(0x87e00000,0x87e80000)] 
    0x00e32800 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=47, stack(0x87f00000,0x87f80000)] 
    0x00f68400 JavaThread "Proxy.cleaner" daemon [_thread_blocked, id=45, stack(0x88000000,0x88080000)] 
    0x00f67400 JavaThread "Thread-10" [_thread_blocked, id=44, stack(0x88100000,0x88180000)] 
    0x00f66400 JavaThread "Timer-1" [_thread_blocked, id=43, stack(0x88200000,0x88280000)] 
    0x00f64c00 JavaThread "CORBA cleaner" daemon [_thread_blocked, id=42, stack(0x88300000,0x88380000)] 
    0x00f64400 JavaThread "Timer-0" [_thread_blocked, id=41, stack(0x88400000,0x88480000)] 
    0x0117e800 JavaThread "BaseComExecutorThread-4" [_thread_blocked, id=40, stack(0x88500000,0x88580000)] 
    0x01fbd400 JavaThread "BaseComExecutorThread-3" [_thread_blocked, id=39, stack(0x88600000,0x88680000)] 
    0x011ad400 JavaThread "BaseComExecutorThread-2" [_thread_blocked, id=38, stack(0x88700000,0x88780000)] 
    0x011acc00 JavaThread "Product.scheduler.thread" [_thread_blocked, id=37, stack(0x88800000,0x88880000)] 
    0x00a9f800 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=35, stack(0x88e80000,0x88f00000)] 
    0x00e95000 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" [_thread_in_native, id=34, stack(0x89400000,0x89480000)] 
    0x00c14400 JavaThread "BaseComExecutorThread-1" [_thread_blocked, id=32, stack(0x89500000,0x89580000)] 
    0x00e92000 JavaThread "ConfigCacheCleaner4" daemon [_thread_blocked, id=31, stack(0x89600000,0x89680000)] 
    0x00f4e400 JavaThread "ConfigCacheCleaner3" daemon [_thread_blocked, id=30, stack(0x89700000,0x89780000)] 
    0x00cb6000 JavaThread "Orbacus:GIOPConnectionThreaded:ReceiverThread" daemon [_thread_in_native, id=29, stack(0x89800000,0x89880000)] 
    0x00ba0800 JavaThread "BaseComTimerThread-1" daemon [_thread_blocked, id=25, stack(0x89a00000,0x89a80000)] 
    0x00133000 JavaThread "TaskRouterThread-1" daemon [_thread_blocked, id=24, stack(0x89b00000,0x89b80000)] 
    0x00f3c000 JavaThread "NINJO_CACHE_1333544439197_ScratchConfigCache1.3_2.data" daemon [_thread_blocked, id=23, stack(0x89c00000,0x89c80000)] 
    0x00c00400 JavaThread "NINJO_CACHE_1333544439197_ScratchConfigCache1.3_1.data" daemon [_thread_blocked, id=22, stack(0x89d00000,0x89d80000)] 
    0x00ed6c00 JavaThread "NINJO_CACHE_1333544439197_ScratchConfigCache1.3.data" daemon [_thread_blocked, id=21, stack(0x89e00000,0x89e80000)] 
    0x00ba3c00 JavaThread "com.product.fwk.common.cache.impl.EHCacheManager@b02928" daemon [_thread_blocked, id=20, stack(0x89f00000,0x89f80000)] 
    0x00aa3400 JavaThread "com.product.fwk.common.cache.impl.EHCacheManager@b02928" daemon [_thread_blocked, id=19, stack(0x8a000000,0x8a080000)] 
    0x00ab5400 JavaThread "ConfigCacheCleaner2" daemon [_thread_blocked, id=18, stack(0x8a100000,0x8a180000)] 
    0x00c0d800 JavaThread "ConfigCacheCleaner1" daemon [_thread_blocked, id=17, stack(0x8a200000,0x8a280000)] 
    0x00e40000 JavaThread "ORB-Thread" [_thread_blocked, id=16, stack(0x8a300000,0x8a380000)] 
    0x00e3e800 JavaThread "ORBacus:Server:StarterThread" [_thread_in_native, id=15, stack(0x8a400000,0x8a480000)] 
    0x00127c00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=13, stack(0x8aa80000,0x8ab00000)] 
    0x00126000 JavaThread "CompilerThread1" daemon [_thread_blocked, id=12, stack(0x8ab80000,0x8ac00000)] 
    0x0JavaThread "CompilerThread0" daemon [_thread_blocked, id=11, stack(0x8ac80000,0x8ad00000)] 
    0x00122000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=10, stack(0x8ad80000,0x8ae00000)] 
    0x0010d000 JavaThread "Finalizer" daemon [_thread_blocked, id=9, stack(0x8ae80000,0x8af00000)] 
    0x0010b800 JavaThread "Reference Handler" daemon [_thread_blocked, id=8, stack(0x8af80000,0x8b000000)] 

Other Threads: 
    0x00109400 VMThread [stack: 0x8b080000,0x8b100000] [id=7] 
    0x0012ac00 WatcherThread [stack: 0x8a980000,0x8aa00000] [id=14] 

=>0x0003bc00 (exited) GCTaskThread [stack: 0x8b300000,0x8b380000] [id=6] 

VM state:at safepoint (normal execution) 

VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event]) 
[0x0002f578] Threads_lock - owner thread: 0x00109400 
[0x0002f988] Heap_lock - owner thread: 0x00dd4800 

Heap 
PSYoungGen  total 457280K, used 83928K [0xdbc00000, 0xfbc00000, 0xfbc00000) 
    eden space 391808K, 21% used [0xdbc00000,0xe0df6178,0xf3aa0000) 
    from space 65472K, 0% used [0xf3aa0000,0xf3aa0000,0xf7a90000) 
    to space 61632K, 4% used [0xf7fd0000,0xf8254130,0xfbc00000) 
PSOldGen  total 1048576K, used 331538K [0x9bc00000, 0xdbc00000, 0xdbc00000) 
    object space 1048576K, 31% used [0x9bc00000,0xaffc4b68,0xdbc00000) 
PSPermGen  total 49152K, used 45797K [0x8bc00000, 0x8ec00000, 0x9bc00000) 
    object space 49152K, 93% used [0x8bc00000,0x8e8b9690,0x8ec00000) 

Dynamic libraries: 
0x00010000  /opt/product/release_1.6/server/jre/bin/java 
0xff3a0000  /lib/libthread.so.1 
0xff370000  /opt/product/release_1.6/server/jre/bin/../lib/sparc/jli/libjli.so 
0xff350000  /lib/libdl.so.1 
0xff200000  /lib/libc.so.1 
0xff390000  /platform/SUNW,SPARC-Enterprise/lib/libc_psr.so.1 
0xfe400000  /opt/product/release_1.6/server/jre/lib/sparc/server/libjvm.so 
0xff1d0000  /lib/libsocket.so.1 
0xff1f0000  /usr/lib/libsched.so.1 
0xff1b0000  /lib/libm.so.1 
0xff180000  /usr/lib/libCrun.so.1 
0xff160000  /lib/libdoor.so.1 
0xff080000  /lib/libnsl.so.1 
0xfef80000  /lib/libm.so.2 
0xff050000  /lib/libscf.so.1 
0xff140000  /lib/libuutil.so.1 
0xff030000  /lib/libgen.so.1 
0xfef50000  /lib/libmd.so.1 
0xfef30000  /lib/libmp.so.2 
0xfeea0000  /opt/product/release_1.6/server/jre/lib/sparc/libverify.so 
0xfee60000  /opt/product/release_1.6/server/jre/lib/sparc/libjava.so 
0xfe3e0000  /opt/product/release_1.6/server/jre/lib/sparc/native_threads/libhpi.so 
0xfe2e0000  /lib/nss_files.so.1 
0xfe2b0000  /usr/lib/nss_vas4.so.1 
0xfe280000  /opt/quest/lib/libvtsmartcache.so.1 
0xfe180000  /opt/quest/lib/libvtutil.so.1 
0xfe230000  /opt/quest/lib/libvtcacheipc.so.1 
0xfe120000  /lib/libresolv.so.2 
0xfe100000  /opt/product/release_1.6/server/jre/lib/sparc/libzip.so 
0xfe0b0000  /usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3 
0x8ae50000  /opt/product/release_1.6/server/jre/lib/sparc/libnet.so 
0x89200000  /opt/product/release_1.6/server/jre/lib/sparc/libawt.so 
0x8ad20000  /opt/product/release_1.6/server/jre/lib/sparc/xawt/libmawt.so 
0x8ae10000  /usr/openwin//lib/libXext.so.0 
0x89080000  /usr/openwin//lib/libX11.so.4 
0x8ac50000  /usr/openwin//lib/libXtst.so.1 
0x8ac20000  /usr/openwin//lib/libXi.so.5 
0x88f80000  /opt/product/release_1.6/server/jre/lib/sparc/libfontmanager.so 
0x8b110000  /opt/product/release_1.6/server/jre/lib/sparc/libnio.so 
0x8ab60000  /lib/librt.so.1 
0x8ab40000  /lib/libaio.so.1 
0x8af10000  /usr/lib/libsendfile.so.1 
0x887d0000  /opt/product/release_1.6/server/jre/lib/sparc/libmanagement.so 
0x887b0000  /usr/openwin/lib/locale/common/xlibi18n.so.2 
0x886d0000  /usr/X11/lib/libXcursor.so.1 
0x886a0000  /usr/openwin/sfw/lib/libXrender.so.1 
0x8a850000  /opt/product/release_1.6/server/jre/lib/sparc/libdcpr.so 

VM Arguments: 
jvm_args: -Xmx1536M -Xms1536M -XX:MaxPermSize=256m -XX:-ReduceInitialCardMarks -Djava.ext.dirs=/opt/product/release_1.6/server/lib/ext -Djava.endorsed.dirs=/opt/product/release_1.6/server/lib/boot -Dorg.omg.CORBA.ORBClass=com.ooc.CORBA.ORB -Dorg.omg.CORBA.ORBSingletonClass=com.ooc.CORBA.ORBSingleton -Dcorba.port=23801 -Dserver.name=ProductServer -Dserver.basedir=/data/product -Dproduct.log.path=../../log -Dproduct.i18n.locale.language=en 
java_command: /opt/product/release_1.6/server/lib/product/ProductMain.jar /opt/product/release_1.6/server/config.properties 
Launcher Type: SUN_STANDARD 

Environment Variables: 
CLASSPATH=/opt/local/opkg/share/oracle/product/10.2.0/jdbc/lib/classes12.zip 
PATH=/usr/bin:/usr/sbin:/usr/local/bin:/opt/local/opkg/bin:/opt/local/opkg/sbin:/home/ext/user/bin:/opt/local/opkg/share/oracle/product/10.2.0/bin:/usr/bin:/usr/sbin:/usr/local/bin:/opt/local/opkg/bin:/opt/local/opkg/sbin:/home/users/user/bin:/usr/bin:/home/users/user/bin 
LD_LIBRARY_PATH=/opt/product/release_1.6/server/jre/lib/sparc/server:/opt/product/release_1.6/server/jre/lib/sparc:/opt/product/release_1.6/server/jre/../lib/sparc:/opt/local/opkg/share/oracle/product/10.2.0/lib32:/opt/local/opkg/share/oracle/product/10.2.0/lib:/opt/local/opkg/share/oracle/product/10.2.0/jdbc/lib32 
SHELL=/bin/bash 
DISPLAY=localhost:63.0 

Signal Handlers: 
SIGSEGV: [libjvm.so+0x84dd5c], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGBUS: [libjvm.so+0x84dd5c], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGFPE: [libjvm.so+0x1c0330], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGPIPE: [libjvm.so+0x1c0330], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGXFSZ: [libjvm.so+0x1c0330], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGILL: [libjvm.so+0x1c0330], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000 
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000 
SIGQUIT: [libjvm.so+0x72c934], sa_mask[0]=0xffbffeff, sa_flags=0x00000004 
SIGHUP: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000 
SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000 
SIGTERM: [libjvm.so+0x72c934], sa_mask[0]=0xffbffeff, sa_flags=0x00000004 
SIG39: [libjvm.so+0x730360], sa_mask[0]=0x00000000, sa_flags=0x00000008 
SIG40: [libjvm.so+0x1c0330], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c 


--------------- S Y S T E M --------------- 

OS:      Solaris 10 10/09 s10s_u8wos_08a SPARC 
      Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. 
         Use is subject to license terms. 
          Assembled 16 September 2009 

uname:SunOS 5.10 Generic_142909-17 sun4u (T2 libthread) 
rlimit: STACK 8192k, CORE infinity, NOFILE 8192, AS infinity 
load average:7.79 3.58 3.31 

CPU:total 4 has_v8, has_v9, popc, has_vis1, has_vis2, is_ultra3 

Memory: 8k page, physical 15728640k(1399920k free) 

vm_info: Java HotSpot(TM) Server VM (16.3-b01) for solaris-sparc JRE (1.6.0_20-b02), built on Apr 12 2010 14:27:21 by "" with Workshop 5.8 

time: Wed Apr 4 13:50:18 2012 
elapsed time: 2982 seconds 

Answer 1

Der Faden Garbage Collector wird durch die Objekte im Speicher zu Fuß und versucht zu dereferenzieren eines nicht vorhandenen Zeiger 0x8b828bfb.

Ich würde alle jni Aufrufe untersuchen, die Objekte erstellen und an den jvm zurückgeben.
Es scheint, dass Sie CORBA verwenden, die Objekte für die JVM generieren kann. Je nachdem, wie Sie die Corba-Datei verwenden, liegt möglicherweise ein Fehler im Datenstrom oder in der Art der Verwendung vor, der ein Problem verursacht.

Da Sie ein Problem mit "top" haben, möchten Sie vielleicht auch überprüfen, dass der Speicher gut ist. Ein schlechter Speicherchip kann ein System zum Wackeln bringen. Es gibt eine Speicherdiagnoseroutine im Startabschluß von Sparc-Systemen. Die für den Austausch verwendete Festplatte könnte ebenfalls Probleme mit dem Arbeitsspeicher verursachen.