jvm crash 的崩溃日志详细分析及注意点
Posted onjvm crash 的崩溃日志详细分析及注意点
生成
生成error 文件的路径：你可以通过参数设置-XX:ErrorFile=/path/hs_error%p.log, 默认是在java运行的当前目录 [default: ./hs_err_pid%p.log]
参数-XX:OnError 可以在crash退出的时候执行命令，格式是-XX:OnError=“string”,
可以是命令的集合，用分号做分隔符, 可以用"%p"来取到当前进程的ID. 例如：
// -XX:OnError="pmap %p" // show memory map
// -XX:OnError="gcore %p; dbx - %p" // dump core and launch debugger
在linux中系统会fork出一个子进程去执行shell的命令，因为是用fork可能会内存不够的情况，注意修改你的 /proc/sys/vm/overcommit_memory 参数,不清楚为什么这里不使用vfork
- -XX:+ShowMessageBoxOnError 参数，当jvm crash的时候在linux里会启动gdb 去分析和调式，适合在测试环境中使用。
什么情况下不会生成error文件
linux 内核在发生OOM的时候会强制kill一些进程, 可以在/var/logs/messages中查找
Error crash 文件的几个重要部分
a. 错误信息概要
[plain] view plaincopy
- /# A fatal error has been detected by the Java Runtime Environment:
- /#
- /# SIGSEGV (0xb) at pc=0x0000000000043566, pid=32046, tid=1121192256
- /#
- /# JRE version: 6.0_17-b04
- /# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.3-b01 mixed mode linux-amd64 )
- /# Problematic frame:
- /# C 0x0000000000043566
- /#
- /# If you would like to submit a bug report, please visit:
- /# http://java.sun.com/webapps/bugreport/crash.jsp
- /# The crash happened outside the Java Virtual Machine in native code.
- /# See problematic frame for where to report the bug.
SIGSEGV 错误的信号类型
pc 就是IP/PC寄存器值也就是执行指令的代码地址
pid 就是进程id
/# Problematic frame:
/# V [libjvm.so+0x593045]
就是导致问题的动态链接库函数的地址
pc 和 +0x593045 指的是同一个地址，只是一个是动态的偏移地址，一个是运行的虚拟地址
b.信号信息
Java中在linux 中注册的信号处理函数，中间有2个参数info, ucvoid
[cpp] view plaincopy
- static void crash_handler(int sig, siginfo_t/ info, void/ ucVoid) {
- // unmask current signal
- sigset_t newset;
- sigemptyset(&newset);
- sigaddset(&newset, sig);
- sigprocmask(SIG_UNBLOCK, &newset, NULL);
- VMError err(NULL, sig, NULL, info, ucVoid);
- err.report_and_die();
- }
在crash report中的信号错误提示
[plain] view plaincopy
- siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000043566
信号的详细信息和si_addr 出错误的内存，都保存在siginfo_t的结构体中，也就是信号注册函数crash_handler里的参数info,内核会保存导致错误的内存地址在用户空间的信号结构体中siginfo_t，这样在进程在注册的信号处理函数中可以取得导致错误的地址。
c.寄存器信息
[plain] view plaincopy
- Registers:
- RAX=0x00002aacb5ae5de2, RBX=0x00002aaaaf46aa48, RCX=0x0000000000000219, RDX=0x00002aaaaf46b920
- RSP=0x0000000042d3f968, RBP=0x0000000042d3f9c8, RSI=0x0000000042d3f9e8, RDI=0x0000000045aef9b8
- R8 =0x0000000000000f80, R9 =0x00002aaab3d30ce8, R10=0x00002aaaab138ea1, R11=0x00002b017ae65110
- R12=0x0000000042d3f6f0, R13=0x00002aaaaf46aa48, R14=0x0000000042d3f9e8, R15=0x0000000045aef800
- RIP=0x0000000000043566, EFL=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000014
- TRAPNO=0x000000000000000e
寄存器的信息就保存在b部分的信号处理函数参数 (ucontext_t/*)usVoid中
在X86架构下：
[cpp] view plaincopy
- void os::print_context(outputStream /st, void /context) {
- if (context == NULL) return;
- ucontext_t /uc = (ucontext_t/)context;
- st->print_cr("Registers:");
- /#ifdef AMD64
- st->print( "RAX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RAX]);
- st->print(", RBX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RBX]);
- st->print(", RCX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RCX]);
- st->print(", RDX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RDX]);
- st->cr();
- st->print( "RSP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RSP]);
- st->print(", RBP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RBP]);
- st->print(", RSI=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RSI]);
- st->print(", RDI=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RDI]);
- st->cr();
- st->print( "R8 =" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R8]);
- st->print(", R9 =" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R9]);
- st->print(", R10=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R10]);
- st->print(", R11=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R11]);
- st->cr();
- st->print( "R12=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R12]);
- st->print(", R13=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R13]);
- st->print(", R14=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R14]);
- st->print(", R15=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_R15]);
- st->cr();
- st->print( "RIP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_RIP]);
- st->print(", EFL=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EFL]);
- st->print(", CSGSFS=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_CSGSFS]);
- st->print(", ERR=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_ERR]);
- st->cr();
- st->print(" TRAPNO=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_TRAPNO]);
- /#else
- st->print( "EAX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EAX]);
- st->print(", EBX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EBX]);
- st->print(", ECX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_ECX]);
- st->print(", EDX=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EDX]);
- st->cr();
- st->print( "ESP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_UESP]);
- st->print(", EBP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EBP]);
- st->print(", ESI=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_ESI]);
- st->print(", EDI=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EDI]);
- st->cr();
- st->print( "EIP=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EIP]);
- st->print(", CR2=" INTPTR_FORMAT, uc->uc_mcontext.cr2);
- st->print(", EFLAGS=" INTPTR_FORMAT, uc->uc_mcontext.gregs[REG_EFL]);
- /#endif // AMD64
- st->cr();
- st->cr();
- intptr_t /sp = (intptr_t /)os::Linux::ucontext_get_sp(uc);
- st->print_cr("Top of Stack: (sp=" PTR_FORMAT ")", sp);
- print_hex_dump(st, (address)sp, (address)(sp + 8/*sizeof(intptr_t)), sizeof(intptr_t));
- st->cr();
- // Note: it may be unsafe to inspect memory near pc. For example, pc may
- // point to garbage if entry point in an nmethod is corrupted. Leave
- // this at the end, and hope for the best.
- address pc = os::Linux::ucontext_get_pc(uc);
- st->print_cr("Instructions: (pc=" PTR_FORMAT ")", pc);
- print_hex_dump(st, pc - 16, pc + 16, sizeof(char));
- }
寄存器的信息在分析出错的时候是非常重要的 打印出执行附近的部分机器码
[plain] view plaincopy
- Instructions: (pc=0x00007f48f14ef51a)
- 0x00007f48f14ef4fa: 90 90 55 48 89 e5 48 81 ec 98 9f 00 00 48 89 bd
- 0x00007f48f14ef50a: f8 5f ff ff 48 89 b5 f0 5f ff ff b8 00 00 00 00
- 0x00007f48f14ef51a: c7 00 01 00 00 00 c6 85 00 60 ff ff ff c9 c3 90
- 0x00007f48f14ef52a: 90 90 90 90 90 90 55 48 89 e5 53 48 8d 1d 94 00
在instruction 部分中会打印出部分的机器码
格式是
[cpp] view plaincopy
- 地址：机器码
第一种使用udis库里带的udcli工具来反汇编
命令：
[cpp] view plaincopy
- echo '90 90 55 48 89 e5 48 81 ec 98 9f 00 00 48 89 bd' | udcli -intel -x -64 -o 0x00007f48f14ef4fa
显示出对应的汇编
第二种可以用
[plain] view plaincopy
- objectdump -d -C libjvm.so >> jvmsodisass.dump
查找偏移地址 0x593045， 就是当时的执行的汇编，然后结合上下文，源码推测出问题的语句。
d.寄存器对应的内存的值
[cpp] view plaincopy
- RAX=0x0000000000000000 is an unknown value
- RBX=0x000000041a07d1e8 is an oop
- {method}
- klass: {other class}
- RCX=0x0000000000000000 is an unknown value
- RDX=0x0000000040111800 is a thread
- RSP=0x0000000041261b88 is pointing into the stack for thread: 0x0000000040111800
- RBP=0x000000004126bb20 is pointing into the stack for thread: 0x0000000040111800
- RSI=0x000000004126bb80 is pointing into the stack for thread: 0x0000000040111800
- RDI=0x00000000401119d0 is an unknown value
- R8 =0x0000000040111c40 is an unknown value
- R9 =0x00007f48fcc8b550:
in /usr/java/jdk1.6.0_30/jre/lib/amd64/server/libjvm.so at 0x00007f48fc206000 - R10=0x00007f48f8ca7d41 is an Interpreter codelet
- method entry point (kind = native) [0x00007f48f8ca7ae0, 0x00007f48f8ca8320] 2112 bytes
- R11=0x00007f48fc98f270:
in /usr/java/jdk1.6.0_30/jre/lib/amd64/server/libjvm.so at 0x00007f48fc206000 - R12=0x0000000000000000 is an unknown value
- R13=0x000000041a07d1e8 is an oop
- {method}
- klass: {other class}
- R14=0x000000004126bb88 is pointing into the stack for thread: 0x0000000040111800
- R15=0x0000000040111800 is a thread
jvm 会通过寄存器的值对找对应的对象，也是一个比较好的参考
e. 其他的信息
error 里面还有一些线程信息，还有当时内存映像信息，这些都可以作为分析的部分参考
crash 报告可以大概的反应出一个当时的情况，特别是在没有core dump的时候，是比较有助于帮助分析的，但如果有core dump的话，最终还是core dump能快速准确的发现问题原因。 来源： [http://blog.csdn.net/raintungli/article/details/7642575/#comments](http://blog.csdn.net/raintungli/article/details/7642575#comments)
