CentOS / RHEL 6 : How to configure kdump

kdump is an advanced crash dumping mechanism. When enabled, the system is booted from the context of another kernel. This second kernel reserves a small amount of memory, and its only purpose is to capture the core dump image in case the system crashes. Since being able to analyze the core dump helps significantly to determine the exact cause of the system failure, it is strongly recommended to have this feature enabled. This Note explains how to configure, test, and use the kdump service in CentOS/RHEL 6.

1. Install the kexec-tools package if not already installed
To use the kdump service, you must have the kexec-tools package installed.

# yum install kexec-tools

2. Configuring Memory Usage in GRUB
To configure the amount of memory that is reserved for the kdump kernel, modify file /boot/grub/grub.conf and add the crashkernel=[size]M (or crashkernel=auto) parameter to the list of kernel options.And then reboot it to make it effect.

For example:

# vi /boot/grub/grub.conf
...
title Oracle Linux Server-uek (2.6.39-200.24.1.el6uek.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.39-200.24.1.el6uek.x86_64 ro root=/dev/mapper/vg_ol6desktop-lv_root crashkernel=128M
    initrd /initramfs-2.6.39-200.24.1.el6uek.x86_64.img
...

# reboot

3. Configuring Dump Location
To configure kdump, we need to edit the configuration file /etc/kdump.conf. The default option is to store the vmcore file in the /var/crash/ directory of the local file system. To change the local directory in which the core dump is to be saved, remove the hash sign (“#”) from the beginning of the #path /var/crash line, and replace the value with a desired directory path.

For example:

path /usr/local/cores

Optionally, you can also save the core dump directly to a raw partition.
For example:

raw /dev/sdb4

To store the dump to a remote machine using the NFS protocol, remove the hash sign (“#”) from the beginning of the #net my.server.com:/export/tmp line, and replace the value with a valid hostname and directory path.
For example:

net my.server.com:/export/cores

4. Configuring Core Collector
To reduce the size of the vmcore dump file, kdump allows you to specify an external application to compress the data, and optionally leave out all irrelevant information. Currently,the only fully supported core collector is makedumpfile.
To enable the core collector, modify configuration file /etc/kdump.conf , remove the hash sign (“#”) from the beginning of the #core_collector makedumpfile -c –message-level 1 -d 31 line, and edit the command line options as described below.
For example:

core_collector makedumpfile -c

5. Changing Default Action
By default, when the kernel crash is captured, the root file system is mounted, and /sbin/init is run. To change this behavior, open the /etc/kdump.conf configuration file, remove the hash sign (“#”) from the beginning of the #default shell line, and replace the value with a desired action as described .

For example:

default halt

6. Start kdump daemon
Check and make sure kernel command line includes the kdump config and memory was reserved for crash kernel:

# cat /proc/cmdline
ro root=/dev/mapper/vg_ol6desktop-lv_root crashkernel=128M

Set kdump service can be started when system rebooted.

# chkconfig kdump on

To start the service in the current session, use the following command:

# service kdump start
No kdump initial ramdisk found.                            [WARNING]
Rebuilding /boot/initrd-2.6.39-200.24.1.el6uek.x86_64kdump.img
Starting kdump:                                            [  OK  ]

7. Testing kdump
To test the configuration, reboot the system with kdump enabled, and make sure that the service is running.
For example:

# service kdump status
Kdump is operational

Then type the following commands at a shell prompt:

# echo 1 > /proc/sys/kernel/sysrq
# echo c > /proc/sysrq-trigger

This will force the Linux kernel to crash, and the address-YYYY-MM-DD-HH:MM:SS/vmcore file will be copied to the location you have selected in the configuration (that is, to /var/crash/ by default)