In large organizations, mostly the servers would be managed remotely through the console (ilo/drac/imm). The large company's also change the root password on all servers periodically which mostly happen through scripts. If, due to any reason, the root password is not updated on server and if the server crash due to any file system related issues, the system admin has to spent lot of time to recover the server because the system admin cannot even get into single user mode due to the fact that file system has issues and the root password is not known. In these situations, the system admin has to boot the system with the Linux Boot CD and get into rescue mode to recover server/root password. The large organizations also mostly would not have a CD at the datacenter and the system admins are expected to use boot.iso (which comes with the linux distribution) to get into rescue mode. Since, the iso image is connected from user desktop, mostly the server (high end) would take huge time to boot up and get into rescue mode.
However, there is an easy method to quickly recover the server in these situations. Linux allows system admins to provide kernel parameters at boot time (grub screen). rc.sysinit script is responsible for checking the file system(s) and mounting them in the read/write mode. If the system admin can skip the init process and get a root prompt, the kernel will not check for the file system and drop you to the # prompt. The following steps needs to be followed to skip init from loading into the memory in the booting process.
1. Reboot the server.
2. At the Grub Prompt, press “a” to provide additional kernel parameters.
3. Add “init=/bin/bash” at the end and press “Enter”. Basically, we are just telling the kernel to load /bin/bash in place of init to skip init from loading into the memory. The kernel would load /bin/bash in place of init and would drop you to the # Prompt. But, since rc.sysinit script is not yet executed, the root file system is mounted in read-only mode. Hence you need to mount the root file system in read/write mode before making any changes to the system configuration.
4. mount the root file system in read/write mode using the following command :
bash-3.1# mount -o remount,rw /
5.Now you can do everything, which you can do from single user mode. Update any config file, scan file systems for errors or even enable/disable services at boot time. Once you are done, type exit and hit enter to reboot the server. You would get an error message, “Kernel panic – not syncing : Attempted to kill init !”. Ignore the error and hard reboot the server.
Cheers,
Vaibhav
Rebooting to "init=/bin/bash" is not really an improvement over rebooting into single user mode . It's still rescue mode . Essentially, you've done the same thing . To be impressive , you would need to recover Linux without rebooting in the first place .
ReplyDelete