Monday, November 10, 2014

Booting AIX into Maintenance Mode

There is a variety of media that can be used to boot an AIX system into Maintenance Mode. These consist of:
  1. A non-autoinstall mksysb taken from the same system or another system running the same level of AIX, either on tape or CD/DVD media.
  2. AIX bootable installation media (CD or DVD).
  3. A NIM server with a SPOT configured, and set up to boot this machine for maintenance work.

For certain work it is important to have the exact same level (AIX version, Technology Level, and Service Pack) on the boot media as is installed on disk. In these cases if the system is booted with different levels, the rootvg filesystems and commands may not be available to use.
These instructions are for pSeries systems with PCI architecture.
Assuming the media is in the optical drive, or a NIM spot has been set up we can boot off that media in this manner:
For Systems Managed by an HMC (such as an LPAR or full system partition)
  • Log on to the HMC as a user who has authority to start or stop systems (usually user "hscroot").
  • Go to Systems Management -> Servers -> choose the server containing the LPAR you wish to boot.

Check the box to the left of the LPAR's name, and from the menu to the right of the name select 
Operations -> Activate -> Profile. Make sure the check box is checked to open a console window.
  • If you have a profile that is configured to boot to SMS, choose that.
  • If you have a profile that is configured to boot to normal mode, choose that one, and click the "Advanced" button.
  • Choose "SMS" as the boot mode temporarily
  • Click "OK", then OK again on the previous window to activate the LPAR.
  • After the firmware is started, the console window should show the power up screen and then the SMS menu:


IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 

1 = SMS Menu 5 = Default Boot List 
8 = Open Firmware Prompt 6 = Stored Boot List 

Memory Keyboard Network SCSI Speaker 


PowerPC Firmware 
Version SF240_358 
SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. 
---------------------------------------------------------------------------- 
Main Menu 
1. Select Language 
2. Setup Remote IPL (Initial Program Load) 
3. Change SCSI Settings 
4. Select Console 
5. Select Boot Options 



--------------------------------------------------------------------------- 
Navigation Keys: 

X = eXit System Management Services 
--------------------------------------------------------------------------- 
Type menu item number and press Enter or select Navigation key:5 
  • Select "5" to access the boot options menu

PowerPC Firmware 
Version SF240_358 
SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. 
---------------------------------------------------------------------------- 
Multiboot 
1. Select Install/Boot Device 
2. Configure Boot Device Order 
3. Multiboot Startup <OFF> 






---------------------------------------------------------------------------- 
Navigation keys: 
M = return to Main Menu 
ESC key = return to previous screen X = eXit System Management Services 
---------------------------------------------------------------------------- 
Type menu item number and press Enter or select Navigation key:1 
  • Select 1 to choose your boot device.
PowerPC Firmware 
Version SF240_358 
SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. 
---------------------------------------------------------------------------- 
Select Device Type 
1. Diskette 
2. Tape 
3. CD/DVD 
4. IDE 
5. Hard Drive 
6. Network 
7. List all Devices 
  • If you know what your boot device is, you can drill down to it from these device types. For a NIM boot choose 6 "Network".
Example: 

PowerPC Firmware 
Version SF240_358 
SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. 
--------------------------------------------------------------------------- 
Select Device 
Device Current Device 
Number Position Name 
1. - Virtual Ethernet 
( loc=U9133.55A.105C2EH-V7-C2-T1 ) 
  • Next select the task to perform. Either Normal Mode or Service Mode is appropriate, both will work.

Select Task 

Virtual Ethernet 
( loc=U9133.55A.105C2EH-V7-C2-T1 ) 

1. Information 
2. Normal Mode Boot 
3. Service Mode Boot 

------------------------------------------------------------------------------- 
Are you sure you want to exit System Management Services? 
1. Yes 
2. No 
  • Choose 1 to boot from the device
  • Next you will see this screen:

IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM STARTING SOFTWARE IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM PLEASE WAIT... IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM 

and if booting from NIM some bootp and packet information as it loads the kernel via tftpboot. 

  • After the kernel boots it will ask you to define the console:

Elapsed time since release of system processors: 113533 mins 36 secs 

---------------------------------------------------------------------------- 
Welcome to AIX. 
boot image timestamp: 17:30 04/22 
The current time and date: 17:39:02 04/22/2013 
processor count: 4; memory size: 1024MB; kernel size: 28308139 
boot device: /vdevice/l-lan@30000002:9.70.90.113,,9.70.90.117,9.70.90.113,00,00 
---------------------------------------------------------------------------- 


******* Please define the System Console. ******* 


Type a 1 and press Enter to use this terminal as the 
system console. 
  • And next to choose your language

>>> 1 Type 1 and press Enter to have English during maintenance. 
  • At this point you are in the maintenance screen. Choose to access a root volume group in order to get into maintenance mode.


Maintenance 

Type the number of your choice and press Enter. 

1 Access a Root Volume Group 
2 Copy a System Dump to Removable Media 
3 Access Advanced Maintenance Functions 
4 Erase Disks 
5 Configure Network Disks (iSCSI) 
6 Select Storage Adapters 

Warning: 

If you choose to access a root volume group, you will NOT be able to return 
to the Base Operating System Installation menus without rebooting. 


Type the number of your choice and press Enter. 

0 Continue 
    The previous warning means once you have committed to maintenance mode you cannot perform an OS install or upgrade without rebooting from media or NIM server.
  • The next screen will allow you to choose which disk contains the rootvg. If you choose one volume group, the screen after will show the names of the logical volumes in it. If you have chosen the wrong volume group, you may go back and choose another one.

Access a Root Volume Group 

Type the number for a volume group to display the logical volume information 
and press Enter. 

1) Volume Group 00050a850000d6000000013c4fcbb3ba contains these disks: 
hdisk0 10240 vscsi 
2) Volume Group 00cda8df00004c000000010a622ba81c contains these disks: 
hdisk3 10240 vscsi hdisk1 10240 vscsi 
3) Volume Group 00050a850000d60000000116c60aefc3 contains these disks: 
hdisk5 10240 vscsi hdisk2 10240 vscsi 
4) Volume Group 00050a850000d6000000013c203de90d contains these disks: 
hdisk4 10240 vscsi 


Choice: 1 



Volume Group Information 

--------------------------------------------------------------------------- 
Volume Group ID 00050a850000d6000000013c4fcbb3ba includes the following 
logical volumes: 

loglv01 loglv00 fslv02 fslv03 
--------------------------------------------------------------------------- 



Type the number of your choice and press Enter. 

1) Access this Volume Group and start a shell 
2) Access this Volume Group and start a shell before mounting filesystems 

99) Previous Menu 

Choice [99]: 99 


Choosing another volume group: 


Choice: 3 


Volume Group Information 

---------------------------------------------------------------------------- 
Volume Group ID 00050a850000d60000000116c60aefc3 includes the following 
logical volumes: 

hd5 hd6 hd8 hd4 hd2 hd9var 
hd3 hd1 hd10opt hd11admin dumplv fslv00 
fslv01 
---------------------------------------------------------------------------- 



Type the number of your choice and press Enter. 

Maintenance Mode Options 

At this point a decision must be made. 

Option 1 will attempt to mount the rootvg filesystems and load the ODM from /etc/objrepos. If this works you will have full access to the rootvg filesystems and ODM, so you may run commands such as bosboot, rmlvcopy, syncvg, etc. If the version of AIX you have booted from (either from media or NIM SPOT) is not exactly the same as on disk, this will error and fail to mount the filesystems. 

Option 2 will import the rootvg and start an interactive shell before mounting any filesystems. This interactive shell has very few commands available to it. As it has not mounted any filesystems from the rootvg it does not have access to rootvg files or the ODM. Use this option when performing maintenance on the rootvg filesystems themselves, such as fsck, rmlv, or logform. 

1) Access this Volume Group and start a shell 
2) Access this Volume Group and start a shell before mounting filesystems 

99) Previous Menu 

Choice [99]: 2 

Importing Volume Group... 
rootvg 

Checking the / filesystem. 

The current volume is: /dev/hd4 
Primary superblock is valid. 
J2_LOGREDO:log redo processing for /dev/hd4 
Primary superblock is valid. 

Checking the /usr filesystem. 

The current volume is: /dev/hd2 
Primary superblock is valid. 

Exit from this shell to continue the process of accessing the root 
volume group. 

# df 
Filesystem 512-blocks Free %Used Iused %Iused Mounted on 
/dev/ram0 262144 214704 19% 481 1% / 
labsystem.ibm.com:/export/nim/spot/6100-06-07/6100-06-07-spot/usr 28049408 17122488 39% 141285 6% /SPOT/usr 
/proc - - - - - /proc 



Notes on Maintenance Mode 
  1. The terminal type is not usually set up correctly for using the vi editor (in Option 2 only). To set this type:

    # export TERM=xterm
  2. If you mount any rootvg filesystems (either automatically under Option 1 or by hand under Option 2 ) and change any files you must manually sync the data from filesystem buffer cache to disk. Normally the syncd daemon does this for you every 30 seconds, but no daemons are running in maintenance mode. To sync the data type:

    # sync; sync; sync
  3. Typically there is no network connectivity in maintenance mode, so FTP or telnet are not available.
  4. If you are in Option 2 with no filesystems mounted and wish to mount the filesystems and load the ODM you can type:

    # exit


Leaving Maintenance Mode 

If you have chosen Option 2 and you have not mounted any filesystems by hand, just shut down the LPAR (via the HMC) or power off a standalone server. If you are ready to boot AIX to multiuser then activate the LPAR or if a standalone server power it on via the front panel. 

If you have chosen Option 1 type these commands to reboot the system: 

# sync; sync; sync; reboot