Solaris 10 Jumpstart from FreeBSD

My little SunFire v120 has been asking me to run the latest Solaris 10, and do some testing :), so I set up Sol10 on my Linux jumpstart server, which was happily running my current Sol9 jumpstart. Needless to say, I ran into a multitude of problems, and found many other folks that have had similar issues… The end result of hacking and troubleshooting? The Linux implementation of NFS is simply broken, and needs to be fixed at the kernel level if Linux is to be used for a Solaris 10 jumpstart.

The Solaris 10 NFSv4 client cannot properly mount it’s root filesystem on the Linux NFS server, and the jumpstart fails. Connections to the same Linux NFS server from Sol10 are successful if one uses mount options of either ‘-o vers=3’ or ‘-o proto=udp’ – unfortunately, passing these options for jumpstart in /etc/bootparams with ‘rootopts=:’ seem to be ignored by the Solaris 10 installer.

Several folks reported that they have been successful with implementing jumpstart with the current versions of NetBSD and FreeBSD, so I made backups of my Sol9 flash archives and data, and installed FreeBSD 5.4 on the Compaq 5000R that I was using for jumpstart, as well as serve NFS for my LAN. Following a basic guideline I found, I ran into couple of additional issues, so here is my how-to on Solaris 10 Jumpstart from FreeBSD:

I did a fresh install of FreeBSD 5.4, updated the full source and ports tree via cvsup, and went through a buildworld/buildkernel/installkernel/installworld, as per /usr/src/UPDATING, so that my new install was completely patched and I had a stripped down, SMP-aware kernel for my beloved old quad-processor 7U.

As Jason Heiss’ guideline indicates, here are the bits of software we need to configure:

* RARP
* TFTP
* bootparams
* ICMP address mask reply
* NFS

All of the above software is installed by default in my “Developer” installation of 5.4-RELEASE, so I only added a few packages from the ports collection for keeping current and for my comfort – cvsup-without-gui, lsof, rsync, and bash.

First thing was to grab the DVD iso, mount it, and copy all the files to the filesystem. I don’t have a CD/DVDROM in the Sun, only a CDROM in the 5000R, and dealing with one DVD image is better than several CDs… After all, this is why I am setting up jumpstart. Go to sun.com and follow the download instructions – I will not cover them here.

root@daemon# mdconfig -a -t vnode -f /home/michael/sol10/sol-10-GA-sparc-dvd-iso.iso -u 0
root@daemon# mount -t cd9660 /dev/md0 /mnt/
root@daemon# mkdir -p /usr/exports/sol10/
root@daemon# rsync -av /mnt/ /usr/exports/sol10/
root@daemon# umount /mnt/
root@daemon# mdconfig -d -u 0

* RARPD

Rarpd answers arp broadcasts, based on the MAC address, hostname, and IP mapping in /etc/ethers and /etc/hosts, and hands off network booting to TFTP. Here is the contents of my /etc/ethers file – the mac is gathered from the serial console on the Sun box and given a hostname:

# added for jumpstart
0:3:ba:16:85:6d sol

And here is my /etc/hosts (yeah, I like the hostname first – keeps netstat output clean):

127.0.0.1 localhost localhost.pbandjelly.org
192.168.1.100 daemon daemon.pbandjelly.org

# added for jumpstart
192.168.1.105 sol sol.pbandjelly.org

* TFTP

Rarpd sends a reply to the client with “here is your hostname and your IP address”, and the client attempts to access a network bootstrap kernel, via trivial file transfer protocol. This request made to the tftp server is for a file with the name matching the hex address derived from the IP address, which in my case is C0A80169. I calculated the hex address, copied the NFS inetboot file for my architecture to /tftpboot, and symlinked the hex address that will be requested:

root@daemon# perl -e 'printf "%02x"x4 ."\n",192,168,1,105;'|tr a-z A-Z
C0A80169
root@daemon# cp -p /usr/exports/sol10/Solaris_10/Tools/Boot/usr/platform/sun4u/lib/fs/nfs/inetboot /tftpboot/inetboot_sun4u_sol10
root@daemon# cd /tftpboot/
root@daemon# ln -s inetboot_sun4u_sol10 C0A80169
root@daemon# ls -l
total 240
lrwxr-xr-x 1 root wheel 20 Jul 25 12:43 C0A80169 -> inetboot_sun4u_sol10
-rw-r--r-- 1 root sys 214880 Jan 22 2005 inetboot_sun4u_sol10

* /etc/bootparams

Bootparams is called by the inetboot bootstrap kernel and requests “ok, what do I do now?” So, we tell the client, here is your root filesystem and where to install from. The magnificently verbose Sun docs cover jumpstart from head to toe, so feel free to add on as you see fit – this is a minimal /etc/bootparams configuration to initiate an interactive network installation (all one line):

sol root=192.168.1.100:/usr/exports/sol10/Solaris_10/Tools/Boot install=192.168.1.100:/usr/exports/sol10 boottype=:in rootopts=:rsize=4096

* ICMP address mask

Jason Heiss said to set this up, and to be honest, I have not researched if it is needed, but I believe this corrects the need to ping the jumpstart client, just as it is trying to set up networking, as I did from the Linux jumpstart server:

root@daemon# echo "net.inet.icmp.maskrepl=1" >> /etc/sysctl.conf

* NFS

This is where I ran into a little trouble – attempts to use -alldirs caused errors, so I specified each directory explicitly, and all worked out fine – my /etc/exports (all one line):

/usr/exports/sol10 /usr/exports/sol10/Solaris_10/Tools/Boot -maproot=0 -network 192.168.1.0 -mask 255.255.255.0

* /etc/inetd.conf and /etc/rc.conf

OK – let’s get all this stuff enabled! With the editor of your choice, uncomment the tftp line in /etc/inetd.conf:

tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /tftpboot

I then added the following to /etc/rc.conf:

# added for jumpstart
rpcbind_enable="YES"
nfs_server_enable="YES"
inetd_enable="YES"
rarpd_enable="YES"
rarpd_flags="-asv"
bootparamd_enable="YES"
bootparamd_flags="-s -r 0.0.0.0"

The bootparamd_flags addition of ‘-r 0.0.0.0’ was necessary due to the gateway failing to be configured properly on the client – got a crazy kernel panic and core dump from the installer. Once I searched for several of those errors, I put it together and found that the gateway is set as 127.0.0.1 by Solaris, for some reason, and networking chokes on itself – a setting of the actual gateway router works, but 0.0.0.0 was suggested in a mailing list post I found, and the booting client basically asks the bootparam server, “what is your gateway, and I will use that one” – worked for me!

Reboot the server or enable all the services by hand – happy jumpstarting!

8 thoughts on “Solaris 10 Jumpstart from FreeBSD”

  1. How did you get around the multi partitioned CD mount? I cant copy inetboot because Boot is link to ../../s1 and there is no s1.

  2. I used the DVD iso image and there is no issue with the partioning:

    mdconfig -a -t vnode -f /home/michael/sol10/sol-10-GA-sparc-dvd-iso.iso -u 0
    mount -t cd9660 /dev/md0 /mnt/
    rsync -av /mnt/ /usr/exports/sol10/

    —–

    I also wrote a lengthy reply on the Sun Forums for somone with the same question on mounting Sun CD partitions on a Linux box – the reply was monitored and removed… whatever…

    Here are the notes from my Sol9 Jumpstart setup on Linux, which I emailed to another admin:

    —–

    This is an example with CD-1 of Solaris 9 U7.

    Problem: We need the Solaris installation media for Jumpstart, and mounting a Solaris install CD on Linux only gets us one slice – so, where is the root filesystem for the install? Well, we don’t have a Sun box with a CDROM either…

    root@gaea 06:25:15 ~ # mount /mnt/cdrom
    root@gaea 06:27:07 ~ # ls -l /mnt/cdrom
    total 17
    -r–r–r– 2 root root 85 2004-06-17 18:27 .cdtoc
    lrwxrwxrwx 1 root bin 32 2004-06-29 14:11 .install_config -> ./Solaris_9/Misc/.install_config/
    -r–r–r– 2 root root 462 2004-06-17 18:27 .slicemapfile
    -r–r–r– 2 root root 20 2004-06-17 18:27 .volume.inf
    -r–r–r– 2 root root 7052 2004-06-17 18:48 Copyright
    drwxr-xr-x 7 root root 2048 2004-06-29 14:26 Solaris_9/

    The mount only pulls in the first slice ‘usr’ (s0) – there is reference to the second slice ‘root’ (s1) in the installer, so this determines how we set up the files locally:

    root@gaea 06:33:13 ~ # ls -l /mnt/cdrom/Solaris_9/Tools/
    total 125
    -rw-r–r– 2 root daemon 14267 2004-05-10 19:17 .wanboot_unneeded_packages
    lrwxrwxrwx 1 root bin 11 2004-06-29 14:11 Boot -> ../../../s1/
    -rwxr-xr-x 2 root daemon 63732 2004-05-10 19:17 add_install_client*
    -rwxr-xr-x 2 root sys 1325 2004-05-10 19:17 dial*
    -rwxr-xr-x 2 root daemon 18523 2004-05-10 19:17 rm_install_client*
    -rwxr-xr-x 2 root daemon 24333 2004-05-10 19:17 setup_install_server*

    Let’s grab all the stuff off the first slice into our jumpstart dir at s0:

    root@gaea 06:33:13 ~ # rsync -av /mnt/cdrom/ /jumpstart/sol9/s0/

    —–

    Now, how the hell do we get to the second slice on a Linux box, in the same way that Solaris mounting this CD will give us all the partitions mounted…

    root@gaea 06:25:03 ~ # fdisk -l /dev/sr0
    Note: sector size is 2048 (not 512)

    Disk /dev/sr0 (Sun disk label): 1 heads, 640 sectors, 2048 cylinders
    Units = cylinders of 640 * 512 bytes

    Device Flag Start End Blocks Id System
    /dev/sr0p1 r 0 1018 325760 4 SunOS usr
    /dev/sr0p2 r 1018 2079 339520 2 SunOS root
    /dev/sr0p3 2079 2087 2560 0 Empty
    /dev/sr0p4 2087 2095 2560 0 Empty
    /dev/sr0p5 2095 2103 2560 0 Empty
    /dev/sr0p6 2103 2111 2560 0 Empty

    —–

    Determine the offset of the second slice from the starting block:

    /dev/sr0p2 r 1018 2079 339520 2 SunOS root

    OK – calculate the offset:

    root@gaea 06:39:54 ~ # echo 1018*640*512|bc
    333578240

    Umount the CD, then mount the second slice:

    root@gaea 06:49:02 ~ # mount -t ufs -o ufstype=sun,offset=333578240 /dev/sr0 /mnt/cdrom
    root@gaea 06:49:18 ~ # ls -l /mnt/cdrom/
    total 44
    drwxr-xr-x 3 root sys 512 2004-06-29 15:13 .tmp_proto/
    drwxr-xr-x 2 root sys 512 2004-06-29 14:30 a/
    lrwxrwxrwx 1 root bin 9 2004-06-29 15:12 bin -> ./usr/bin/
    drwxr-xr-x 2 root sys 512 2004-06-29 14:30 cdrom/
    drwxr-xr-x 10 root sys 1024 2004-06-29 15:12 dev/
    drwxr-xr-x 3 root sys 512 2004-06-29 15:12 devices/
    drwxr-xr-x 32 root sys 3072 2004-06-29 15:12 etc/
    drwxr-xr-x 10 root sys 512 2004-06-29 15:12 kernel/
    lrwxrwxrwx 1 root bin 9 2004-06-29 15:12 lib -> ./usr/lib/
    drwxr-xr-x 2 root sys 512 2004-06-29 14:26 mnt/
    drwxr-xr-x 2 root sys 512 2004-06-29 14:26 opt/
    drwxr-xr-x 51 root sys 2048 2004-06-29 15:12 platform/
    dr-xr-xr-x 2 root root 512 2004-06-29 14:26 proc/
    -rw-r–r– 1 root bin 0 2004-06-29 14:28 reconfigure
    drwxr-xr-x 3 root sys 1024 2004-06-29 15:12 sbin/
    drwxrwxrwt 4 root sys 512 2004-06-29 15:12 tmp/
    drwxr-xr-x 23 root sys 512 2004-06-29 15:11 usr/
    lrwxrwxrwx 1 root bin 14 2004-06-29 15:07 var -> ./tmp/root/var/
    drwxr-xr-x 3 root bin 512 2004-06-29 15:12 webstart/

    Oohh, la la… it’s magic 😉

    Then copy off all data on this slice to s1 in our jumpstart tree, and the references back to this directory will work out fine:

    root@gaea 06:49:30 ~ # rsync -av /mnt/cdrom/ /jumpstart/sol9/s1/

    —–

    Now that you have all the data from the installation CDs you can add stuff to it like additional packages, tar it, and copy it up to a remote Jumpstart, etc.

    Basic outline that I followed for setting up JS on Linux:
    http://www.docbert.org/Sola…

    Can’t remember where I found the info on mounting with offsets…

    Kind Regards,
    Michael

  3. Hi, the offset option …. mount -t ufs -o ufstype=sun,offset=333578240 /dev/sr0 can not be used on my FreeBSD 5.4 installation. Any suggestions Thanks!

  4. Yeah, ‘man mount’ – and understand that the above comments are for a specific CD on a Linux box. The offset, devices, mount options, etc. will all be different on your BSD box. You need to read between the lines – this is just an example…

    For more help, please read the FreeBSD Handbook, or post to a FreeBSD newsgroup, mailing list, or forum…

    Additionally, if you read the initial post, save yourself some time and download the DVD iso – there is no need to fiddle with offset using the DVD.

  5. Hi, Im unable to run jumpstart for solaris 10 from RHEL 3 linux box. it stops at "rpc.bootparamd getfile got question for file "root""
    "rpc.bootparamd get file failed"

    Can anyone help?

  6. Well, first thing first… if you wish to try with RHEL-3, and that is your ONLY option, then jumping Sol-10 will require that you build an experimental NFSv4 kernel and client/server package (since the Linux NFS implementation is borked) and keep hacking on the RHEL-3 server to try to get it working…

    http://www.citi.umich.edu/p…
    http://linux-nfs.org/
    http://developer.osdl.org/d…
    http://nfsv4.bullopensource…

    Or use FreeBSD! – that is exactly the point of this article 😉

    As far a troubleshooting your current bootparamd problem, you must build NFSv4 first – even if all your configs are correct, the install will fail…

    The notes about my Sol-9 install in the comment above, worked perfectly, so if you get NFSv4 running correctly on RHEL-3, there is little reason that a Sol-10 jump from your server wouldn’t work. Keep digging!

    Warm Regards,
    Michael

  8. Hey just wanted to say this guide also worked great with OpenBSD instead of FreeBSD. A few of the daemons I started by hand to avoid rebooting. One small change is that the inetboot file is located in “Solaris_10/Tools/Boot/platform/sun4u/” on the recent version of Solaris 10.

Comments are closed.