PXE Booting SmartOS from SmartOS zone
Motivation
We’ve bought a new Supermicro server – chassis and four blades. The provider installed Ubuntu on one of them, and from this I have already set up SmartOS on three other blades. As you know, host machine running on SmartOS boots from PXE server. But I don’t need a separate blade running on Linux, so to ensure safety I decided that each blade could be used as a loader for the rest of them. It was possible to deploy Linux on each host in KVM, but I found a better solution – to deploy PXE server in native SmartOS zone. Isn’t that wonderful when SmartOS can boot SmartOS?
Here’s how to set up a simple PXE server in a SmartOS zone that will serve up SmartOS
|
1 2 3 |
imgadm update imgadm import 8639203c-d515-11e3-9571-5bf3a74f354f |
create pxe-server.json with following:
Zone Configuration
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
{ “alias”: “pxe-server”, “hostname”: “pxe-server”, “brand”: “joyent”, “max_physical_memory”: 64, “quota”: 2, “image_uuid”: “8639203c-d515-11e3-9571-5bf3a74f354f”, “resolvers”: [ “8.8.8.8”, “8.8.4.4”], “nics”: [ { “nic_tag”: “admin”, “ip”: “192.168.0.2”, “netmask”: “255.255.255.0”, “gateway”: “192.168.0.1”, “dhcp_server”: “1” } ] } vmadm create -f pxe-server.json |
Setting up TFTP
Use zlogin to log into the zone:
|
1 |
zlogin <uuid> |
In the zone:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
pkgin -y install tftp-hpa mkdir /tftpboot echo “tftp dgram udp wait root /opt/local/sbin/in.tftpd in.tftpd -s /tftpboot” > /tmp/tftp.inetd svcadm enable inetd inetconv -i /tmp/tftp.inetd -o /tmp svccfg import /tmp/tftp-udp.xml svcadm restart tftp/udp |
Setting up DHCP (using Dnsmasq)
|
1 |
pkgin -y install dnsmasq |
Edit /opt/local/etc/dnsmasq.conf
|
1 2 3 4 5 6 7 8 9 |
dhcp-range=192.168.0.200,192.168.0.220,2h dhcp-match=set:gpxe,175 dhcp-boot=tag:!gpxe,undionly.kpxe dhcp-boot=smartos.ipxe dhcp-leasefile=/etc/dnsmasq.leases svcadm enable dnsmasq |
Setting up the tftpboot directory
Ben Rockwood provides a version of undionly.kpxe on his site. Run the following to get the PXE chainload binaries in place:
|
1 2 3 |
cd /tftpboot curl http://cuddletech.com/IPXE-100612_undionly.kpxe > undionly.kpxe |
At this point a generic PXE boot server is complete. iPXE will still expect smartos.ipxe, but that can be created with whatever content is needed. For those interested in booting SmartOS, what follows are the steps to provide SmartOS boot services on this server.
Providing SmartOS PXE Boot Services
A template iPXE config is useful both upfront and when updating to new platform releases. Create /tftpboot/smartos.ipxe.tpl with the following content (-B smartos=true is essential, otherwise logins will fail):
|
1 2 3 4 5 6 7 8 9 10 11 |
#!ipxe # /var/lib/tftpboot/smartos.ipxe.tpl kernel /smartos/$release/platform/i86pc/kernel/amd64/unix -B smartos=true initrd /smartos/$release/platform/i86pc/amd64/boot_archive boot cd /tftpboot mkdir smartos |
Deploy/Update to the latest SmartOS platform release
The steps in this section work for both initial deployment and upgrades as Joyent releases them.
Next get the latest SmartOS platform and massage it into a workable shape for our iPXE config:
|
1 2 3 |
cd /tftpboot/smartos curl https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/platform-latest.tgz > /var/tmp/platform-latest.tgz |
(Just now URL https://download.joyent.com/pub/iso/platform-latest.tgz is invalid, 404… )
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
cat /var/tmp/platform-latest.tgz | tar xz directory=`ls | grep platform- | sort | tail -n1` release=${directory:9} mv $directory $release cd $release mkdir platform mv i86pc platform cd /tftpboot cat smartos.ipxe.tpl | sed -e”s/\$release/$release/g” > smartos.ipxe |
Make sure PXE boot is enabled and that it is the first in the boot sequence.
Thanks
Thanks to Alain O’Dea for his notes about his experience in setting up Ubuntu Server 12.04.1 LTS as a PXE server to boot SmartOS and big thanks to Ben Rockwood for creating and maintaining the PXE Booting SmartOS wiki page. Without their instructions I would not have done it.
Enjoy and stay tuned!
Leave a Reply