Installing Zones on Nexenta

From Docupedia

Revision as of 05:03, 14 February 2008

A ZFS file system has been created for this zone.
    __                    _          _____                      
  /\ \ \_____  _____ _ __ | |_ __ _  / _  / ___  _ __   ___  ___ 
 /  \/ / _ \ \/ / _ \ '_ \| __/ _` | \// / / _ \| '_ \ / _ \/ __|
/ /\  /  __/>  <  __/ | | | || (_| |  / //\ (_) | | | |  __/\__ \
\_\ \/ \___/_/\_\___|_| |_|\__\__,_| /____/\___/|_| |_|\___||___/

Introduction

This howto is designed to take you through installation of a zone into your new shinny nexenta server. The use for zones is vast. I tend to explain zones to people as light weight virtual machines. All the zones on the box share the same kernel. This helps keep overhead down and administration ease up. The basic process we are going to follow is: creating the zone, installing it, then doing some post configuration clean up to remove some errors. This howto also assumes that you have gotten yourself a working sudo install. If you haven't set that up just su - to root and give all the commands without the sudo preface.

Creating a Zone

Lets start by making the directory. Please notice that the permissions are required to be set at 700 on the directory. If you don't do this you will hear about it later during the zone install.

 jsmith@sol01:~$ sudo mkdir -m 700 /data/zones/zone01

This creates a folder for the zone. This assumes that you have a folder named data. In this case both data and data/zones exist in a zfs pool.

jsmith@sol01:~$ zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
data               993M   877G  38.2K  /data
data/zones         993M   877G  39.6K  /data/zones
data/zones/zone01  534M   877G   534M  /data/zones/zone01

Please note the sizes here are from an already created and installed zone. So yours won't show that big of a size. This is really just included for you to realize what the underlying file system structure is. Now that we have all this done we can create the zone.

jsmith@sol01:~$ sudo zonecfg -z zone1
zone1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:zone1> create -b
zonecfg:zone1> set zonepath=/data/zones/zone01
zonecfg:zone1> set autoboot=true
zonecfg:zone1> add net
zonecfg:zone1:net> set address=192.168.1.200
zonecfg:zone1:net> set physical=e1000g0
zonecfg:zone1:net> end
zonecfg:zone1> add attr
zonecfg:zone1:attr> set name=comment
zonecfg:zone1:attr> set type=string
zonecfg:zone1:attr> set value="zone1.domain.com"
zonecfg:zone1:attr> end
zonecfg:zone1> verify
zonecfg:zone1> commit
zonecfg:zone1> exit

Thats it for configuring the zone. I do at this point like to open the created xml file manually and just verify that all the contents are sane.

jsmith@sol01:~$ sudo vi /etc/zones/zone1.xml

Once you have opened the file the contents should be something like this:

<?xml version="1.0" encoding="UTF-8"?> 
<!DOCTYPE zone PUBLIC "-//Sun Microsystems Inc//DTD Zones//EN"
"file:///usr/share/lib/xml/dtd/zonecfg.dtd.1">
<zone name="zone01" zonepath="/data/zones/zone01" autoboot="true">
 <inherited-pkg-dir directory="/lib"/>
 <inherited-pkg-dir directory="/platform"/>
 <inherited-pkg-dir directory="/sbin"/>
 <inherited-pkg-dir directory="/usr"/>
 <network address="192.168.1.200" physical="e1000g0"/>
 <attr name="comment" type="string" value="zone01.alkaloid.net"/>
</zone>

I prefer to remove the inherited-pkg-dir lines. These only need to be here if you don't want your zones to be able to install different packages then the global zone. I usually leave the platform folder as being inherited. If you have these set when you try and do a sudo apt-get install you will get a read-only permission denied error. This is because you are inheriting those directories from the global zone. So make it look something like this:

 <?xml version="1.0" encoding="UTF-8"?> 
<!DOCTYPE zone PUBLIC "-//Sun Microsystems Inc//DTD Zones//EN"
"file:///usr/share/lib/xml/dtd/zonecfg.dtd.1">
<zone name="zone01" zonepath="/data/zones/zone01" autoboot="true">
 <inherited-pkg-dir directory="/platform"/>
 <network address="192.168.1.200" physical="e1000g0"/>
 <attr name="comment" type="string" value="zone01.alkaloid.net"/>
</zone>

Installing the domain

This is basically the part where the new zone installs its operating system. Pretty easy just give the command

jsmith@sol01:~$ sudo zoneadm -z zone1 install

After running this command be prepared to wait a while. A base zone install is around 500mbs that needs to be downloaded. Once this is done you will be ready to boot your new zone.

In newer Nexenta builds, there is an error that causes the new zone to built in the root directory, named by the UUID of the zone. For example, you might see the following:

# ls /
4ac63c20-29a3-699a-e43e-c6e5a0b610e5  etc         media
bin                                   export      proc
...

Inside this directory, you will see the `root' directory that belongs just inside of the zonepath. Move it into the zonepath before you boot the zone, and then you can remove the UUID-named directory.

Booting your new zone

Now that your install has completed you are ready to boot your zone. The initial boot takes a little while so be patient. Your computer is now running multiple little computers now so it can take a little more time.

jsmith@sol01:~$ sudo zoneadm -z zone1 boot

You can connect to the console to see how the boot process is going by doing

jsmith@sol01:~$ sudo zlogin -C zone01

You will be prompted for the root password which at this point is blank so just press enter. Now that your on the console you'll want to type passwd and set a root password.

Your first connection to the console after a system has installed could take a while. You should notice in the bottom right corner there is a counter starting up towards 80. Something like

[Connected to zone 'zone01' console]                                                                                                            
                                                                           39/80 

When this number gets to 80 the zone will be booted.

Fixing all the errors

If your like me your anal about boot errors and when you first login to the console you will see a lot of them. Here is a list some of the alkaloid members came up with for getting ride of some of those errors. These must all be run from a root prompt or using sudo.

jsmith@sol01~$ sudo svcadm disable eeprom
jsmith@sol01~$ sudo svcadm disable dumpadm
jsmith@sol01~$ sudo svcadm disable fmd
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/nlockmgr:default
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/client:default
jsmith@sol01~$ sudo svcadm disable svc:/system/filesystem/autofs:default
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/server:default
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/status:default
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/cbd:default
jsmith@sol01~$ sudo svcadm disable svc:/network/nfs/mapid:default

Then reboot your zone

jsmith@sol01~$ sudo reboot

After your reboot also disable these services

jsmith@sol01~$ sudo svcadm disable svc:/network/rpc/bind:default
jsmith@sol01~$ sudo svcadm disable svc:/network/rpc-100235_1/rpc_ticotsord:default
jsmith@sol01~$ sudo svcadm disable power

One more reboot and you should be left with only one set of errors. I haven't figured out this last set of errors. Only two of the three of us that were building systems say this last set of errors. We are not sure if that is because we are less leet then the guy who didn't see it or what?? For now you'll just have to live with it. It doesn't seem to effect system stability. The error I speak of might look something like this after a successful boot.

Mar 24 07:13:50 zone01 syseventd[9218]: unable to open channel to syseventconfd
Mar 24 07:13:50 zone01 syseventd[9218]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:50 zone01 syseventd[9218]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:50 zone01 syseventd[9218]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:50 zone01 syseventd[9237]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:50 zone01 syseventd[9237]: unable to open channel to syseventconfd
Mar 24 07:13:50 zone01 syseventd[9237]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:50 zone01 syseventd[9237]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:50 zone01 syseventd[9237]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:50 zone01 syseventd[9247]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:50 zone01 syseventd[9247]: unable to open channel to syseventconfd
Mar 24 07:13:50 zone01 syseventd[9247]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:50 zone01 syseventd[9247]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:50 zone01 syseventd[9247]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:50 zone01 syseventd[9257]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:50 zone01 syseventd[9257]: unable to open channel to syseventconfd
Mar 24 07:13:50 zone01 syseventd[9257]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:50 zone01 syseventd[9257]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:50 zone01 syseventd[9257]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:51 zone01 syseventd[9267]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:51 zone01 syseventd[9267]: unable to open channel to syseventconfd
Mar 24 07:13:51 zone01 syseventd[9267]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:51 zone01 syseventd[9267]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:51 zone01 syseventd[9267]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:51 zone01 syseventd[9277]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:51 zone01 syseventd[9277]: unable to open channel to syseventconfd
Mar 24 07:13:51 zone01 syseventd[9277]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:51 zone01 syseventd[9277]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:51 zone01 syseventd[9277]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:51 zone01 syseventd[9287]: Invalid ops vector for module 'devfsadmd_mod.so'
Mar 24 07:13:51 zone01 syseventd[9287]: unable to open channel to syseventconfd
Mar 24 07:13:51 zone01 syseventd[9287]: Invalid ops vector for module 'sysevent_conf_mod.so'
Mar 24 07:13:51 zone01 syseventd[9287]: sysevent_reg_mod: Can not open subscriber channel: Bad address
Mar 24 07:13:51 zone01 syseventd[9287]: Unable to establish door name with kernel: 'Not owner'
Mar 24 07:13:51 zone01 svc.startd[9071]: system/sysevent:default failed repeatedly: 
transitioned to  maintenance (see 'svcs -xv' for details)

This appears very bad but doesn't seem to cause any problems. The smart ones in the group are going to try and disable the system/sysevent:default. This will then casue a whole bunch of other errors. I will post a fix once one of these other more smarterist alkaloid fellows figures it out. So just hit enter and you should be rewarded with the login screen for your new zone.

Detaching from a Zone

Now that you've used zlogin to get a console on the zone. Your probably wondering how to get outta the console. This one took me a second or two to figure out. The key command you will need to exit is as follows:

~.

(That is a tilda then a period.)

This sequence assumes you are directly on the console of the system hosting the Zone. OpenSSH uses the same escape sequence for its connections, and typing this will disconnect your "outermost" OpenSSH connection. To get your OpenSSH client to send a `~.' to the remote end, you have to type `~~.' into your client. If you have two OpenSSH connections open to connect to your Zone, then you would type three tilda's to send the escape. You get the idea.

A good rule of thumb is keep pressing ~ until one appears on the shell. Your number of `escapes' will be equal to how many ~'s didn't appear. Type that many plus a period and you'll disconnect from the Zone.

Showing off your new Zones

Ok so now you have made you new zone. You probably are going to want to login in to IRC right away and show all your ub3r g33k buddies what you've just done. So I should probably give you some commands to checkout your zones.

jsmith@sol01:/data$ zoneadm list -vc
 ID NAME             STATUS         PATH                           BRAND     
  0 global           running        /                              native    
  1 zone01           running        /data/zones/zone01             native    

That should list your zones for you. Allow you to copy and paste into IRC and show off a little.

Cloning your new zone

This section will take you through cloning your newly built zone. Cloning isn't the only option but its a good one in some situations. Before you clone a new zone you must use zonecfg to create the target zone. This must be done before running the clone command on the base zone your cloning. By setting the environment variable SUN_PERSONALITY before calling the clone command you make sure that the sun utilities are used rather then the gnu ones. As an example with SUN_PERSONALITY=1 set a call to cpio would get suns version of cpio instead of the gnu version.

jsmith@sol01:/data$ sudo su 
root@sol01:/$ export SUN_PERSONALITY=1
root@sol01:/$ zoneadm -z target clone -m copy base

After running the clone you may encounter this error

A ZFS file system has been created for this zone.
Copying /data/zones/base...
/sbin/sh: /usr/sbin/sys-unconfig: not found
sys-unconfig failed
zoneadm: zone 'target': post-clone configuration failed.

Sys-unconfig is another sun utility that isn't currently shipped in nexenta. A work around found by Acidtrux is to simply edit the master xml file.

jsmith@sol01:/data$ sudo vi /etc/zones/index

There will be an entry in there that looks like this

target:incomplete:/data/zones/target:42f3bc18-24b5-6834-gh55-c43d2b6a86ge

Update this line to read

target:installed:/data/zones/target:42f3bc18-24b5-6834-gh55-c43d2b6a86ge

Then simply boot the zone. Then we will have to manually do all the stuff that sys-unconfig would have done for us.

jsmith@sol01:/data$ sudo zoneadm -z target boot && sudo zlogin -C -e ] target

This will boot the zone and get you right onto the console with the ability to see all the kernel messages. HAHA I bet at this point your at the zones console but you can't get out. Don't worry I was in the same spot. Short of closing the terminal your working in you could always try the keystrokes that follow: ]. that should drop you back into your shell on the global zone.

Moving and Renaming a zone

Ok so now you've installed your new zone. Then you realized you made a mistake in the name. Now you want to go back and rename and move your zone. Well thats no problem. Follow along with me.

First thing to remember is that in order to modify stuff in a zones configuration it needs to be halted.

jsmith@sol01:/data$ sudo zoneadm -z oldzone halt

Now we have to use zonecfg to update the parameters for the zone.

jsmith@sol01:/data$ sudo zonecfg -z oldzone
zonecfg:oldzone> info

This will let us see all the information that we have to update:

zonename: oldzone
zonepath: /data/zones/oldzone
brand: native
autoboot: true
bootargs: 
pool: 
limitpriv: 
inherit-pkg-dir:
        dir: /platform
net:
        address: 192.168.1.100
        physical: e1000g0
attr:
        name: comment
        type: string
        value: oldzone.alkaloid.net

We want this info to all be updated to the new name of the zone. We use the set command for this:

zonecfg:oldname> set zonename=newname
zonecfg:newname> set zonepath=/data/zones/newname
zonecfg:newname> remove attr name=comment
zonecfg:newname> add attr
zonecfg:newname:attr> set name=comment
zonecfg:newname:attr> set type=string
zonecfg:newname:attr> set value="newname.alkaloid.net"
zonecfg:newname:attr> end
zonecfg:newname:> verify
zonecfg:newname:> commit
zonecfg:newname:> exit

Ok thats it the new zone has its info all updated now we just have to move the zfs filessystem from its old zonepath to the new zonepath.

jsmith@sol01:/data$ sudo zfs rename data/zones/oldzone data/zones/newzone

Ok now you should be ready to boot up your zone with the new name:

jsmith@sol01:/data$ sudo zoneadm -z newzone boot