As I’ve alluded to previously, I’ve been exploring Joyent’s SmartOS lately. I’d like to talk a little bit more about it. It seems to me that a lot of my peers haven’t yet heard about it, or don’t really understand it much at all. The documentation isn’t really there yet, so I think that it is a little hard to get the right first impression if you’re coming into it with no Solaris background.
That’s right, I said Solaris. SmartOS is not Solaris, but it shares a common heritage with Solaris. SmartOS is based on the Illumos kernel, which is based on the OpenSolaris project (which no longer exists). Oracle has since closed the Solaris source, and most of the best engineers have found themselves reuniting under the Illumos banner. Many of them are under the employ of Joyent and Nexenta.
Illumos provides a common kernel and some of the fundamental system tools. Distributions build on top of Illumos and offer a unique branded experience. There are quite a few out there, but I’ll just mention a few that I’ve experimented with and my impressions of where they fit.
- OpenIndiana is going to be much more familiar to those that came from a Solaris or OpenSolaris background. Of the distributions that I’ve tried so far, this is the distro that I would want to run on a desktop or even a laptop.
- Nexenta CP is the community distro provided by Nexenta. If your background is in Linux, especially Debian or Ubuntu, this is going to be a very familiar environment. Quite a large portion of the Debian user experience has apparently been ported over to sit on top of the Illumos kernel. Debian package management is here, too, but with a twist: on Nexenta CP, ZFS filesystem-level snapshots are leveraged when you install packages, giving you an easy way to roll back if you need to. This distribution is undergoing some major changes right now and becoming what is known as “Illumian” (think “Illumos” + “Debian”).
- SmartOS is the community distro provided by Joyent. It seems to mostly be a subset of their commercial SmartDataCenter product, minus the secret sauce that gives you a full-blown multi-datacenter cloud infrastructure. Of the three Illumos distributions, I think this is the one that is the most innovative. You’re not going to use this for your desktop. But you might use this to run your cloud applications.
Tonight, I want to talk to you a little more about SmartOS. It’s been sort of a poorly kept secret infatuation of mine lately, one that is blossoming into a full-blown romance.
Like any good romance story, this one had an unlikely beginning. I’m a system administrator. I’ve been one for about 18 years now. And while I’ve managed quite a variety of platforms in my time, ranging from Windows to AIX to Solaris, a little HP-UX, more recently FreeBSD, for the last 14 years or so it would be accurate to say that I’ve been a Linux guy at heart.
Linux has been my bread and butter for many years. Like any good IT guy back in the 1990’s, I snuck Linux into environments where it wasn’t welcome. I helped to get IBM’s Linux Users Group off the ground, which was actually a rather dangerous move during those years. I was also part of a rebellious group of geeks that thought it would be a good idea to have an enterprise-class Linux distribution that everyone could afford (read: free beer), so CentOS was born (though for awhile, it was cAosity, and I’m really glad the name got changed). Linux has been an amazing and wild ride for me for all of these years.
Solaris was always around, but always so damned ugly to me. I mean, let’s be honest. The command line tools were mostly not as good as GNU, and some really great common commands just weren’t there at all. Coming from Linux or *BSD, everything was just… weird. I never really got the hang of it, and I didn’t really want to. At every opportunity, I was replacing Solaris machine with Linux or OpenBSD. Quite a few times, I was installing OpenBSD right on the sparc hardware, which helped to keep up the illusion with management that we were still a Sun shop.
I’m not going to lie and say that the Illumos distros are going to be familiar or friendly to someone who only knows Linux. There is going to be a learning curve. Accept that now, and get over it. You’ll be glad you did. I’m still climbing that curve, and I’m being richly rewarded for it. The fact of the matter is, there are a number of killer features in SmartOS and other Illumos distros that you just don’t have in Linux. OK, so there is NWAM but everybody admits that pretty much sucks (unless you’re using a laptop, in which case it might be tolerable).
SmartOS especially is fighting an uphill battle when it comes to attracting new users. They are doing things very differently from everybody else, and like children pushing peas around on the plate, we’re all sort of looking at it fearfully. I’ve heard some less forward looking sysadmins make complaints about why it doesn’t have feature parity with VMware (it’s not VMware, you big dummy!) or why in the world it boots from PXE or USB or a CDROM instead of from hard disk (actually this model works really well once you “get it”). The userland tools aren’t exactly Solaris, aren’t exactly GNU. The package management isn’t exactly NetBSD. And some of the virtualization tools are straight out of the future.
SmartOS has almost no footprint on your hard disks. Your hard disks are for zones, where your virtual machines are stored. If you really want to, you can virtualize Linux or Windows guests within KVM containers. Oh yeah, SmartOS has KVM. Did I mention that? Well, it does. The other type of zone you can provision is a native zone, called a “Joyent” zone. If you’re coming from a Linux background, the closest thing you’ve got to this is probably OpenVZ as a point of comparison. But this is so much better than that.
Ideally, you would want to set up a PXE boot server for your SmartOS hosts. I plan on writing an article on that at a later date. SmartOS is delivered as a CDROM ISO image, or you can also boot it from a USB thumb drive. My old frankenbox isn’t booting from USB happily, so I’m booting from CDROM for now. An upgrade to PXE booting is planned, and I’ll take you with me through that.
The first time you boot SmartOS on a new machine, it checks out the disks and notices that the ZFS filesystem that it’s looking for doesn’t exist. All of your zones are kept in this ZFS filesytem (that sounds so redundant when I say that redundantly). So SmartOS will give you a list of disks that it sees, asks you which ones should be included in ZFS, and then it creates the filesystem for you. You’ll also need to answer the prompts for your TCP/IP configuration before rebooting. When it comes back up, it’s ready to go. You can log in at the console as “root” with the password of “root” (gotta love it… I hope for some improvement there). Hopefully the very first thing you do will be to run passwd and change root’s password.
Now it gets fun. Let’s spin up a new virtual machine, shall we?
Your hostname will look like a string of hex digits and you might be saying “WTF” to yourself. Relax. Everything in SmartOS is treated like a UUID (Universally Unique Identifier). Get comfortable with that now, because you’re going to be seeing more of it.
To provision a new virtual machine, first you’re going to want to install a template. We can do that pretty easily. First thing we need to do is install a dataset, which is sort of like a virtual machine template, that we’ll clone to make our guest zone from. Let’s see what datasets are available:
[root@00-1b-fc-58-ce-97 ~]# dsadm avail
UUID OS PUBLISHED URN
467ca742-4873-11e1-80ea-37290b38d2eb smartos 2012-02-14 sdc:sdc:smartos64:1.5.3
7ecb80f6-4872-11e1-badb-3f567348a4b1 smartos 2012-02-14 sdc:sdc:smartos:1.5.3
1796eb3a-48d3-11e1-94db-3ba91709fad9 smartos 2012-01-27 sdc:sdc:riak:1.5.5
86112bde-43c4-11e1-84df-8f7fd850d78d linux 2012-01-25 sdc:sdc:centos6:0.1.1
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd smartos 2011-12-19 sdc:sdc:smartos64:1.5.1
39c4372c-2a50-11e1-a7fa-4bebdf5eb6e6 smartos 2011-12-19 sdc:sdc:smartos:1.5.1
362f57e8-24d0-11e1-bbe6-e74758fd20c6 smartos 2011-12-12 sdc:sdc:riak:1.5.3
b4afcce4-1f8d-11e1-b4e4-fb3ae78b3768 smartos 2011-12-05 sdc:sdc:riak:1.5.2
e05dbcac-1d44-11e1-b8ab-bf1bc04c2d65 smartos 2011-12-05 sdc:sdc:smartosplus64:3.0.7
fcc5996a-1d34-11e1-899e-7bd98b87947a smartos 2011-12-05 sdc:sdc:smartosplus:3.0.7
0ac8200c-13b5-11e1-a042-7b2948257b3b linux 2011-11-21 sdc:sdc:centos57:1.1.6
fa07b5b6-118e-11e1-913c-3f246c23837c linux 2011-11-19 sdc:sdc:fedora14:0.1.0
a77907fc-123b-11e1-86d8-2bb3c7c6aaf2 linux 2011-11-18 sdc:sdc:centos6:0.1.0
56108678-1183-11e1-83c3-ff3185a5b47f linux 2011-11-18 sdc:sdc:ubuntu10.04:0.1.0
e483afce-10b2-11e1-86bc-ff468add832f linux 2011-11-17 sdc:sdc:debian603:0.1.0
70ce41ae-0cbe-11e1-8c2c-3f2bb88d2a6a smartos 2011-11-14 sdc:sdc:smartosplus64:3.0.6
aa7aeea4-0cbc-11e1-b3cb-cf1c7e291b56 smartos 2011-11-14 sdc:sdc:smartosplus:3.0.6
5fef6eda-05f2-11e1-90fc-13dac5e4a347 smartos 2011-11-03 sdc:sdc:percona:1.2.2
d91f80a6-03fe-11e1-8f84-df589c77d57b smartos 2011-11-01 sdc:sdc:percona:1.2.1
475a1026-00a4-11e1-a2df-a71425f9c46b smartos 2011-10-27 sdc:sdc:nodejs:1.3.0
8f61457a-f40a-11e0-87bd-dbd638565dbb smartos 2011-10-11 sdc:sdc:riak:1.5.1
a9380908-ea0e-11e0-aeee-4ba794c83c33 smartos 2011-09-28 sdc:sdc:percona:1.0.7
df3589dc-df9a-11e0-a3a3-07ceee3e7d54 smartos 2011-09-15 sdc:sdc:smartosplus64:3.0.4
aded640a-df98-11e0-b050-1f55ff3ddfa7 smartos 2011-09-15 sdc:sdc:smartosplus:3.0.4
9199134c-dd79-11e0-8b74-1b3601ba6206 smartos 2011-09-12 sdc:sdc:riak:1.4.1
3fcf35d2-dd79-11e0-bdcd-b3c7ac8aeea6 smartos 2011-09-12 sdc:sdc:mysql:1.4.1
141194fa-dd77-11e0-8539-27dd8d8264b8 smartos 2011-09-12 sdc:sdc:smartos64:1.4.7
f8ea0bb8-dd75-11e0-87c3-af5352ad3bd6 smartos 2011-09-12 sdc:sdc:smartos:1.4.7
a1175b2e-db23-11e0-9cef-d7ba3111e77b smartos 2011-09-09 sdc:sdc:smartos:1.4.4
47e6af92-daf0-11e0-ac11-473ca1173ab0 smartos 2011-09-09 sdc:sdc:smartos:1.3.18
80716640-cfec-11e0-8eb8-bf491eb79cd3 smartos 2011-08-28 sdc:sdc:smartos:1.4.3
be1d48ec-c4ca-11e0-9935-eb833f38dc43 smartos 2011-08-12 sdc:sdc:smartos64:1.4.2
47edd64e-c071-11e0-98c4-5bc02808198f smartos 2011-08-10 sdc:sdc:smartos:1.4.2
bb6d5a10-c330-11e0-8f18-9fbfcd26660b smartos 2011-08-10 sdc:sdc:smartos:1.3.17
47e7cf16-bf5d-11e0-9c9d-77106185df1a smartos 2011-08-05 sdc:sdc:smartos:1.3.16
7abad50e-b865-11e0-8462-730bcfdde020 smartos 2011-07-27 sdc:sdc:smartos64:1.4.1
70856d1a-b860-11e0-9bbf-bf2a7494dd82 smartos 2011-07-27 sdc:sdc:smartos:1.4.1
184c9b38-ad3d-11e0-bad6-1b7240aaa5fc smartos 2011-07-13 sdc:sdc:smartos:1.3.15
63ce06d8-7ae7-11e0-b0df-1fcf8f45c5d5 smartos 2011-05-10 sdc:sdc:smartos:1.3.13
41da9c2e-7175-11e0-bb9f-536983f41cd8 smartos 2011-04-28 sdc:sdc:nodejs:1.1.4
7456f2b0-67ac-11e0-b5ec-832e6cf079d5 smartos 2011-04-15 sdc:sdc:nodejs:1.1.3
febaa412-6417-11e0-bc56-535d219f2590 smartos 2011-04-11 sdc:sdc:smartos:1.3.12
[root@00-1b-fc-58-ce-97 ~]#
OK, so we can see there are a bunch of SmartOS branded datasets available, each providing different capabilities or different versions. There are also some Linux branded zones, which run inside of a KVM container. The first column gives us the UUID of the dataset, which is what we’ll need to actually import the dataset. Let’s try grabbing the latest 64 bit SmartOS dataset.
[root@00-1b-fc-58-ce-97 ~]# dsadm import f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd doesnt exist. continuing with install
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd successfully installed
dataset f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd successfully imported
[root@00-1b-fc-58-ce-97 ~]#
That will take a minute or two to run, maybe longer depending on your connectivity. It’s actually going out to Joyent’s cloud and making an API call to fetch the dataset.
Let’s see what datasets are available locally now.
[root@00-1b-fc-58-ce-97 ~]# dsadm list
UUID OS PUBLISHED URN
f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd smartos 2011-12-19 sdc:sdc:smartos64:1.5.1
[root@00-1b-fc-58-ce-97 ~]#
Excellent. Now we need to create a new zone from this dataset. You’re going to want to
man vmadm
and make some notes on the options. We’re basically going to put together a plaintext file in JSON format that contains the options for our new zone.
{
"alias": "honeybadger",
"brand": "joyent",
"dataset_uuid": "f9a7ab78-2a5f-11e1-a2c5-2bc2fa1d27dd",
"dns_domain": "example.com",
"quota": "10",
"nics": [
{
"nic_tag": "admin",
"ip": "192.168.0.5",
"netmask": "255.255.255.0",
"gateway": "192.168.0.1"
}
]
}
Create a file like the example above. You can adjust the IP addresses to suit your own network topology, as well as the DNS domain name. Save it to
/tmp/honeybadger
. Note the leading whitespaces in my example above are tabs, and your leading whitespace must also be in the form of tabs or this won’t parse properly.
Creating your new zone couldn’t be easier. This might take a minute or so to run so be patient:
[root@00-1b-fc-58-ce-97 ~]# vmadm create -f /tmp/honeybadger
Successfully created 0720ce96-f0dd-40c7-b01c-36356bb2e915
[root@00-1b-fc-58-ce-97 ~]#
Yes, it’s that easy. Don’t believe me? Ping the IP address that you put in
/tmp/honeybadger
:
[root@00-1b-fc-58-ce-97 ~]# ping 192.168.0.5
192.168.0.5 is alive
[root@00-1b-fc-58-ce-97 ~]#
Hey, let’s try logging into the console. You can use the UUID for your zone that the
vmadm create
command gave you as output.
[root@00-1b-fc-58-ce-97 ~]# zlogin 0720ce96-f0dd-40c7-b01c-36356bb2e915
[Connected to zone '0720ce96-f0dd-40c7-b01c-36356bb2e915' pts/5]
[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]#
Outstanding! You might wonder what packages are installed:
[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]# pkgin list
zlib-1.2.3 General purpose data compression library
sudo-1.7.7 Allow others to run commands as root
sqlite3-3.7.7.1 SQL Database Engine in a C Library
readline-6.2 GNU library that can recall and edit previous input
postfix-2.8.5 Postfix SMTP server and tools
pkgin-0.5.1 Apt / yum like tool for managing pkgsrc binary packages
pkg_install-info-4.5nb3 Standalone GNU info file installation utility
pkg_install-20110805 Package management and administration tools for pkgsrc
pkg_alternatives-1.5 Generic wrappers for programs with similar interfaces
pdksh-5.2.14nb5 Free clone of the AT&T Korn shell
pcre-8.13 Perl Compatible Regular Expressions library
patch-2.5.9 Patch files using diff output
openssl-0.9.8qnb3 Secure Socket Layer and cryptographic library
ncurses-5.9 CRT screen handling and optimization package
nbsed-20040821nb1 NetBSD-current's sed(1)
nawk-20050424nb3 Brian Kernighan's pattern-directed scanning and processing language
mozilla-rootcerts-1.0.20110902 Root CA certificates from the Mozilla Project
man-1.6e Man page viewer and utilities
libiconv-1.14nb2 Character set conversion library
libarchive-2.8.4 Library to read/create different archive formats
less-444 Pager similar to more and pg
joyent-tools-1.0 Various tools relevant to Joyent SmartOS and products
install-sh-20100824 install script compatible with the BSD install program
gtar-base-1.26 The GNU tape archiver with remote magnetic tape support
gsed-4.2.1nb1 GNU implementation of sed, the POSIX stream editor
groff-1.20.1nb5 GNU roff text processing suite
grep-2.5.3nb1 GNU grep
gettext-lib-0.18.1.1 Internationalized Message Handling Library (libintl)
genbfpw-0 Simple tool to generate Blowfish crypts of passwords
gcc-runtime-4.6.1 GNU Compiler Collection 4.6 Runtime libs
gawk-3.1.8nb1 GNU awk
findutils-4.2.33nb1 The GNU find, xargs and locate utilities
diffutils-2.8.1nb5 GNU diff utilities - find the differences between files
db4-4.8.30 Berkeley DB version 4 from Oracle
cyrus-sasl-2.1.25 Simple Authentication and Security Layer
coreutils-6.12nb4 GNU basic file, shell and text manipulation utilities
changepass-1.3.3 Simple tool to set user passwords non-interactively
bzip2-1.0.6 Block-sorting file compressor
bootstrap-mk-files-20090807nb2 *.mk files for the bootstrap bmake utility
bmake-20110606 Portable (autoconf) version of NetBSD 'make' utility
[root@0720ce96-f0dd-40c7-b01c-36356bb2e915 ~]#
The
pkgin
tool is how you’re going to install new packages inside of your zone. If you run it without any arguments, it’ll give you a nice summary of its options. I’ll be honest, a lot of the packages now are pretty old and some of the dependencies are questionable. This community could really use some packaging help. I’m hoping to adopt a few packages myself as I get to know the ins and outs of SmartOS better.
If you want to be able to ssh into your new zone, you’ll want to paste your OpenSSH public key into
~/.ssh/authorized_keys
and then
chmod 600
the same filename. You cannot ssh in as root with a password.
I’m already in excess of 2,200 words now so I should probably shut my trap at this point and allow you to digest this much. We’ll get into more depth and learn some cool tricks in upcoming installments of this blog. I hope this was a useful introduction to building your first zone, even if it’s a very high level overview. I’m still learning this myself, and will gladly accept corrections and suggestions for other articles about SmartOS.
Discovering SmartOS by Magnus Hedemark is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Based on a work at opusmagnus.wordpress.com.