I’ve been a SysAdmin for … since the last millennium. Long enough to see certain fads come and go and come again. There was a time when folks got keen on the advantages of chroot jails, but that time faded, then resurged in the form of containers! All the rage!
My own bias is that bare metal systems and VMs are what I am used to: a Unix SysAdmin knows how to manage systems! The advantages and desire for more contained environments seems to better suit certain types of programmers, and I suspect that the desire for chroot-jail-virtualenv-containers may be a reflection of programming trends.
On the one hand, you’ve got say C and Java … write, compile, deploy. You can statically link C code and put your Java all in a big jar, and then to run it on a server you’ll need say a particular kernel version, or a particular version of Java, and some light scaffolding to configure, start/stop and log. You can just write up a little README and hand that stuff off to the Ops team and they’ll figure out the mysterious stuff like chmod and the production database password. (And the load balancer config..eek!)
On the other hand, if you’re hacking away in an interpreted language: say Python or R, you’ve got a growing wad of dependencies, and eventually you’ll get to a point where you need the older version of one dependency and a bleeding-edge version of another and keeping track of those dependencies and convincing the OS to furnish them all for you … what comes in handy is if you can just wad up a giant tarball of all your stuff and run it in a little “isolated” environment. You don’t really want to get Ops involved because they may laugh at you or run in terror … instead you can just shove the whole thing in a container, run that thing in the cloud, and now without even ever having to understand esoteric stuff like chmod you are now DevOps!
(Woah: Job Security!!)
From my perspective, containers start as a way to deploy software. Nowadays there’s a bunch of scaffolding for containers to be a way to deploy and manage a service stack. I haven’t dealt with either case, and my incumbent philosophy tends to be “well, we already have these other tools” …
Container Architecture is basically just Legos mixed with Minecraft (CC: Wikipedia)
Anyway, as a Service Provider (… I know “DevOps” is meant to get away from that ugly idea that Ops is a service provider …) I figure if containers help us ship the code, we’ll get us some containers, and if we want orchestration capabilities … well, we have what we have now and we can look at bringing up other new stuff if it will serve us better.
ASIDE: One thing that has put me off containers thus far is not so much that they’re reinventing the wheel, so much that I went to a DevOps conference a few years back and it seemed every single talk was about how we have been delivered from the evil sinful ways of physical computers and VMs and the tyranny of package managers and chmod and load balancers and we have found the Good News that we can build this stuff all over in a new image and it will be called Docker or Mesos or Kubernetes but careful the API changed in the last version but have you heard we have a thing called etcd which is a special thing to manage your config files because nobody has ever figured out an effective way to … honestly I don’t know for etcd one way or another: it was just the glazed, fervent stare in the eyes of the guy who was explaining to me the virtues of etcd …
It turns out it is not just me who is a curmudgeonly contrarian: a lot of people are freaked out by the True Believers. But that needn’t keep us from deploying useful tools, and my colleague reports that Kubernetes for containers seems awfully similar to the Ganeti we are already running for VMs, so let us bootstrap some infrastructure and provide some potentially useful services to the development team, shall we?
Upon my return to work this week, one question was on the tongues of polite colleagues: “how was Alaska?” I start to explain that I didn’t experience much of Alaska because I spent the week on a cruise ship, which involved a fair bit of eating, drinking, reading, taking pictures, and trying to keep Tommy amused. I don’t trouble these nice people with too much detail. After all, there is now a blog post for those who care to know too much. Welcome to the verbose answer.
Part I: Cruise Ship Life
This is what Alaska looks like from a cruise ship.
Last week the family went on a cruise aboard Holland America’s MS Westerdam. The ship went from Seattle up the coast of Alaska and back. From that vantage, Alaska is days and days of unpopulated, beautiful vistas, floating by as you dine on an endless buffet, and catch up on reading as friendly Indonesians bring reasonably-priced drinks. The ship has something like twenty bars, a casino, a jewelry store, an “art gallery” and a modest library with absolutely no books about modern cruise ships, but various board games with missing pieces. This idyll is punctuated every day or two by our collective descent, like a plague of locusts, onto remote little towns who have decided to augment their fishing and lumber industries with tourist entrapment. “Diamonds Cheaper than on the Ship” touted several stores adjacent to the port in Juneau.
“I don’t know why Juneau has so many diamond shops,” said our driver. “Diamond isn’t even our state gem stone. You know what that is? Jade! Now if you look out to our right as we go over this bridge, you’ll see a bald eagle …” We were riding a bus out to a shore excursion where we got to ride a wheeled cart pulled by sled dogs. This was fun: you get about six tourists on a cart and a dozen or more eager dogs pull us around some roads on a loop in the woods for not more than a mile. Our musher was a guy from Michigan who explained that the hardest part of the year is driving his dogs up from Michigan, but now that tourists would pay to ride the cart the mushers could just stay up North for the Summer. His concern is that the dogs do best around -20F, so when they pull tourists around on wheeled sleds at 50F he wants to make sure they don’t work too hard and keep hydrated.
I spent a lot of my time keeping Tommy entertained. As a lady explained to the grownups about sled dogs and the annual races we got to pet a friendly dog and wander over by the musher’s camp. When the lecture was done the puppies were brought out and fondled. After the hot chocolate we got on the bus back to port. “If you look out on our left you’ll see that same eagle in the same spot.” It turns out that bald eagles spend a lot of time sitting up high enjoying the scenery and contemplating their next meal. As a cruise ship passenger, I felt I could relate.
Homo sapiens caring for its young on a cruise ship off the coast of Alaska.
Part II: Glacier Bay
The high point of the cruise, in my opinion, is when the ship sails up glacier Bay and spends an hour or so floating in front of a giant glacier:
The passengers took turns meandering on to the front deck to take pictures. Even Tommy wanted a cut of the action.
Tommy takes a picture of the glacier. Camera and wardrobe supplied by Mom.
As we floated away from the glacier I caught some of a talk from a Park Ranger about how 300 years ago Glacier Bay was more of a Glacier Valley populated by Tlingit people. But then the Little Ice Age caused havoc world wide, and the Tlingit recorded that the glacier came down through the valley at the speed of a running dog. The people ran to their canoes and evacuated. Eventually, the glacier reached the ocean. Upon contact with salt water the glacier then dried back up the valley, scraping away the ground and all evidence of Tlingit habitation, leaving what we now call Glacier Bay.
Enter John Muir. You have probably heard of him. His interest in Yosemite led him to Glacier Bay, on the idea that Yosemite may have been carved by glaciers, so he should go and study them. It was some rough adventure, and the Park Ranger digressed into a tale of how one day John went out to check out the glaciers, alone, except for one weird little dog who insisted on following him. The day consisted of a lot of jumping across crevasses and the dog kept up, until on the way home, as it was getting dark and cold, there was a crevasse that was too wide for either to jump, but there was a narrow ice bridge about ten feet down. John pulled out his axes and made it down one side, scooted across the ice bridge, and pulled himself up the far side, and looked back at the dog.
The dog looked at John, looked at the crevasse, and then began wailing. John persuaded the dog to calm down, then patiently explained that he had to try the crossing, as the only way to make it across was to try, and that if he failed to make it across that at least his bones would have a nice resting place. The dog thought it over, managed to climb down and across the ice bridge and back up to John, and they were then such BFFs that John published a book.
“Hush your fears, my boy, we will get across safe, though it is not going to be easy. No right way is easy in this rough world. We must risk our lives to save them. At the worst we can only slip, and then how grand a grave we will have, and by and by our nice bones will do good in the terminal moraine.”
–John Muir to Stickeen
It came to pass that Glacier Bay came under the protection of the federal government, which was well and good until the Tlingit came to note that it was an ancestral homeland, and the administrators of the time didn’t know what to make of that. So, after the Park Ranger spoke, a Tlingit woman came on stage to tell her own story.
The story began with an introduction to Tlingit culture. They identify by moiety, clan, and tribe. The moiety is interesting because you are either Raven or Eagle, you inherit your moiety from your mother, and you are required to marry a person of the opposite moiety. I haven’t done the logic here but it is understood to function as a system to limit in-breeding, which is a valid concern for a tribal people living at the edge of the Earth.
Anyway, her real story was of the time of forced assimilation. Her Grandmother died young, so on the pretext that a father can not raise his own children, at the age of six she and her siblings were relocated and scattered to live with families across the continental United States and thereby leave their barbarian ways behind them and become modern civilized folk. At the age of eighteen the lady’s mother returned to Alaska, where she knew nobody. She found a job and in time a nice fellow courted her, but she did not wish to marry because she did not know where she came from, or what her clan was. They conferred with elders who viewed the union as acceptable and they adopted her into a clan. In time, she learned of her birth clan, and that is how the woman speaking to us explained that due to her mother’s story, she identified with two clans.
The story gets happier with time. The woman married a Czech and has a multilingual daughter. The daughter lives in Washington but is learning Tlingit now from the University of Alaska … via Skype! And now the government has seen to the erection of a Tlingit Tribal House, which actually just opened on Thursday, August 25, 2016.
Part III: Sitka
Sitka is an island with no road connection. You arrive and depart by water or by air. Our modern cruise ship pulled up to a wooden dock on a gravel lot with piles of shipping containers. We walked on up to a little gift shop from which a fleet of buses ferried us into the city center. Our bus driver was apologetic that he didn’t know much to say about Sitka as he had been flown in from Juneau just the night before, owing to a local shortage of bus drivers, but he shared a factoid or two he had had a chance to pick up from Wikipedia. Once we got to town we had 45 minutes until another bus would whisk us on a tour to see raptors, salmon, and bears. (Oh my…) Adjacent to the bus terminal was the public library where Tommy made friends in the children’s area while his parents availed themselves of free wifi.
Oh, you were wondering: the ship has some slow, expensive wifi which we did not use except to look at the New York Times which sponsors the ship library and is therefor the only “free” site on the ship’s wifi. I don’t know if this is by design or by an oversight of the firewall configuration, but there’s no “ten article per month” limit. This is more Internet then you really need for a week at sea. More Internet that you really need on land, in all honesty. The ship is also equipped with a mobile device tower, but as with every town we stopped at in Alaska, there was no free roaming for T-Mobile.
I had to carry Tommy out of the auditorium because he was getting excited and we had been cautioned not to freak out this magnificent bald eagle, which Mommy photographed.
The Raptor Center is for rehabilitation of injured raptors, particularly bald eagles. Behind the raptor center was a nice trail with bear poop on it. It led to a stream where we figured out that dark spot in the water was a huge mass of salmon. It was all very pleasant but our time was up and we walked back up the trail, one eye out for bears, then we were off to …
… the Fortress of the Bear! Which is a refuge for orphaned bears, situated in what look to me like retired water clarifiers. Groovy stuff.
Finally, to the Sitka Science Center, where they study the life cycle of salmon and run a small hatchery operation. Since messing up the ecosystem mid-way through the last century, the state has since developed a system of hatcheries which annually release something like a billion fry a year, so there will always be plenty of tasty fish to eat. Adjacent to the center was a stream fairly choked with salmon who were returning to spawn. Someone asked if they were good eating, and the kid giving the tour explained that no, the flesh of the fish swimming upstream was already decomposing as at this point all metabolic energy they have is dedicated to the mission of spawning. The fish could still be used for animal feed and the like but no, you wouldn’t want to eat them.
Part IV: Ketchikan
A view from our cruise ship of three more cruise ships and the ever-present tourist trap fixture: a diamond store.
Daddy managed to send some postcards.
Tommy acquired a bag of blue kettle corn.
Mommy acquired some souvenirs and saw some salmon.
After the rigors of Ketchikan, Tommy is spent.
Part V: Cruise Ship Operations
I signed up for a tour of ship operations. Thanks in part to the fiber content of swiss-style muesli and a devotion to coffee, I had to excuse myself mid-way through the early-morning bridge tour, but the “hotel operations” portion of the tour was sufficiently fascinating. I was able to fill in the gap from my “bridge tour” by attending a separate talk from the Captain. If you really want to see the bridge and engines, this guy has you covered.
The ship is basically a collection of massive diesel generators. They burn a cleaner gas near shore and cheaper bunker oil at sea. The generators supply electricity to the guest facilities, the galley, and finally, to the ship’s engines, which consist of a pair of azipods mounted on both sides of the bottom rear of the ship. The azipods can rotate 160 degrees each, which combined with a set of bow thrusters, give the captain plenty of ability to park a ginormous cruise ship at little Alaskan ports. The captain noted that at 11pm when the galley shuts down, the power available for the engines goes up, and the speed ticks up a notch.
We started at the galley, which is massive. There are a handful of restaurants on the ship, and the food is all prepared in the galley, which is strategically located for quick service. If you’ve seen an industrial-sized kitchen before, then you know what’s up.
The galley. Huge. Stainless. Spotless.
Next, the bakery, which is compact, maybe the size of a two-car garage, yet still supplies the entire ship with fresh pastries throughout the week. We saw the alcohol storage room, and so of course mimosas were served.
Drinking alcohol is a favorite activity on board cruise ships.
We saw dry stores–they pointed out “the most important fuel on the ship”–a pallet of rice.
The Indonesian and Philippine crew collectively consume 500 lbs a day of rice. Any less would assure mutiny.
There is a small refrigerated room labeled “Coffin Store” which it turns out normally stores flowers–the ship has two florists–but should any of the thousands of people on board the ship expire prematurely, flowers are removed from the Coffin Store until there is enough room for the newly deceased. If the dead are capable of appreciating anything, I like to think they share my admiration and respect for the elegant efficiency of keeping the Coffin Store pre-loaded with flowers.
B Deck is under water, so you’ll see waterproof doors, and you can tell you are on a ship. You see nothing like this in guest areas.
After stores, we saw the waste management section. The ship generates an amount of waste comparable to a small city, with less room to store it. Everything that can be recycled is separated, shredded, compacted, sealed, stowed, and then sold at port when possible. Retired linens are converted into rags for cleaning the engines, and the oily rags from cleaning the engines are sealed into casks which I assume are disposed of properly. Organic (food) waste, at a rate of 3 cubic meters per day, is released into the ocean at night while the ship is chugging along. The organic waste is released in 1cm cubes so as not to attract seagulls into forming an entourage behind the aft staterooms.
During the Bridge Tour the Captain noted that waste water from the toilets is used for ballast. This makes the “wet sewage wastes” cask all the more mysterious.
There’s a mess for ship’s crew and another, larger, more aesthetically appealing mess for the Indonesian and Philippine crew, where the bulk of the ship’s rice is consumed. We had to wait until Friday prayers were completed before we could see the latter mess area, which makes an attempt to remind folks who are at sea serving well-off Americans of the life and vibrant color of their home land. Whether the canteen decor does anything for morale I do not know, but I reckon the ritual use of a clean laundry bin filled with prayer rugs helps more than a few lonely souls keep their spirits up.
The mess hall decor tries to remind the staff of home.
Later in the tour we breached American etiquette to learn a bit about the salary on board ship. One assumes the money is good enough to convince folks to leave home, typically for ten months at a time. We were informed that stateroom attendants, after tips, can take home $1,500-$1,800/mo. It was noted that stateroom attendants make considerably better money than other staff, especially compared to, say, a porter, whose job is mainly to carry stuff around.
Checking up on the Internet, the average salary in Indonesia is about $1,200, and the median is about $750. Kitchen Staff average $90/mo, and a Waiter $300/mo. A Call Center job around $700/mo. The World Bank ranks Indonesia as “lower middle income” … I’m not sure I will ever have gotten my head and my heart around the disparities of our world. I reckon it is better that I never do.
We visited the ship tailors, whose main occupation is in keeping the staff properly attired. The hotel laundry has a lot of busy men and machines: washers, dryers, hospital-grade sanitizers, automated presses for pants and for suits. The dry cleaning is … look, in all honestly the wonders of the laundry were pretty much lost on me, save for the existence of a $400,000 machine about the size of our living room that folds sheets. There is a separate laundromat on the ship for the staff to do their own laundry at no charge.
Part VI: Victoria
The night before our return to Seattle was a stop in Victoria, Canada. Before our arrival the captain made a ship-wide, long-winded announcement in his thick Dutch accent, explaining that overnight, they had a problem with one of the azipods, so they had to stop it, turn the ship around, turn the azipod back on, then resume course. But they hadn’t been able to make up the time so we would arrive in Victoria about 45 minutes late, and this is why he was deeply apologetic to those whose shore excursions would consequently be rescheduled or canceled outright.
Nobody cared to see our passports. I grabbed some Canadian cash and we rode a double-decker bus into town, which resembled France. As it was around his bed time, Tommy fell asleep on my shoulder and I got to carry him around town. We bought some chocolates and I took a seat near one of the buskers down at the waterfront while Mommy took some pictures. We later strolled around the kiosks at the waterfront and Tommy managed to awake in time to catch site of a food truck containing an industrial robot serving ice cream. If there is one thing every parent knows about Canada it is that children are entitled to any robot-dispensed soft-serve ice cream that they can spot.
At one point the robot encountered some imperceptible difficulty, and three humans instantly appeared to render technical support. Among other things someone had to fiddle with the robot’s computer, which is a Japanese version of Windows XP.
We took a taxi back to the ship. Nobody cared to see our passports.
