My preschool was in a decrepit VA facility. One day a pipe burst and one of the classrooms was flooding. So they worked around it but to me it felt apocalyptic and I cried accordingly.
Another time I made a bird feeder with a roll of toilet paper, slathered in peanut butter, rolled in bird seed.
Another time we helped the teachers unwrap hella taffy cubes that they melted in a pot and dipped apples in to make us taffy apples. That was excellent.
Another time they gave us eye exams and I was upset that half the exam I couldn’t see the promised capital E. I felt I had been misled.
We would walk as a class down the halls of the VA hospital to the playground, and broken men slept along the walls. We never spoke of them, that I recall.
Part of my day job involves looking at Nagios and checking up on systems that are filling their disks. I was looking at a system with a lot of large files, which are often duplicated, and I thought this would be less of an issue with de-duplication. There are filesystems that support de-duplication, but I recalled the fdupes command, a tool that “finds duplicate files” … if it can find duplicate files, could it perhaps hard-link the duplicates? The short answer is no.
But there is a fork of fdupes called jdupes, which supports de-duplication! I had to try it out.
It turns out your average Hadoop release ships with a healthy number of duplicate files, so I use that as a test corpus.
> du -hs hadoop-3.3.4
1.4G hadoop-3.3.4
> du -s hadoop-3.3.4
1413144 hadoop-3.3.4
> find hadoop-3.3.4 -type f | wc -l
22565
22,565 files in 1.4G, okay. What does jdupes think?
> jdupes -r hadoop-3.3.4 | head
Scanning: 22561 files, 2616 items (in 1 specified)
hadoop-3.3.4/NOTICE.txt
hadoop-3.3.4/share/hadoop/yarn/webapps/ui2/WEB-INF/classes/META-INF/NOTICE.txt
In the ls output, the “2” in the second column indicates the number of hard links to a file. Before we ran jdupes, each file only linked to itself. After, these two files link to the same spot on disk.
> du -s hadoop-3.3.4
1388980 hadoop-3.3.4
> find hadoop-3.3.4 -type f | wc -l
22566
The directory uses slightly less space, but the file count is the same!
But, be careful!
If you have a filesystem that de-duplicates data, that’s great. If you change the contents of a de-duplicated file, the filesystem will store the new data for the changed file and the old data for the unchanged file. If you de-duplicate with hard links and you edit a deduplicated file, you edit all the files that link to that location on disk. For example:
Both files are now 4 bytes longer! Maybe this is desired, but in plenty of cases, this could be a problem.
Of course, the nature of how you “edit” a file is very important. A file copy utility might replace the files, or it may re-write them in place. You need to experiment and check your documentation. Here is an experiment.
> ls -l hadoop-3.3.4/libexec/hdfs-config.cmd hadoop-3.3.4/libexec/mapred-config.cmd
-rwxr-xr-x 2 djh djh 1640 Mar 23 16:19 hadoop-3.3.4/libexec/hdfs-config.cmd
-rwxr-xr-x 2 djh djh 1640 Mar 23 16:19 hadoop-3.3.4/libexec/mapred-config.cmd
> cp hadoop-3.3.4/libexec/hdfs-config.cmd hadoop-3.3.4/libexec/mapred-config.cmd
cp: 'hadoop-3.3.4/libexec/hdfs-config.cmd' and 'hadoop-3.3.4/libexec/mapred-config.cmd' are the same file
The cp command is not having it. What if we replace one of the files?
> cp hadoop-3.3.4/libexec/mapred-config.cmd hadoop-3.3.4/libexec/mapred-config.cmd.orig
> echo foo >> hadoop-3.3.4/libexec/hdfs-config.cmd
> ls -l hadoop-3.3.4/libexec/hdfs-config.cmd hadoop-3.3.4/libexec/mapred-config.cmd
-rwxr-xr-x 2 djh djh 1644 Mar 23 16:19 hadoop-3.3.4/libexec/hdfs-config.cmd
-rwxr-xr-x 2 djh djh 1644 Mar 23 16:19 hadoop-3.3.4/libexec/mapred-config.cmd
> cp hadoop-3.3.4/libexec/mapred-config.cmd.orig hadoop-3.3.4/libexec/mapred-config.cmd
> ls -l hadoop-3.3.4/libexec/hdfs-config.cmd hadoop-3.3.4/libexec/mapred-config.cmd
-rwxr-xr-x 2 djh djh 1640 Mar 23 16:19 hadoop-3.3.4/libexec/hdfs-config.cmd
-rwxr-xr-x 2 djh djh 1640 Mar 23 16:19 hadoop-3.3.4/libexec/mapred-config.cmd
When I run the cp command to replace one file, it replaces both files.
Back at work, I found I could save a lot of disk space on the system in question with jdupes -L, but I am also wary of unintended consequences of linking files together. If we pursue this strategy in the future, it will be with considerable caution.
I wondered about the name “Las Vegas.” I am a California resident who dabbles in Duo Lingo. “Las Vegas” means “The Meadow,” named by a Spaniard who enjoyed stopping over at this grassy meadow in the desert. A meadow fed by natural springs. Later, the Americans would come and build pumps, draining the springs, and leaving the city to sink several feet into the desert sand: vegas no más. Other minds beheld this sunken place near the Hoover Dam, nodded at the lack of prohibitions on gambling, and began building casinos. The casinos begat a nice airport. Other minds contemplated diversification: cheap power, cheap land and easy logistics are all good for the datacenter business. The party will last as long as the Colorado River flows. It is a visit to the dimmed lights of the datacenter that brought me.
When America allows itself to take all its worst instincts and run with them, we get Las Vegas. The City feels like any suburb in America: a series of large shopping malls in the middle of town sealed in to their own reality, surrounded by humbler strip malls where the locals satisfy their day-to-day commercial needs. All generously connected by a grid of six lane roads and choked highways. The area leading to the Buy N Large datacenter is rocky desert strewn with trash. A developing country missing its stray dogs.
It was from the driver of the rental car shuttle bus that I gleaned a potential use for Las Vegas. As we rode from the Airport Terminal to the Car Rental complex across the street, he took to the mic to entertain and inform. The weather was in the fifties that day, but in a few days the forecast called for ninety. He then explained that the airport had recently been renamed from “McCarran” to “Harry Reid” and fortunately most of the signs have been changed, but for the first year a lot of folks had been confused. Our driver then informed us of a list of national parks and how many hours of driving they were from Las Vegas: Bryce, Zion, Death Valley, and the North Rim and the South Rim of the Grand Canyon. Ah, I thought: as an air hub, Las Vegas could be a good spot to fly to with the family to rent a car and explore. Coastal Northern California is surpringly far from The West. Perhaps I would be back.
An early morning line at the airport Starbucks, where I’m pretty sure I won’t be served.
The morning was long. I had been up at 5:30 to shower, drive, park, and catch my flight. I had hoped for a pastry and coffee at the airport Starbucks, but the line was long and moved at zombie speed. I stood patiently for a few minutes until the airport loud speaker announced that my plane was now boarding, and I had the length of the terminal to cross. On the plane, the pilot announced that due to the unusual presence of Weather in California, the flight could be choppy, so no “service” would be attempted, for safety. At Harry Reid’s rental car terminal, there was a big Starbucks in the middle, but I had an orientation to get to at the Buy N Large datacenter, and at that point I had achieved my “cruising altitude” for the morning and didn’t need anything. I made my way to the cars and picked the Blue One. When asked to choose a rental car, I try to go for the most unusual color, in hopes of remembering which car I was driving.
Driving rental cars is its own pleasure for me, because at any given point in my life I am probably driving an older car. Behold, the crisp video feed from the backup camera! How does the cruise control work? Lane keeping! That’s neat. Where do I put my phone for easy navigation. Once I discover that I can pair my phone to the car’s video screen I am in a good place. At some point I ask myself whether I might want to be the kind of person who always drives a newer car. And I explain to myself that sounds nice but what is even better than that is to be the kind of person who doesn’t spend enough time driving for car quality to be important.
The Buy N Large datacenter has several entries in the Maps App. Because I had shipped some hardware last month, I recognized the street name of the one I needed to get to. I met my colleague and an armed security guard gave us the orientation, and guided us through our navigation of the sectors. Buy N Large is the largest data center I have ever worked in. It is one of those monuments which people in my line of work are likely to visit at some point in our careers. I recalled an old colleague who gave up living in Oakland, because The Company was content to have him work remote near the datacenter. He was content to rent a Large House to share with his cats, and drive out into the desert some nights to look up at the stars. Las Vegas was a home base from which he could visit The Universe.
Between orientation and getting work done I needed to eat. I asked The App for Brunch and settled on a place called “Mr Mamas.” A diner in a strip mall. Clean and efficient and delicous, with American portions. I had French Toast and eggs and a lot of coffee and was in a great mood for an afternoon of Moving Cables Around. At one point, I realized I would need More Cables which can be a problem because after all, Fry’s Electronics is no more. But the app suggested that Kiesub Electronics was on the way to Grainger. I hopped in the Blue Car and found The Cables that I needed at Kiesub. I had wanted to buy Extra but they had exactly Enough for my purposes. I got to chatting with the guy and he noted that while Fry’s had come and gone, Kiesub had stood for fifty years. We chatted some more. He inquired about me and I enumerated my blessings, and noted that for me, everything was pretty great. For Now. I’ll always remember the Lean Years after 2001. The guy had been married some decades and explained that while Marriage is Work, it really helps if you don’t take yourself too seriously. Amen.
Back at Buy N Large, I got the cables moved around and around 5:30pm, I called it A Day. I checked in at the hotel and asked for advice regarding dinner. The clerk kindly explained her favorite options which I duly checked out but I just wasn’t Feeling It. I wanted to sit at The Bar, somewhere quiet. I resorted to asking The App for Irish Pubs. After all, that is our comfort in Sunnyvale, which is the name we settled on when the Post Office told us we couldn’t call our town Murphy. The first on the list was in Mandalay Bay, which is a massive golden cube. I drove up to it, and pulled into a driveway. I passed a line of taxis wondering what the parking situation would be. I was deposited back out onto another six lane street. I asked The App again, and scrolled West into the Sprawl. I found my way to an Irish Pub in a Strip Mall. The parking lot was full, but a local vouched that the No Parking Tow Zone filled with parked cars was a place he parked Every Week. For tonight was Trivia Night.
I sat at the bar and the menu bragged that the Fish and Chips were the best in the US in 2019. I had travelled to an inland desert and I ordered The Fish and it was tasty. As suggested, I filled out the trivia cards. Brian the Owner stood near and we chatted. I told him about Buy N Large and he recounted a friend who was gifted in the ways of computers who had a confidence that he could talk himself out of anything, who had met a violent end from a neighbor who had mental problems. It was Halloween, and another Body in the yard had initially been mistaken by the kids as a decoration.
Come morning, I surveyed the Hotel Breakfast. Eager guests fed themselves off styrofoam plates, as is The Custom at American Hotel Breakfast Buffets. I allowed myself to recoil and to drive back over to Mr Mamas to enjoy the same damn meal I had enjoyed the day before. It did not disappoint. I dropped by Buy N Large to check on my colleague. My work done, I dropped in at a local coffee shop, which was okay. Back to Buy N Large, to bring my colleague to the Rental Car Return and on to the airport, where we parted ways, to our different airlines serving different sectors of the Bay Area.
I had a few hours to kill. I walked the length of the terminal, studying my options for sustenance and souveniers. I eventually settled on a $4 Nathan’s hot dog and discovered another Irish Bar next to my gate. A guy left a Blue Moon at the counter, which the barkeep acknowledged would be an insult in Ireland. I took this neglected pint under my care, which I nursed alongside my own Goose Island IPA. Another Illinoisan from Naperville who had matriculated from the same High School as the girl I had once dated from Naperville asked the Irish Bartender what he thought of the mixed drink known as an “Irish Car Bomb.” The bartender named a woman who he had known who got blown up in the early eighties. “It wasn’t intended for her, but her boyfriend, an English soldier.” If an Irish Pub promoted “Car Bombs” you could tell it was run by Americans. Conversation passed well through a third beer.
“Nevada” means “snow” and this year the “Sierra Nevada” lives up to its name.
The flight home was pleasant. I was served a Coke and a snack and had time for the buzz to recede so that I could drive home safely from the Long Term Parking, and help my sweetheart put our boys to bed.
I have friends who in 2023 have occasion to leave their house and they return with breathless reports of how few people at the airport were wearing masks. They cluck to each other at how sad it is that humanity seems to have given up the fight against the Coronavirus. Meanwhile my prison friend says that he’s probably had Coronavirus a half dozen times. He says he doesn’t want to get a vaccination because he really doesn’t trust the government.
Rags
Time Magazine likes to talk about how great it is to have been Time magazine in the past. An attentive reader is in and out in fifteen minutes.
The New Yorker Magazine will explain that living in New York today has it’s moments, but that the world is full of meaty goings on and it can be fascinating to explore a few of these things in depth if you have a few hours to kill. Or just look at the cartoons. This is New York, after all.
Detour Spiral
Public Transit advocates are concerned that because of a lack of funding and a lack of riders, public transit could soon enter a death spiral which means they cut back services so fewer people ride so they cut back services, etc.
This morning, the highway was closed down on both sides because of a multiple car pileup. Death Spiral. My children and I rode our bikes to school. On the way back some douchebag in a truck felt obliged to honk his horn at me because he had to detour around the Death Spiral and what kind of jerk rides a bicycle down a narrow neighborhood street?
SFO Outbound
On our way to the gate, we saw a pack of soldiers, dressed in fatigues. I got excited, a tingle, to see they were Ukraine soldiers. “Slava Ukraine,” said I. I noticed at least three prosthetics among them. A nice blade foot and two guys with claw hands. My guess is that they hadn’t come to the US for training, but for some leave, earned hard.
They got their bearings, turned around and returned in the direction they had come. Were they coming or going? When people are surviving a war, the future is especially hazy.
Any Questions?
“Any questions,” asks the waiter.
“… why is it so expensive?” Asks our older son.
“Questions about the food,” we prompt him.
But his is a good question. The food is expensive, but we have money. But when I was growing up, we wouldn’t eat at a place this expensive. We had money but not the kind of money his family grows up with. He is aware of his privilege. We want him to grow up not to be an entitled jerk. If he is occasionally questioning the Price of Things, I guess we aren’t doing so bad?
He knows he has Privilege. Why does his family have more money than others, I ask myself. Shouldn’t we all make the same … shouldn’t we be equal?
I think to myself, I have said it before, for the same money, I would wait tables. Computers are engaging but helping people is emotionally rewarding. The market economy says pay the computer technicians more to incentivize them to use the rare skills we all so desperately need. You can’t have all the computer guys wandering off to serve in more emotionally rewarding roles!
Or can you? Necessity .. invention ..
Ursula Le Guin. I think of the novel about a planet where the people have no gender, except for the brief periods where they need to mate. Their planet is Socialist. Or was it Anarchist? People are assigned jobs for a period of time by a computer. A fair system. Maybe not as efficient as we prize.
I think I would enjoy not doing the same career forever. But the money … I can not complain too loudly. This frustration is enviable.
Helping people is a reward in itself. Early on, I preferred IT. Or, as I called it: Information Services. But the economic path of the profession seeks to divorce itself from the “cost center” of “helping people work more effectively” to the prestige concept of “Engineering” … Systems Administrators call themselves DevOps now, which is a nonsense word that connotes “Developer Operators,” I guess?
I was thinking about National Service the other day. I have long thought it would be maybe not such a bad thing if we “earned” the right to vote by demonstrating our personal commitment to our collective success. But it needn’t just be a year or six months in your youth. The tree of liberty needs constant watering. Every decade or so, spend a few months helping out. In the classroom. In the streets. On the land. Cleaning a public restroom. Doing what needs doing. Helping a family with paperwork at the hospital or the funeral home. Learning the skills we will all need at some point.
The people who run the computers. The people who run our businesses. The people in charge. The People with Privilege. These are all folks who could use some better context in their “day jobs” just as anyone and everyone could use an open pair of eyes. To ask the questions worth asking.
Under Water
I have a friend who posts trench warfare videos on his Facebook. I see a lot of Russian soldiers get killed each week. I take a dose of joy and sadness at the same time.
My sympathy for people who had everything and paid a bunch of money to a charlatan and signed all the disclaimers for the adventure of riding in a janky submarine … good for them. They died as they lived. Lives of privilege. They have no need of my sympathy.
Some guy from a rust belt mining town in the Ural mountains who signed up to die in a shit-stained trench in a propaganda video on Facebook. He made a bad choice among bad options. My feels for that guy, and his family.
I wait in the lobby, sprawled on a bench. At one point there’s more of a crowd. I sit up and set the carrier on the floor.
I wait. And I wait. Staff have called in sick, but the triage nurse is on top of things.
I watch the parade. A family comes in with a dog. The dog is in trouble. They go back right away.
I wait. And I wait. Another family comes in with another dog. This dog is in trouble. They go back right away.
I wait, and I watch the families return, one by one, teary eyes. No dog.
I wait. The front desk quietly chatter about clients selecting urns.
I wait. The front desk staff leave.
It is 2am. I see the Doctor. We run through The History together. I am not very good answering questions. My middle aged mind doesn’t run full tilt at 2am. The bulb on the cat’s nose that is filled with puss, one can squeeze like a zit. “Your cat is not very happy with me now.” Fair enough. A prescription for something that can stimulate appetite. Wear gloves and rub it in the ear. Alright.
We get home. The cat is hungry. I feed her. I may not need to rub medicine in her ears.
The article “Teenagers’ accidents expose e-bike risks” published on July 30 is a master class in victim blaming. We are provided several examples of someone riding an e-bike who is then injured or killed when being forced to mix with motorized traffic.
The problem isn’t e-bikes, the problem is that we have chosen not to provide safe routes for people to get around on bicycles. E-bikes magnify this failure by making it easier for more people to ride.
E-bikes can also lead to the fix: as more people ride bicycles, there will be more pressure to build safe routes for people to get around on bikes. More bicycles means fewer cars on the road, reduced Carbon emissions, and less road congestion.
We need to stop blaming our children for our failures and get to work.
