dannyman.toldme.com


Technical

A Complex Cron Entry Explained

Link: https://dannyman.toldme.com/2017/01/16/crontab-backtick-returncode-conditional-command/

A crontab entry I felt worth explanation, as it illustrates a few Unix concepts:

$ crontab -l
# m h dom mon dow command
*/5 8-18 * * mon-fri grep -q open /proc/acpi/button/lid/LID0/state && mkdir -p /home/djh/Dropbox/webcam/`date +%Y%m%d` && fswebcam -q -d /dev/video0 -r 1920x1080 /home/djh/Dropbox/webcam/%Y%m%d/%H%M.jpg ; file /home/djh/Dropbox/webcam/`date +%Y%m%d/%H%M`.jpg | grep -q 1920x1080 || fswebcam -q -d /dev/video1 -r 1920x1080 /home/djh/Dropbox/webcam/%Y%m%d/%H%M.jpg

That is a huge gob of text. Let me un-pack that into parts:

*/5 8-18 * * mon-fri

Run weekdays, 8AM to 6PM, every five minutes.

grep -q open /proc/acpi/button/lid/LID0/state &&
mkdir -p /home/djh/Dropbox/webcam/`date +%Y%m%d` &&
fswebcam -q -d /dev/video0 -r 1920x1080 /home/djh/Dropbox/webcam/%Y%m%d/%H%M.jpg

Check if the lid on the computer is open by looking for the word “open” in /proc/acpi/button/lid/LID0/state AND
(… if the lid is open) make a directory with today’s date AND
(… if the directory was made) take a timestamped snapshot from the web cam and put it in that folder.

Breaking it down a bit more:

grep -q means grep “quietly” … we don’t need to print that the lid is open, we care about the return code. Here is an illustration:

$ while true; do grep open /proc/acpi/button/lid/LID0/state ; echo $? ; sleep 1; done
state: open
0
state: open
0
# Lid gets shut
1
1
# Lid gets opened
state: open
0
state: open
0

The $? is the “return code” from the grep command. In shell, zero means true and non-zero means false, that allows us to conveniently construct conditional commands. Like so:

$ while true; do grep -q open /proc/acpi/button/lid/LID0/state && echo "open lid: take a picture" || echo "shut lid: take no picture" ; sleep 1; done
open lid: take a picture
open lid: take a picture
shut lid: take no picture
shut lid: take no picture
open lid: take a picture
open lid: take a picture

There is some juju in making the directory:

mkdir -p /home/djh/Dropbox/webcam/`date +%Y%m%d`

First is the -p flag. That would make every part of the path, if needed (Dropbox .. webcam ..) but it also makes mkdir chill if the directory already exist:

$ mkdir /tmp ; echo $?
mkdir: cannot create directory ‘/tmp’: File exists
1
$ mkdir -p /tmp ; echo $?
0

Then there is the backtick substitition. The date command can format output (read man date and man strftime …) You can use the backtick substitution to stuff the output of one command into the input of another command.

$ date +%A
Monday
$ echo "Today is `date +%A`"
Today is Monday

Once again, from the top:

grep -q open /proc/acpi/button/lid/LID0/state &&
mkdir -p /home/djh/Dropbox/webcam/`date +%Y%m%d` &&
fswebcam -q -d /dev/video0 -r 1920x1080 /home/djh/Dropbox/webcam/%Y%m%d/%H%M.jpg ;
file /home/djh/Dropbox/webcam/`date +%Y%m%d/%H%M`.jpg | grep -q 1920x1080 ||
fswebcam -q -d /dev/video1 -r 1920x1080 /home/djh/Dropbox/webcam/%Y%m%d/%H%M.jpg

Here is where it gets involved. There are two cameras on this mobile workstation. One is the internal camera, which can do 720 pixels, and there is an external camera, which can do 1080. I want to use the external camera, but there is no consistency for the device name. (The external device is video0 if it is present at boot, else it is video1.)

Originally, I wanted to do like so:

fswebcam -q -d /dev/video0 -r 1920x1080 || fswebcam -q -d /dev/video1 -r 1920x1080

Unfortunately, fswebcam is a real trooper: if it can not take a picture at 1920×1080, it will take what picture it can and output that. This is why the whole cron entry reads as:

Check if the lid on the computer is open AND
(… if the lid is open) make a directory with today’s date AND
(… if the directory was made) take a timestamped snapshot from web cam 0
Check if the timestamped snapshot is 1920×1080 ELSE
(… if the snapshot is not 1920×1080) take a timestamped snapshot from web cam 1

Sample output from webcam. Happy MLK Day.

Why am I taking these snapshots? I do not really know what I might do with them until I have them. Modern algorithms could analyze “time spent at workstation” and give feedback on posture, maybe identify “mood” and correlate that over time … we’ll see.

OR not.

Feedback Welcome


Linux, Technical, Technology

Duct Tape Ops

Link: https://dannyman.toldme.com/2017/01/27/duct-tape-ops/

Threading specification, via Wikipedia

Yesterday we tried out Slack’s new thread feature, and were left scratching our heads over the utility of that. Someone mused that Slack might be running out of features to implement, and I recalled Zawinski’s Law:

Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

I think this is a tad ironic for Slack, given that some people believe that Slack makes email obsolete and useless. Anyway, I had ended up on Jamie Zawiski’s (jwz) Wikipedia entry and there was this comment about jwz’s law:

Eric Raymond comments that while this law goes against the minimalist philosophy of Unix (a set of “small, sharp tools”), it actually addresses the real need of end users to keep together tools for interrelated tasks, even though for a coder implementation of these tools are clearly independent jobs.

This led to The Duct Tape Programmer, which I’ll excerpt:

Sometimes you’re busy banging out the code, and someone starts rattling on about how if you use multi-threaded COM apartments, your app will be 34% sparklier, and it’s not even that hard, because he’s written a bunch of templates, and all you have to do is multiply-inherit from 17 of his templates, each taking an average of 4 arguments … your eyes are swimming.

And the duct-tape programmer is not afraid to say, “multiple inheritance sucks. Stop it. Just stop.”

You see, everybody else is too afraid of looking stupid … they sheepishly go along with whatever faddish programming craziness has come down from the architecture astronauts who speak at conferences and write books and articles and are so much smarter than us that they don’t realize that the stuff that they’re promoting is too hard for us.

“At the end of the day, ship the fucking thing! It’s great to rewrite your code and make it cleaner and by the third time it’ll actually be pretty. But that’s not the point—you’re not here to write code; you’re here to ship products.”

jwz wrote a response in his blog:

To the extent that he puts me up on a pedestal for merely being practical, that’s a pretty sad indictment of the state of the industry.

In a lot of the commentary surrounding his article elsewhere, I saw all the usual chestnuts being trotted out by people misunderstanding the context of our discussions: A) the incredible time pressure we were under and B) that it was 1994. People always want to get in fights over the specifics like “what’s wrong with templates?” without realizing the historical context. Guess what, you young punks, templates didn’t work in 1994.

As an older tech worker, I have found that I am more “fad resistant” than I was in my younger days. There’s older technology that may not be pretty but I know it works, and there’s new technology that may be shiny, but immature, and will take a lot of effort to get working. As time passes, shiny technology matures and becomes more practical to use.

(I am looking forward to trying “Kubernetes in a Can”)

Feedback Welcome


Ganeti, Linux, Technical

Ganeti: Segregate VMs from Running on the Same Hardware

Link: https://dannyman.toldme.com/2017/02/16/ganeti-exclusion-tags/

We have been using this great VM management software called Ganeti. It was developed at Google and I love it for the following reasons:

It is frustrating that relatively few people know about and use Ganeti, especially in the Silicon Valley.

Recently I had an itch to scratch. At the recent Ganeti Conference I heard tell that one could use tags to tell Ganeti to keep instances from running on the same node. This is another excellent feature: if you have two or more web servers, for example, you don’t want them to end up getting migrated to the same hardware.

Unfortunately, the documentation is a little obtuse, so I posted to the ganeti mailing list, and got the clues lined up.

First, you set a cluster exclusion tag, like so:

sudo gnt-cluster add-tags htools:iextags:role

This says “set up an exclusion tag, called role

Then, when you create your instances, you add, for example: --tags role:prod-www

The instances created with the tag role:prod-www will be segregated onto different hardware nodes.

I did some testing to figure this out. First, as a control, create a bunch of small test instances:

sudo gnt-instance add ... ganeti-test0
sudo gnt-instance add ... ganeti-test1
sudo gnt-instance add ... ganeti-test2
sudo gnt-instance add ... ganeti-test3
sudo gnt-instance add ... ganeti-test4

Results:

$ sudo gnt-instance list | grep ganeti-test
ganeti-test0 kvm snf-image+default ganeti06-29 running 1.0G
ganeti-test1 kvm snf-image+default ganeti06-29 running 1.0G
ganeti-test2 kvm snf-image+default ganeti06-09 running 1.0G
ganeti-test3 kvm snf-image+default ganeti06-32 running 1.0G
ganeti-test4 kvm snf-image+default ganeti06-24 running 1.0G

As expected, some overlap in service nodes.

Next, delete the test instances, set a cluster exclusion tag for “role” and re-create the instances:

sudo gnt-cluster add-tags htools:iextags:role
sudo gnt-instance add ... --tags role:ganeti-test ganeti-test0
sudo gnt-instance add ... --tags role:ganeti-test ganeti-test1
sudo gnt-instance add ... --tags role:ganeti-test ganeti-test2
sudo gnt-instance add ... --tags role:ganeti-test ganeti-test3
sudo gnt-instance add ... --tags role:ganeti-test ganeti-test4

Results?

$ sudo gnt-instance list | grep ganeti-test
ganeti-test0 kvm snf-image+default ganeti06-29 running 1.0G
ganeti-test1 kvm snf-image+default ganeti06-09 running 1.0G
ganeti-test2 kvm snf-image+default ganeti06-32 running 1.0G
ganeti-test3 kvm snf-image+default ganeti06-24 running 1.0G
ganeti-test4 kvm snf-image+default ganeti06-23 running 1.0G

Yay! The instances are allocated to five distinct nodes!

But am I sure I understand what I am doing? Nuke the instances and try another example: 2x “www” instances and 3x “app” instances:

sudo gnt-instance add ... --tags role:prod-www ganeti-test0
sudo gnt-instance add ... --tags role:prod-www ganeti-test1
sudo gnt-instance add ... --tags role:prod-app ganeti-test2
sudo gnt-instance add ... --tags role:prod-app ganeti-test3
sudo gnt-instance add ... --tags role:prod-app ganeti-test4

What do we get?
$ sudo gnt-instance list | grep ganeti-test
ganeti-test0 kvm snf-image+default ganeti06-29 running 1.0G # prod-www
ganeti-test1 kvm snf-image+default ganeti06-09 running 1.0G # prod-www
ganeti-test2 kvm snf-image+default ganeti06-29 running 1.0G # prod-app
ganeti-test3 kvm snf-image+default ganeti06-32 running 1.0G # prod-app
ganeti-test4 kvm snf-image+default ganeti06-24 running 1.0G # prod-app

Yes! The first two instances are allocated to different nodes, then when the tag changes to prod-app, ganeti goes back to ganeti06-29 to allocate an instance.

Feedback Welcome


News and Reaction

The Economist: notes on “Free Parking”

Link: https://dannyman.toldme.com/2017/04/06/the-economist-notes-on-free-parking/

Notes from a great article in the Economist on the hidden costs of free parking.

Apple’s new “Spaceship” in Cupertino contains 318,000 m2 of offices and 325,000 m2 of parking.

Cupertino, the suburban city where the new headquarters is located, demands it. Cupertino has a requirement for every building. A developer who wants to put up a block of flats, for example, must provide two parking spaces per apartment, one of which must be covered. For a fast-food restaurant, the city demands one space for every three seats; for a bowling alley, seven spaces per lane plus one for every worker. Cupertino’s neighbours have similar rules. With such a surfeit of parking, most of it free, it is little wonder that most people get around Silicon Valley by car, or that the area has such appalling traffic jams.

Cars sit idle 95% of the time.

Water companies are not obliged to supply all the water that people would use if it were free, nor are power companies expected to provide all the free electricity that customers might want. But many cities try to provide enough spaces to meet the demand for free parking, even at peak times. Some base their parking minimums on the “Parking Generation Handbook”, a tome produced by the Institute of Transportation Engineers. This reports how many cars are found in the free car parks of synagogues, waterslide parks and so on when they are busiest.

Car parking takes up space. Parking lots dominate the downtown area of Kansas City, MO. As space gets stretched out, walking and bicycling lose their appeal. “Besides, if you know you can park free wherever you go, why not drive?”

The rule of thumb in America is that multi-storey car parks cost about $25,000 per space and underground parking costs $35,000. Donald Shoup, an authority on parking economics, estimates that creating the minimum number of spaces adds 67% to the cost of a new shopping centre in Los Angeles if the car park is above ground and 93% if it is underground. Parking requirements can also make redevelopment impossible. Converting an old office building into flats generally means providing the parking spaces required for a new block of flats, which is likely to be difficult. The biggest cost of parking minimums may be the economic activity they prevent.

There Is No Such Thing As A Free Lunch: everyone pays for free parking:

And that has an unfortunate distributional effect, because young people drive a little less than the middle-aged and the poor drive less than the rich. In America, 17% of blacks and 12% of Hispanics who lived in big cities usually took public transport to work in 2013, whereas 7% of whites did. Free parking represents a subsidy for older people that is paid disproportionately by the young and a subsidy for the wealthy that is paid by the poor.

When autonomous cars become available, many will likely operate like taxis. Less parking will be needed for homes and businesses. There will be more demand for drop-off and pick-up areas. There will be more demand for service garages, where the autonomous cars can go to charge, clean, and receive maintenance.

Existing parking minimums, which provide a subsidy for individual car ownership, will retard the adoption of autonomous vehicles in the United States. Personal vehicles will be subject to a parking subsidy, whereas autonomous car operators will need to supply maintenance garages at their expense. See Also: streetcars versus buses, railroads versus trucking.

Market-rate parking permits for public streets are logical, but culturally unpopular, even in transit-based European cities like Amsterdam.

The result is a perpetual scrap for empty kerb. As San Francisco’s infuriated drivers cruise around, they crowd the roads and pollute the air. This is a widespread hidden cost of under-priced street parking. Mr Shoup has estimated that cruising for spaces in Westwood village, in Los Angeles, amounts to 950,000 excess vehicle miles travelled per year. Westwood is tiny, with only 470 metered spaces.

In the 1950s, while Japan was still poor, Tokyo required motorists to show proof of access to a dedicated parking space. There is no overnight street parking in Tokyo.

Freed of cars, the narrow residential streets of Tokyo are quieter than in other big cities. Every so often a courtyard or spare patch of land has been turned into a car park—some more expensive than others. Once you become accustomed to the idea that city streets are only for driving and walking, and not for parking, it is difficult to imagine how it could possibly be otherwise. Mr Kondoh is so perplexed by an account of a British suburb, with its kerbside commons, that he asks for a diagram. Your correspondent tries to draw his own street, with large rectangles for houses, a line representing the kerb and small rectangles showing all the parked cars. The small rectangles take up a surprising amount of room.

Feedback Welcome


Arrr! . . . Avast!
Site Archive