I like to virtualize my workstation using VMWare Workstation. Lately, my Ubuntu (kubuntu) 12.04 guest would exhibit really annoying behavior whereby it would insert lots of extra letters as I typed, seemingly at random.
So, I really like Ubuntu. Its Linux and it just mostly works. Except when they try to force everyone into some experimental new desktop environment. That is pretty awful, but I’m happy again now that I switched to kubuntu-desktop. (apt-get install kubuntu-desktop)
Kubuntu is Ubuntu with a nicely set-up KDE environment. They try to get you to use their own home-grown web browser, and the file manager takes some getting used to, but you can pretty quickly get under the hood, set up all your little window manager preferences, and get back to jamming. (Focus Follows Mouse in my house!)
The only thing that was missing is the fonts were rendering . . . not as pretty as regular Ubuntu. Kubuntu is set up to use the Ubuntu font, but in KDE things render kind of pixelly looking, like I was still in the 90s. A bit of searching and they seem to look nicer:
System Settings > Application Appearance > Fonts
Use anti-aliasing: Enabled
Configure…
Use sub-pixel rendering: RGB
Hinting style: Slight
By the way, let’s say you have to print, sign, scan, and email back a document using the flatbed scanner attached to your Linux workstation. No trouble scanning in each page as an individual PDF document using XSane, but assembling the multipage project into one document proved a modest challenge last time I tried.
Solution? Install pdftk, and then the command goes something like:
pdftk AVID000*.pdf cat output AVID.pdf
That’s will merge all the documents named AVID0001.pdf, AVID0002.pdf, etc into AVID.pdf. Or to put it another way:
pdftk <a bunch of PDF files> cat output <target PDF file>
I like Ubuntu. Or at least I liked it a few years back when you got a very nice functional desktop out of the box … but that is a different gripe.
I really like an OS that updates the software for me. Really, downloading and installing updates is for chumps! Way to go, Ubuntu!
But here’s a feature that has been bugging me for years: the system pops up a window saying “hey, I’m going to update the system software for you.”
And I’m like “sure, go ahead, be my guest!”
Then it’s like “okay, please give me the administrator password.”
And I’m like “well, okay . . . but . . .”
Each time the computer pops up a window unbidden offering to do me a favor in exchange for my password, I am wondering when some bastard will get around to writing a bit of malware that offers to do something nice for me in exchange for my password. Maybe a web site can launch a convincing-looking software-update window and prompt me for a password. It is going to be epic just how many users can be convinced to type their system passwords into a malware site.
If you are a part of the system that has access to do heavy lifting, please do not approach the user asking for a password. It teaches the user that “hey, its normal for your computer to pop up some window and ask for a password and when that happens you should totally humor the computer and give it your password.” You want to update my software? Great! You want to check with the person using the computer to make sure its a good time? Great, ask away, tell them your plans. But when they say yes, just fricking do it, and don’t ask them for their password.
I have had bad luck trying to coax this out of Google, so here’s a Perl one-liner:
perl -pi -e 's/[\x80-\xEF]//g' file.txt
Where file.txt is a file you want to clean up.
Why this comes up is because we have a web application that was set up to hit a MySQL database, which is incorrectly configured to store text as ASCII instead of UTF-8. The application assumes that all text is Unicode and that the database is correctly configured, and every week or two someone asks me why they are getting this weird gnarly error. Typically they are pasting in some weird UTF-8 whitespace character sent to us from Ukraine.
Eventually the database will be reloaded as UTF-8 and the problem will be solved. Until then, I can tell folks to use the Perl command above. It just looks for anything with the high bit set and strips it out.
Terminal windows resize themselves stupidly when changing font size. I don’t know how to fix that, but in the CompizConfig Settings Manager, I can enable Resize Info to overlay the dimensions of any window as I resize it.
Often, when dragging a window around, it tries to go full-screen on me. This is obnoxious! Just disable Grid in CCSM.
If anyone knows how to reconcile Focus Follows Mouse with “menu bar at the top of the screen” I would love to hear it! Or if you know how to configure the pager to something besides 2×2 …
While diagnosing why an internal web site is slow for some users, I got data that our overseas colleagues see ping latency to the web site of around 200 ms. This is not unreasonable. Some web sites that attach a lot of additional objects cause remote clients to have to open several connections and make several round-trips to load and render a web page. What might work fine at 20 ms latency can really drag at 200 ms.
How to test this out? As a Linux user, I can use netem to induce added latency on my network interface:
Google Chrome's Developer Tool Network view without extra latency.
Google Chrome's Developer Tool Network view with extra 200ms latency.
So, with my web site, an added 200 ms latency doubles total page load time from 0.8 seconds to 1.6 seconds.
Here’s what I see when I visit the problem web site:
A problem web site at normal latency.
A problem web site at additional 200ms latency.
Total page load time at 7.5 seconds is nearly three times slower than without latency.
A very crude way to measure things is with wget on the command-line.
The wget man page mentions the -p (page requisites) option, then the author suggests wget -E -H -k -K -p <URL>. (You’ll need to RTFM yourself…) So I do:
$ cd /tmp
$ sudo tc qdisc change dev wlan0 root netem delay 0ms
$ time wget -q -E -H -k -K -p http://google.com
real 0m0.160s
user 0m0.010s
sys 0m0.000s
$ sudo tc qdisc change dev wlan0 root netem delay 200ms
$ time wget -q -E -H -k -K -p http://google.com
real 0m3.832s
user 0m0.010s
sys 0m0.000s
Of course, even with all those options, wget behaves very differently from a modern GUI web browser: there’s no caching, it doesn’t parse the DOM and it will blindly download requisites it doesn’t actually need. (Even a large font file found in a CSS comment.) And it does all its requests serially, whereas a modern GUI web browser will fetch several objects in parallel. And whatever web browser you use over a connection with induced latency is not going to replicate the experience of remote users pulling page requisites from zippy local CDNs.
At the end of the day, I proposed the following advice to my remote colleagues:
Try tuning the web browser to use more concurrent TCP connections.
Try modifying browser behavior: middle-click faster-loading sub-pages in to new tabs, work on those tabs, then refer back to the slower-loading “dashboard” screen, reloading only when needed.
I also tweaked the web application to make it possible to show a more lightweight “dashboard” screen with fewer objects hanging off of it. This seems to improve load time on that page about 50%.
If you have a self-hosted WordPress blog, you really ought to keep it up to date. Popular software is a popular security target, and as new exploits are discovered, new patches are deployed. Fortunately, WordPress makes this super-easy. Just go to Dashboard > Updates and you can update with one click. I basically get a free update any time I get it in my head to write something.
Except this morning, when I was told that an update was available, but:
On my CentOS VM, this was addressed by:
sudo yum update
sudo service httpd restart
Actually, it was a little difficult, because we’re replacing php with php53:
0-13:11 djh@www0 ~$ cat /etc/redhat-release
CentOS release 5.6 (Final)
0-13:11 djh@www0 ~$ rpm -q php
php-5.1.6-27.el5_5.3
0-13:11 djh@www0 ~$ yum list installed | grep ^php
php.x86_64 5.1.6-27.el5_5.3 installed
php-cli.x86_64 5.1.6-27.el5_5.3 installed
php-common.x86_64 5.1.6-27.el5_5.3 installed
php-gd.x86_64 5.1.6-27.el5_5.3 installed
php-mysql.x86_64 5.1.6-27.el5_5.3 installed
php-pdo.x86_64 5.1.6-27.el5_5.3 installed
1-13:11 djh@www0 ~$ sudo service httpd stop
Stopping httpd: [ OK ]
0-13:11 djh@www0 ~$ yum list installed | grep ^php | awk '{print $1}'
php.x86_64
php-cli.x86_64
php-common.x86_64
php-gd.x86_64
php-mysql.x86_64
php-pdo.x86_64
0-13:12 djh@www0 ~$ sudo yum remove `!!`sudo yum remove `yum list installed | grep ^php | awk '{print $1}'`
Loaded plugins: fastestmirror
Setting up Remove Process
Resolving Dependencies
--> Running transaction check
---> Package php.x86_64 0:5.1.6-27.el5_5.3 set to be erased
---> Package php-cli.x86_64 0:5.1.6-27.el5_5.3 set to be erased
---> Package php-common.x86_64 0:5.1.6-27.el5_5.3 set to be erased
---> Package php-gd.x86_64 0:5.1.6-27.el5_5.3 set to be erased
---> Package php-mysql.x86_64 0:5.1.6-27.el5_5.3 set to be erased
---> Package php-pdo.x86_64 0:5.1.6-27.el5_5.3 set to be erased
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Removing:
php x86_64 5.1.6-27.el5_5.3 installed 6.2 M
php-cli x86_64 5.1.6-27.el5_5.3 installed 5.3 M
php-common x86_64 5.1.6-27.el5_5.3 installed 397 k
php-gd x86_64 5.1.6-27.el5_5.3 installed 333 k
php-mysql x86_64 5.1.6-27.el5_5.3 installed 196 k
php-pdo x86_64 5.1.6-27.el5_5.3 installed 114 k
Transaction Summary
================================================================================
Remove 6 Package(s)
Reinstall 0 Package(s)
Downgrade 0 Package(s)
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Erasing : php-gd 1/6
Erasing : php 2/6
Erasing : php-mysql 3/6
Erasing : php-cli 4/6
Erasing : php-common 5/6
warning: /etc/php.ini saved as /etc/php.ini.rpmsave
Erasing : php-pdo 6/6
Removed:
php.x86_64 0:5.1.6-27.el5_5.3 php-cli.x86_64 0:5.1.6-27.el5_5.3
php-common.x86_64 0:5.1.6-27.el5_5.3 php-gd.x86_64 0:5.1.6-27.el5_5.3
php-mysql.x86_64 0:5.1.6-27.el5_5.3 php-pdo.x86_64 0:5.1.6-27.el5_5.3
Complete!
0-13:13 djh@www0 ~$ sudo yum install php53 php53-mysql php53-gd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: yum.singlehop.com
* epel: mirror.steadfast.net
* extras: mirror.fdcservers.net
* updates: mirror.sanctuaryhost.com
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package php53.x86_64 0:5.3.3-1.el5_6.1 set to be updated
--> Processing Dependency: php53-cli = 5.3.3-1.el5_6.1 for package: php53
--> Processing Dependency: php53-common = 5.3.3-1.el5_6.1 for package: php53
---> Package php53-gd.x86_64 0:5.3.3-1.el5_6.1 set to be updated
--> Processing Dependency: libXpm.so.4()(64bit) for package: php53-gd
---> Package php53-mysql.x86_64 0:5.3.3-1.el5_6.1 set to be updated
--> Processing Dependency: php53-pdo for package: php53-mysql
--> Running transaction check
---> Package libXpm.x86_64 0:3.5.5-3 set to be updated
---> Package php53-cli.x86_64 0:5.3.3-1.el5_6.1 set to be updated
---> Package php53-common.x86_64 0:5.3.3-1.el5_6.1 set to be updated
---> Package php53-pdo.x86_64 0:5.3.3-1.el5_6.1 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
php53 x86_64 5.3.3-1.el5_6.1 updates 1.3 M
php53-gd x86_64 5.3.3-1.el5_6.1 updates 109 k
php53-mysql x86_64 5.3.3-1.el5_6.1 updates 92 k
Installing for dependencies:
libXpm x86_64 3.5.5-3 base 44 k
php53-cli x86_64 5.3.3-1.el5_6.1 updates 2.4 M
php53-common x86_64 5.3.3-1.el5_6.1 updates 605 k
php53-pdo x86_64 5.3.3-1.el5_6.1 updates 67 k
Transaction Summary
================================================================================
Install 7 Package(s)
Upgrade 0 Package(s)
Total download size: 4.6 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): libXpm-3.5.5-3.x86_64.rpm | 44 kB 00:00
(2/7): php53-pdo-5.3.3-1.el5_6.1.x86_64.rpm | 67 kB 00:00
(3/7): php53-mysql-5.3.3-1.el5_6.1.x86_64.rpm | 92 kB 00:00
(4/7): php53-gd-5.3.3-1.el5_6.1.x86_64.rpm | 109 kB 00:00
(5/7): php53-common-5.3.3-1.el5_6.1.x86_64.rpm | 605 kB 00:00
(6/7): php53-5.3.3-1.el5_6.1.x86_64.rpm | 1.3 MB 00:00
(7/7): php53-cli-5.3.3-1.el5_6.1.x86_64.rpm | 2.4 MB 00:00
--------------------------------------------------------------------------------
Total 12 MB/s | 4.6 MB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : php53-common 1/7
Installing : php53-pdo 2/7
Installing : php53-cli 3/7
Installing : libXpm 4/7
Installing : php53 5/7
Installing : php53-mysql 6/7
Installing : php53-gd 7/7
Installed:
php53.x86_64 0:5.3.3-1.el5_6.1 php53-gd.x86_64 0:5.3.3-1.el5_6.1
php53-mysql.x86_64 0:5.3.3-1.el5_6.1
Dependency Installed:
libXpm.x86_64 0:3.5.5-3 php53-cli.x86_64 0:5.3.3-1.el5_6.1
php53-common.x86_64 0:5.3.3-1.el5_6.1 php53-pdo.x86_64 0:5.3.3-1.el5_6.1
Complete!
0-13:14 djh@www0 ~$ sudo service httpd start
Starting httpd: [ OK ]
And now I have successfully upgraded via the web UI.
Most days, I am not a CentOS admin, so if there is a better way to have done this, I am keen to hear.
My employer (Cisco) makes IPv6 available internally on a test basis. Once I configured the isatap hostname on my system, all I had to do to get my Ubuntu laptop on IPv6 was:
sudo apt-get install isatapd
I look forward to trying this on a few more systems:
I’ll give Amazon.com credit for making their mp3 downloader available to Linux users:
I clicked on Ubuntu, and Chrome downloaded amazonmp3.deb and the Ubuntu Software Center fired up and told me:
Wrong architecture ‘i386’
That was frustrating. I sent a note to Amazon.com thanking them for their Linux support and asking them to please consider rolling some x86_64 packages. Then I asked Google for advice, and got this fine post:
(Your browser should hand the package off to an installer, else you’ll just do something like sudo dpkg -i Downloads/getlibs-all.deb.)
3) Run getlibs!
Looks something like this:
0-13:08 ~$ sudo getlibs /usr/bin/amazonmp3
libglademm-2.4.so.1: libglademm-2.4-1c2a
libgtkmm-2.4.so.1: libgtkmm-2.4-1c2a
libgiomm-2.4.so.1: libglibmm-2.4-1c2a
libgdkmm-2.4.so.1: libgtkmm-2.4-1c2a
libatkmm-1.6.so.1: libgtkmm-2.4-1c2a
libpangomm-1.4.so.1: libpangomm-1.4-1
libcairomm-1.0.so.1: libcairomm-1.0-1
libglibmm-2.4.so.1: libglibmm-2.4-1c2a
No match for libboost_filesystem-gcc42-1_34_1.so.1.34.1
No match for libboost_regex-gcc42-1_34_1.so.1.34.1
No match for libboost_date_time-gcc42-1_34_1.so.1.34.1
No match for libboost_signals-gcc42-1_34_1.so.1.34.1
No match for libboost_iostreams-gcc42-1_34_1.so.1.34.1
No match for libboost_thread-gcc42-mt-1_34_1.so.1.34.1
The following i386 packages will be installed:
libcairomm-1.0-1
libglademm-2.4-1c2a
libglibmm-2.4-1c2a
libgtkmm-2.4-1c2a
libpangomm-1.4-1
Continue [Y/n]?
Downloading ...
Installing libraries ...
3.1) If, like me, you got “no match for libboost” as above, or you get “amazonmp3: error while loading shared libraries: libboost_filesystem-gcc42-1_34_1.so.1.34.1: cannot open shared object file: No such file or directory” then do this bit:
Old SysAdmin tip: keep your frequent-but-long-running cron jobs from running concurrently by adding some lightweight file locking to your cron entry. For example, if you have:
Read up on the lockf or flock man pages before you go putting this in. This can be a bit tricky because these can also be system calls. Try “man 1 lockf” or the like to nail it down to the manual for the user-executable command.
I have run in to this a zillion times. You SSH to a Unix server, type your password, and then wait a minute or two before you get the initial shell prompt, after which everything is reasonably zippy.
The short answer is “probably, something is wrong with DNS . . . your server is trying to look up your client but it can not, so it sits there for a couple of minutes until it times out, and then it lets you in.”
Yesterday I was working with an artist who had a hosting account, and when he got in, I said:
sudo vim /etc/resolv.conf
He admitted that he had just copied the DNS configuration from his previous server. How to fix this? Well, he could check what nameservers are provided by his current hosting company . . . . or, I changed his file to read:
A while back I outfitted my personal workstation with 4GB of RAM. That’s plenty of memory and since disk space was tight I didn’t want to waste any on a swap partition, and I lived happily without swap for a very long time until I began using more virtualization. (I freed up space on my physical desk top by migrating my work environment from a laptop to a virtual machine on my personal workstation.)
I wrote a script to add a “temporary” swap file on demand but what I wanted was a swap file at boot. That actually turns out to be pretty simple. In this case, I just prepare the swap file:
I recently migrated this web site to a virtual server at the Rackspace Cloud. I started with the cheapest, 256M slice, but after hitting some performance issues really quickly, I figured that the extra $10/month to grow the slice to 512M was worthwhile. Even so, I don’t think MySQL and Apache are tuned, out-of-the-box, for such “small” systems.
I am running WordPress, a PHP application, on Apache and MySQL, running on CentOS. I am using Munin to track the system health. Since MySQL performance tuning can be extremely complicated I have focussed my initial efforts on Apache.
So, the memory graph would ideally top out at 512MB, but in practice, the system and programs allocate excess memory they rarely touch, and that gets paged out to swap. It is when the swap activity gets high that system performance goes to heck. A spike in system load is a good symptom of possible performance issues.
When things went bad on Wednesday, I tuned Apache down somewhat from the defaults and then enabled the Apache status page, and told Munin to go ahead and graph that. I am new to Munin, but Slicehost has some excellent articles on installing and configuring it in my environment.
This morning Munin sent me e-mail that it was having trouble collecting data, so I took a look, and reset performance values for Apache based on the above graph:
Hopefully, this will hold. I would love to audit the system memory use in greater detail to allow Apache some greater flexibility, but in the interests of achieving a stable system quickly, this will hopefully prove to be a good strategy.
After some transient disk issues I was stuck with an Ubuntu VMWare image that was brain dead. I figured I would give the install CD a chance to fix but the Ubuntu install CD doesn’t have a fix-it option. (Really?!) Since the issue was somewhere in how Gnome and X were configured, I launched an xterm session and managed this command:
