system administration

Our current environment uses AWStats to analyze our HTTP server log files and build reports. Because it has privileged access to our data, and because it is open source, we can glean more information out of it than we could from proprietary hosted analytics platforms.

It turns out that there is a PHP front-end to AWStats (called BetterAWStats) that comes complete with a Drupal module. Here, I explain how we've installed and configured this module to get our AWStats data imported into our Atrium server.

I noticed something unexpected in my nginx logs today: There were a bunch of 499 HTTP codes in the access log. Oddly, these didn't show up in Google Analytics, there were no corresponding errors in the error log, but they did show up in my AWStats. What's the deal?nginx

The answer is pretty simple: Nginx uses 499 as the status code when the client unexpectedly terminates a connection. (Thus the client may have already received a 200 in the header, AFAIK). This is consistent with the usage of 4xx errors as indicating a client error condition.

A quick calculation showed me that the 499s accounted for only 0.2% of our total traffic. Not bad. And in fact, I sorta like the ability to see how many times clients terminated connections to my server.

Every once in a while, I have some project that requires working with one of GoDaddy's servers. By far, the biggest frustration for me when dealing with GoDaddy is getting MySQL databases uploaded. I've tried all kinds of crazy tricks, from exporting MySQL databases in "bite sized chunks" to writing SQL processors that break large imports into smaller ones.The _db_backup directory

But today I think I found the Right Solution (TM) to the problem: Use GoDaddy's database restoration tool to load a large SQL file. Basically, instead of treating this as an initial import, we treat it as a restoration from a backup file.

Here's how you do it.

In Jauary we migrated a Drupal site that routinely has 40k+ hits per day. We moved the site from servers in the Pacific Northwest to a datacenter in Virginia. As if that wasn't enough, we moved the servers from Apache to Nginx, as well. But what makes this remarkable to me is that we managed to pull this off without so much as a minute of downtime. This blog explains how we did it (and it uses lots of pretty diagrams, too!).

Tuning Nginx ("engine-X") seems to be something of a black art. Today, I looked closely at the tcp_nopush, sendfile, and keepalive_requests settings for pages rendered from PHP as a FastCGI, and memcached content. We discovered that with a little careful tuning, we could shave off as much as 200-400 msec per request.

I have been working on several speed improvements on the Condition Centers at Spine-Health.com. Initially, these pages were taking upwards of 3.5 seconds just to render the HTML. Through a series of optimizations that I will document in another article, we have the conditions page rendering in around 100 msec now.

Before we get going, let me mention a few details of our system:

• We are running CentOS 5.3 (roughly equivalent to RHEL 5.3)
• We are running Nginx 0.6, which is behind the current stable, but is the latest in the Fedora EPEL repositories that we use.
• Since these settings make use of low-level kernel facilities (like TCP_UNCORK), other platforms may differ.

The typical enterprise-grade publishing model for web sites typically has two or three tiers. Content is created and edited on a staging server, where it is carefully reviewed before publishing it to the live server. Additional development may even happen on a third ("development") server.