linux

This explains how to use a UNIX-like command line (including Linux and OS X) and the find command to search through a subdirectory and find all of the files with a certain extension, and then combine those all into one file. Surprisingly, this isn't a difficult task. It can be accomplished with one command on the command line:

$ find ./src -name '*.txt' -exec cat '{}' \; > test.txt

The above looks through everything in the ./src directory (including all subdirectories) for any files with the .txt extension. Each file it finds, it adds to test.txt. So at the end of the command's run, all of the text files will be combined together into text.txt. You can use this strategy to easily combine lots of files into one.

Using find, it's easy to customize the command above to do all kinds of things with files. I gave a few examples in an earlier post about the UNIX find command.

Often times, I want to create a full directory structure, and I'd like to do it with just one call to mkdir. That is, I want to create a root directory and multiple subdirectories all at once. Here's how to do this.

mkdir -p myProject/{src,doc,tools,db}

The above creates the top-level directory myProject, along with all of the subdirectories (myProject/src, myProject/doc, etc.). How does it work? There are two things of note about the command above:

• The -p flag: This tells mkdir to create any leading directories that do not already exist. Effectively, it makes sure that myProject gets created before creating myProject/src.
• The {} lists: The technical name for these is "brace expansion lists". Basically, the shell interprets this as a list of items that should be appended individually to the preceding path. Thus, a/{b,c} is expanded into a/b a/c.

You can nest brace expansion lists. That means you can create more complex sets of subdirectories like this:

mkdir -p myProject/{src,doc/{api,system},tools,db}

Notice that this creates two directories inside of doc/.

In Jauary we migrated a Drupal site that routinely has 40k+ hits per day. We moved the site from servers in the Pacific Northwest to a datacenter in Virginia. As if that wasn't enough, we moved the servers from Apache to Nginx, as well. But what makes this remarkable to me is that we managed to pull this off without so much as a minute of downtime. This blog explains how we did it (and it uses lots of pretty diagrams, too!).

Tuning Nginx ("engine-X") seems to be something of a black art. Today, I looked closely at the tcp_nopush, sendfile, and keepalive_requests settings for pages rendered from PHP as a FastCGI, and memcached content. We discovered that with a little careful tuning, we could shave off as much as 200-400 msec per request.

I have been working on several speed improvements on the Condition Centers at Spine-Health.com. Initially, these pages were taking upwards of 3.5 seconds just to render the HTML. Through a series of optimizations that I will document in another article, we have the conditions page rendering in around 100 msec now.

Before we get going, let me mention a few details of our system:

• We are running CentOS 5.3 (roughly equivalent to RHEL 5.3)
• We are running Nginx 0.6, which is behind the current stable, but is the latest in the Fedora EPEL repositories that we use.
• Since these settings make use of low-level kernel facilities (like TCP_UNCORK), other platforms may differ.

Nagios is a network monitoring tool. I use it to track web servers, mail servers, and whatever else I have running on the LAN and on the Internet.

One common configuration issue is getting the Service Commands menu to work correctly. By default, it is visible in the UI, but disabled on the server backend. And on Debian, not all of the steps to enable it are particularly evident from the docs. Often, one will recieve the cryptic error Could not stat() command file pointing to /var/lib/nagios3/rw/nagios.cmd. This can be fixed without too much fuss.
Nagios Service Commands

Today I needed to run a simple script against thousands of identically named files nested in a huge directory structure. What I needed was a quick way to recurse through all of the directories, ignore all of the files I didn't care about, but run a specified command on any files with a particular name.

While this sounds like the sort of thing that will require a couple dozen lines of shell scripting, it can actually be accomplished on a single line with the command find. find has been around for decades, and can be found on almost any UNIX-like file system. It is a simple to used for searching for files within the UNIX directory hierarchy. Honestly, there's not much more to it. Like all good UNIX tools, it does one thing well.