09 Apr

Code Has Two Audiences (Are You Alienating One?)

in programming

A programming language is a language designed to be read by both humans and machines. It is an intermediary that saves us (the programmers) from manually reducing all of our high-level designs to straight boolean logic. But it's also structured in such a way that current machine resources can unambiguously interpret it.

When you write a program, your code has two audiences. Humans and machines.

When we alienate one audience, the machines, the result is that the machine either does something wrong or cannot do anything at all. When a programmer makes this kind of error, we call it a bug. Bugs are bad.

But a mistake all too often made is for programmers to intentionally or unwittingly alienate the other audience. I recently inherited a codebase that had a single function whose body was 2, 996 lines long. It had high cyclomatic complexity (lots of control structures), hundreds of variables, and crazy indenting. For the most part, it worked fine. The computer understood it. But trying to add features and find and fix bugs has been a tremendous chore. This program was not written for two audiences. It was only written for one.

If I rewind the clock, I can remember numerous occasions where I wrote cryptic code, failed to document, and committed other such sins. And I can think of many times when I've returned to my own code weeks, months, or years later and have had to re-invest substantial time in figuring out what I wrote. My code was not written for two audiences.

Perhaps what we need is a new term. A counterpart to "bug" that describes the code's failure to remain semantically transparent to humans, and not just to computers. Because like computers, when programmers pick up a piece of code and find it indecipherable, they give up.

Unlike computers, they may just go build an alternative.

08 Apr

Node.js library for HP Cloud

in hpcloud, javascript, node.js, openstack

Today the HP Cloud Node.js JavaScript library was made publically available at GitHub. The new library provides Identity Service (aka Keystone) and Object Storage (Swift) libraries.

Along with writing the actual library, I've worked with it on projects, and it is stable and usable. The APIs are similar to the PHP library that we released last year.

To install:

    $ npm install hpcloud-js

And usage instructions can be found on the official site. We also have complete API documentation, thanks to Matt Farina.

18 Mar

Convert DOS to UNIX (and UNIX to DOS) in VIM

in programming, vim

Once upon a time I knew a vi mantra for removing carriage returns from a DOS-formatted text file, thus transforming it into a UNIX file.It was something like :%s/^V^M//g. It has become such an ingrained habit that I can still type it without even thinking about it. Muscle memory.

But VIM knows more about a file than that old clunky version of vi did, and there's an easier and more consistent way of converting between DOS and UNIX in VIM.

To convert from DOS to UNIX:

:e ++ff=dos
:setlocal ff=unix
:w

This basically forces the document into DOS mode, and then tells VIM to switch it to a UNIX file. (That closing :w just writes the change to disk.)

ff is VIM's abbreviation for fileformat. If you want to type that all out, you can. And ++? It tells VIM to effectively reopen the current file. So make sure you save your work before running these commands.

To convert from UNIX to DOS is even easier:

:e ++ff=dos
:w

That forces the file into DOS mode and then saves the change to disk.

Since both of these hook into a deeper level of VIM, they're not as prone to bugs and inconsistencies as relying upon a regular expression.

But wait... I need to do something more complicated...

The above is a great way to do simple conversions, but if your needs are more complex, you might want to take a look at the VIM wiki page in this topic.

06 Mar

How to use Ubuntu's Upstart to Control Node.js Forever

in linux, node.js, system administration, ubuntu

Recently I needed to start a server written in Node.js. This server was deployed on Ubuntu 12.04, and I wanted it to be started using the system's init system. Yet I also wanted the safety of Forever, a script manager for Node.js.

Ubuntu still supports the classic SysV init style, but it also now supports the far more sophisticated Upstart system. It is appealing for a few reasons: One is that it feels simpler and cleaner. Another is that it is more powerful and easily more configurable. I decided to use it for my scripts.

This article explains how to use Upstart, Forever, and Node.js together to run a Node.js server as a daemon process that is automatically started at system startup, and automatically stops at shutdown or reboot.

27 Feb

Installing GlusterFS on HP Cloud

in cloud, devops, gluster, hpcloud, openstack, system administration

Gluster is a distributed filesystem that works well in the cloud. This post explains how to configure GlusterFS on an Ubuntu 12.04 image running in HP's cloud.

Using this setup, I gain all the benefits of a distributed and replicated (redundant) filesystem for my in-cloud services, and can back these servers to persistent block storage if I want. It's a great way to gain stable networked filesystems in the cloud, and reduce or eliminate single points of failure.

04 Jan

PHP SPLObjectStorage v. Arrays: Redux

in benchmarks, php, programming

A few years ago, I tested SPLObjectStorage and Arrays to see which performed better. My conclusion at the time was that SPLObjectStorage is the better performer.

One thing I did not examine there was the implications of size. I ran tests with a fixed size of 10,000 elements. The test was also run on PHP 5.2.

A recent email from Frederik Krautwald caused me to re-evaluate these benchmarks. Using PHP 5.4.10 on Windows 7, Frederik generated this set of benchmarks:

18 Dec

It's always fun to see your project get news coverage

in hpcloud, programming, stackato

Forbes covered HP Cloud's announcement of the new Platform as a Service (PaaS) project.

This is my favorite quote from the article:

With this announcement HP’s Cloud grabs a potentially big technological advantage. Expect other cloud providers to do the same by quickly following HP’s lead of an integrated PaaS technology as key a differentiator. It’s also interesting to see that HP has selected ActiveState to form the core of its new PaaS offering. ActiveState’s Stackato is built atop VMware’s Cloudfoundry project, and is described as an application platform for creating your own private, secure, and flexible enterprise Platform-as-a-Service (PaaS) using any language on any stack on any cloud.

The same story was picked up by other outlets. My favorite "negative" take on it was posted on Diversity, Ltd . The author uses some loaded language (a poisoned chalice? Really?) to describe HP's partnership with ActiveState (Why? No idea). And he makes much of the fact that Stackato is a fork of Cloud Foundry. But I think his conclusion is right:

It’s not all negative for HP however. PaaS is, I believe, the future of the cloud and, more importantly, provides an opportunity for large traditional IT vendors like HP to bridge to offerings that will be compelling in the future world. Of course the fact that they can sell some cloud servers to run the new private PaaS offering doesn’t hurt either and I’m looking forward to seeing how seriously the company articulates the value of this product.

(As usual, I speak for myself, not my employer.)

17 Dec

Run Node.js apps on low ports without running as root

in node.js, programming, system administration

On Linux (and UNIX), to open a port with a number less than or equal to 1024, traditionally a program must run as the root user. This impacts web applications, which use ort 80 (HTTP) and port 443 (HTTPS) to do business. Many programs (Apache being a great example) use this by running a master process as the root user, and farming work off to helper processes that run with lower privileges. But Node.js is designed to work on a single-process model. So how do you run Node.js apps on low ports without running the script as root?

I looked at a number of solutions, and one suggested by my company's security team seems to be the best solution: For operating systems that support capabilities setting (like Ubuntu 12.04 and later), you can configure the operating system to allow non-privileged apps to listen on low ports. Here I show how to do it, and then briefly explain what is going on.

10 Dec

Using the Go Syntax for Janus VIM

in go, golang, janus, ubuntu, vim

To add the Go language (golang) plugins to Janus-flavored VIM, it's not enough to install the VIM plugins into your system's default location. So using, for example, Ubuntu's vim-syntax-go package will not work.

The best way to install all of the Go VIM plugins is to do the following:

  • Create a directory in your .janus directory for the go VIM plugin: mkdir ~/.janus/go
  • Download a fresh copy of the Go source: hg clone https://code.google.com/p/go/ (you may need to sudo apt-get install mercurial first)
  • Inside of the go/misc directory, find the vim directory
  • Copy the contents of the vim directory to the .janus/go directory: cp -a vim/* ~/.janus/go.
  • Restart VIM

This will give VIM access to all of the Go plugins. Syntax highlighting should immediately work on all *.go files.

26 Oct

PHP and curl_multi_exec

in curl, performance, php, programming

This post explain how to get data off of a curl_multi handle. Some time back I posted this snippet of code inside of a larger sample of code:

<?php
  $active = NULL;
  do {
    $ret = curl_multi_exec($multi, $active);
  } while ($ret == CURLM_CALL_MULTI_PERFORM);
 
  while ($active && $ret == CURLM_OK) {
    if (curl_multi_select($multi) != -1) {
      do {
         $mrc = curl_multi_exec($multi, $active);
      } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
  }
?>

I didn't really document or explain it. And so it seems that this code snippet has caused some confusion. Let me explain what it does.