From Go to Rust with an HTTP Server

One of the staples of the system developer's toolbox is building web applications. These days, we might even say it's building HTTP-based API servers.

In The Go Developer's Quickstart Guide to Rust I showed how a Go developer can get a foothold in the Rust world. This time, I'll start with a Go HTTP app and show how to implement it in Rust.

Along the way, we're going to learn quite a bit more about Rust. We'll get the basics of error handling, JSON parsing, and memory management. We'll see generics, closures, and references. And we'll get a deeper view of Rust's trait system.

As with the last post, I want to preface this one by saying that I am learning Rust as I write these posts. So I'll likely make a gaffe or two as we go.

Go + Gin is my Go-to

When it comes to building Go web servers, I long ago gave up starting with net/http. These days, I typically use Gin. I have found it to be the ideal balance between providing high-level libraries, but with access to do weird stuff when I need to.

In the last post we started with Go's official "Hello World" program. This time, let's start with Gin's example:

package main

import "github.com/gin-gonic/gin"

func main() {
    r := gin.Default()
    r.GET("/ping", func(c *gin.Context) {
        c.JSON(200, gin.H{
            "message": "pong",
        })
    })
    r.Run() // listen and serve on 0.0.0.0:8080
}

Essentially, if you send this server a request (curl localhost:8080/ping) it will respond with a short JSON body:

{ "message": "pong" }

Let's build a Rust server that can do the same. We're not going to try to make it equally as concise. Instead, we're going to dwell on the language details.

As Usual, TIMTOWTDI

When it comes to web frameworks in any language, There Is More Than One Way To Do It (TIMTOWTDI, for those perl monks among us). So the first thing to figure out is which library we should use.

For example, there is an interesting actor-based library called Actix that has some cool features. With it, we could definitely write a shorter version of the code here. But the hyper library is closer to what we want to look at today.

And, of course, a search of http on Crates.io turned up 400+ more options, ranging from low-level protocol implementations to middleware to frameworks.

Rust's core libraries are focused on the core language. Very few protocols are built-in, as they are in Go. In some ways, this has helped the Rust community evolve better libraries, avoiding some of the architectural difficulties that, for example, Go's net/http team has had to work through.

Let's see if we can rebuild that Gin app as a Hyper app.

Hyper

Before we get going with Hyper, I want to point out a philosophical difference between Hyper and Gin: Gin is designed to get you running with the fewest number of decisions possible. So it sets a lot of defaults for you. Hyper is closer in theory to Go's net/http plus some helpers. It does not set up many defaults. Consequently, our code will be more verbose.

We'll get started by doing a crates new --bin hello_http, and then adding hyper: "0.12" to the Cargo.toml. We will also include futures: "0.1" as well, since that's how Hyper exposes its asynchronous functionality. The Gin example sends a JSON response, so we'll want serde_json: "1.0" to do that.

[package]
name = "hello_http"
version = "0.1.0"
authors = ["Matt Butcher <mbutcher@example.com>"]

[dependencies]
hyper = "0.12"
futures = "0.1"
serde_json = "1.0"

Creating apps and installing packages is all stuff we covered in the last post. So feel free to dodge back over there if you need a quick refresher.

Next, we can start working on src/main.rs, the entry point to our app.

The Hyper Ping Server

This is going to be a little longer than our Gin example, but we're going to get a tour of several features of Rust as we go.

extern crate hyper;
extern crate futures;

#[macro_use]
extern crate serde_json;

use hyper::{Body, Response, Server, Method, StatusCode};
use hyper::service::service_fn_ok;

// This is here because we use map_error.
use futures::Future;

fn main() {

    let router = || {
        service_fn_ok(|req| {
            match(req.method(), req.uri().path()) {
                (&Method::GET, "/ping") => {
                    Response::new(
                        Body::from(
                            json!({"message": "pong"}).to_string()
                        )
                    )
                },
                (_, _) => {
                    let mut res = Response::new(Body::from("not found"));
                    *res.status_mut() = StatusCode::NOT_FOUND;
                    res
                }
            }
        })
    };

    // Setup and run the server
    let addr  = "127.0.0.1:8080".parse().unwrap();
    let server = Server::bind(&addr).serve(router);
    hyper::rt::run(server.map_err(|e| {
        eprintln!("server error: {}", e);
    }));
}

A goal of this post is to learn about Rust through this example, so after a high-level explanation, we'll take a closer look at some of the pieces of the code above.

At a high level, what we are doing here is implementing a server for HTTP requests. That server (which is router above) is providing a function to be run on each inbound request. And that function is using a match to route requests to responses.

Then, at the bottom, we are starting a new HTTP server, and sending it to be managed by the Hyper runtime.

Now let's look more closely at the parts of this code.

The Libraries We Use

The header of our new code looks like this:

extern crate hyper;
extern crate futures;

#[macro_use]
extern crate serde_json;

use hyper::{Body, Response, Server, Method, StatusCode};
use hyper::service::service_fn_ok;

// This is here because we use map_error.
use futures::Future;

We saw extern in the last post. We're using three libraries:

• hyper is the HTTP library we are using.
• futures provides a Futures implementation for Rust. If you've used JS Promises, this is conceptually the same.
• serde_json is the JSON library for SERialization/DEserialization (serde).

When we extern the serde_json crate, we annotate this with #[macro_use]. This tells Rust that we want to use the macros defined in this create. That's where we get the json! macro used later:

json!({"message": "pong"}).to_string()

We then use use to declare our library usage. Note that we use futures::Future even though we don't directly seem to use this library. The reason is because we do explicitly use map_err, which is implemented on the trait futures::Future, even though the implementor is in the hyper library.

You can think of this analogously to Go and interfaces: In Go, when you treat a particular struct as an implementation of an interface, you must import that interface:

buf = bytes.NewBuffer(nil)

// We must import "io" to treat this buffer as an io.Reader
r := buf.(io.Reader)

In our Rust code, we have:

server.map_err(/* ... */);

When Rust compiles, it does the type inference for us, and reads it as "Treat this hyper::server::Server as a futures::Future and use its map_error() implementation." And in rust, we must use futures::Future to help the compiler make that inference.

Okay, that's it for the headers of this file. Next, let's look at the router.

Using match to Make a Router

One of the things that the Gin Go framework provides is a router that can take a combination of an HTTP verb (GET, POST, PATCH, ...) and a path ("/foo"), and match these to a handler function.

Looking back at our Gin code, we see this in action:

r := gin.Default()
r.GET("/ping", func(c *gin.Context) {
    // Do something
})

Hyper doesn't provide such a high-level library. So instead of getting a default router, we create our own. Then we use pattern matching to handle particular routes.

Personally, I think Hyper's approach is really cool, and also much more flexible, even if it is a little more verbose. Why do I think the added flexibility is good? Because we could build matchers that account for things that Gin-like systems can't, like the presence of a header or a cookie field.

We're going to spend a few sections talking about this part of the code:

let router = || {
    service_fn_ok(|req| {
        match(req.method(), req.uri().path()) {
            (&Method::GET, "/ping") => {
                Response::new(
                    Body::from(
                        json!({"message": "pong"}).to_string()
                    )
                )
            },
            (_, _) => {
                let mut res = Response::new(Body::from("not found"));
                *res.status_mut() = StatusCode::NOT_FOUND;
                res
            }
        }
    })
};

We're going to break it down into these sections:

• The routing closure
• The service handler function
• The match
• The default route
• The /ping route

The Routing Closure

The first thing we do there is create a closure: let router = || { /* ... */ } Later, we'll pass this into the HTTP multiplexer. Basically, we're creating a function that will return a function that can be used to handle each HTTP request. Think of it as a factory function or constructor.

Because of Rust's scoping rules, it is often convenient to pass functions around much as you would do in functional languages. This gives us a nice clean way to manage memory without having to keep references in our top-level code.

The Service Handler Function

The router returns what Hyper calls a Service, which is a trait that knows how to handle a single HTTP request. Hyper provides a couple of simple factories for constructing a Service using just one important parameter: A function (closure) for handling the request. The two factories are service_fn_ok() and service_fn(). The first never returns an error (that is, we always handle the error locally). The second can optionally return an error.

You can think of Service and the service function as analogous to Go's net/http Handler and HandlerFunc. The service_fn_ok() function basically wraps a service (handler) function inside of a service (handler).

In our code, this happens here:

// We get a request (`req`) from the Service
service_fn_ok(|req| {
    // We return a response from here
})

With this pattern, just as with Go's HandlerFunc, all we have to worry about is handling the immediate request (req).

The Match

And now we're to the exciting part. Inside of our service function, we create a router-like match to handle the request:

match(req.method(), req.uri().path()) {
    (&Method::GET, "/ping") => {
        // Handle ping
    },
    (_, _) => {
        // Return a 404
    }
}

It is sometimes useful to think of a match as a highly capable Go switch. In this case, in the match clause we are defining a tuple of HTTP method and request URI path. So any matching item must match both method and path.

Then we define two matches.

The first matches an HTTP GET operation on the path /ping. It will answer a request like curl http://localhost:8080/ping.

The second is using two wildcards (_) to match ANY method and ANY URI. This is our default match, and we want it to return a 404 error for each thing it matches. Were we to add new routes, we'd want to put these new ones before the default route.

The Default Route

Let's look at the surprisingly rich default route. Then we'll backtrack to the ping route.

(_, _) => {
    let mut res = Response::new(Body::from("not found"));
    *res.status_mut() = StatusCode::NOT_FOUND;
    res
}

There is probably a more elegant way to do this little block, but I'm still learning Rust. Here's what I did: Each time the default route is hit, create a response that simply sets a plain text body (not found) and returns a 404 (StatusCode::NOT_FOUND).

But we can learn a couple of interesting things about Rust from these three lines of code.

First, we are going to construct a new response and set the body all in one go: Response::new(Body::from("not found")). But we have to modify this response, so we need to mark this response as mutable. That's why our declaration is let mut res, where mut tells Rust to allow us to modify the object.

This is a key difference between Rust and Go. Go makes all variables mutable (whether created with var foo or with foo := ...). There is, in fact, no way to prevent a variable from being modified in Go (though we can try to hide the variable to keep it from being modified).

Rust, in contrast, defaults all variables to immutable. You must declare (typically with mut) that you want to allow a variable to be modifiable after initialization.

We mark our response as mutable, and then a line later, we set the status code on that response:

*res.status_mut() = StatusCode::NOT_FOUND;

I have to be honest: I found that line in the Hyper documentation and had to stare at it for a minute before understanding how it worked. My first thought was "Wait... you can't assign a status code to a function's return value!" But actually, you can. And that's exactly what it is doing.

Rust provides references, which are similar to Go's pointers. What the line above is doing is dereferencing the return value of res.status_mut(), and then setting that value to a different code. You can think of the above line as saying "Tell me where you keep your status code, and I'm going to swap out your old code with my new 404 code."

Lastly, I added the res line (with no semi-colon) to return the Response object. Rust does not use an explicit return keyword; functions just return the last statement.

The /ping Route

Now we're up to the ping handler:

(&Method::GET, "/ping") => {
    Response::new(
        Body::from(
            json!({"message": "pong"}).to_string()
        )
    )
},

Again, this one matches any incoming HTTP request whose method is GET and whose URI is /ping.

I stretched this one out over three lines, but we already saw most of it: Response::new(Body::from(/* ... */)) just says "Create a new response, and set the body to a string".

My favorite part, though, is this line:

json!({"message": "pong"}).to_string()

This json!() macro is pretty cool. It allows you to pass JSON data directly into the macro, which will then transform it into the JSON library's Value. This is a really useful feature for testing, building simple responses, and illustrating stuff in overly long blog posts.

So the line above basically transforms a simple JSON doc into a Rust object, and then calls to_string() on it. That gets wrapped as the body of the response, and returned.

Starting the Server

At this point, we've walked through the request handling portion of the code, and we're up to the very last bit. We just need to create a new server and start it up.

let addr  = "127.0.0.1:8080".parse().unwrap();
let server = Server::bind(&addr).serve(router);
hyper::rt::run(server.map_err(|e| {
    eprintln!("server error: {}", e);
}));

There are a couple of interesting technical details in the code above, but in a nutshell, this is what we're doing:

1. Create an IP address and port from the string 127.0.0.1:8080
2. Create a new server, and tell it to use that address, and route using the router we created earlier in this post
3. Run the server, and if the server experiences a fatal error, print it out on STDERR

Parsing the Address (Yay, Turbofish!)

When we parse the address, we use a pattern that you are likely to see all over the Rust codebase:

"127.0.0.1:8080".parse().unwrap()

parse() here is a trait granted on a String object, but actually provided by multiple different parsers. In this case, it is provided by std::net::SocketAddr. In our particular context, Rust basically infers from our usage (when we pass &addr into Server::bind) that we want to use the SocketAddr implementation of parse(), and not the one (for example) that parses strings into integers.

The documentation for parse() suggests that we might consider taking some of the guesswork out of which parser Rust should use. It says:

Because parse is so general, it can cause problems with type inference. As such, parse is one of the few times you'll see the syntax affectionately known as the 'turbofish': ::<>. This helps the inference algorithm understand specifically which type you're trying to parse into.

So let's try out the turbofish:

let addr  = "127.0.0.1:8080".parse::<std::net::SocketAddr>().unwrap();
let server = Server::bind(&addr).serve(router);

(Note that we could have added a use line to avoid typing the fully qualified name std::net::SocketAddr)

In this case, we are telling parse to use the parser implementation found on std::net::SocketAddr. Again, we got away with not using this because our original code made it clear to the Rust compiler that there was only one parser that could satisfy the call to Server::bind().

Using unwrap() and Handling Errors

Once we parse the address, we call unwrap(). This is a common pattern in Rust. And it is deeply illustrative of a difference between how Rust handles errors, and how Go does.

In Go, we'd expect the parse function to look like this:

func (self string) Parse() (SocketAddr, error) {}

Then we use the everpresent if err != nil startegy:

if err != nil {
    // handle the error, or more frequently just...
    return err
}

Rust, in contrast, has some standard conventions and tooling to more elegantly handle errors.

In Rust, we can think of the String::parse function like this (somewhat simplified):

fn parse(&self) -> Result<T, Err> {}

The error type is slightly more complicated, but understanding that is not necessary here.

To understand what's going on, there are two basic differences between Go and Rust that come into play here:

1. Rust does not allow multiple return values, while Go does
2. Rust does have generics, while Go does not

Go's error handling pattern is usually explained as "return the real value plus an error, and let a higher-level conditional sort things out".

Rust's most prominent error handling pattern is "wrap the result and the error in a Result type, and add some utility functions to help sort things out"

To get the value back out of the Result, we call unwrap(), which will return the value if no error occurred, or will panic. For us, we desire a panic because we hard-coded the address in. But there are all kinds of other ways we could handle that Result.

For example, if we wanted to parse an address, but start on a default address/port if an error occurred, we could do this:

addr.parse().unwrap_or(([127, 0, 0, 1], 3000).into());

That's more compact than the Go equivalent.

Sharing A Reference

There is one last thing to note about the code to start the server, and it occurs here:

Server::bind(&addr).serve(router);

Why add the & before addr? Similarly to Go, we can use the & to pass a reference to the addr object. But the idea in Rust is that we let the bind() function use our address value, but not own it. This means two things:

• When bind cleans up its values, it will not destroy addr.
• More importantly, bind can read the address, but cannot alter it. addr is immutable to bind().

Rust has a sophisticated memory model that is explained well in the official docs. Rust docs also provide lots and lots of examples.

Conclusion

The main point of this article was to show how to write a web service in Rust. But we weren't optimizing for speed or code length. We were optimizing for exposure to more of the Rust language. So along the way we saw:

• External library usage
• Traits
• Closures
• match patterns
• Error handling
• Generics
• Memory management

There's still more to Rust, though with these basics you can definitely build programs.

• go
• golang
• rust
• web