Why Glide 0.8 Is Our Biggest Release
The Go package management ecosystem has been missing something -- something most other language ecosystems have had for a long time. The new version of Glide brings it.
Historically, Go dependency management has shied away from a package management philosophy. Instead, two approaches have been advocated:
- Copy all your dependencies into your project and manually manage them.
- Use a tool that pins your dependencies to a particular VCS commit (e.g. a Git SHA).
(An earlier approach, "always run the most recent version" unsurprisingly seems to have fallen by the wayside.)
Until recently, Glide largely remained a tool to help with that second approach (and extend it just a bit to better accommodate developer workflow).
But we realized that we were doing it wrong. These approaches do not yield a satisfactory workflow for many (if not most) development organizations.
Scouting the Field
Glide grew out of real-world usage. Hitting the limits of tools like Godeps and GPM, we incremented our ideas, adding small workflow changes here and there. This model worked well until we hit that watershed moment when our issue queue was filled with contrasting requirements: More flexibility to specify versions, but rigid reproducibility of the build.
That was the moment when we realized that we had begun with the wrong set of assumptions. We assumed the two methods above (store it locally or pin to a particular VCS commit) were the correct options to select from.
So we paused and did something we should have done at the outset. We looked at what other languages were doing.
We read code, we asked questions, we read issue queues and tutorials and API docs. Among the core and frequent contributors, we evaluated Gems, Bundler, Mix, npm, CPAN, NuGet, Cargo, pip, Composer, PEAR/PECL, and various Java tools. We even blogged a little bit about what we were learning.
We came away with three major lessons:
- Dependencies should be expressible using semantic versioning. And (because Go devs aren't reliable about versioning) VCS markers like branches, commits, and tags.
- Builds must be reproducible.
- The
vendor/directory, added in Go 1.5, is the right place for all applications to manage their dependencies.
As we learned these lessons, we began to incrementally make changes. First, we supported GOVENDOREXPERIMENT within a day or so of Go 1.5's release. Then we added deep support for SemVer 2 semantic versioning.
But the big change is the one we've introduced in Glide 0.8: We have added a glide.lock file.
Lock It!
Previously, the glide.yaml file could be used either for the developer to specify the version ranges of various packages or a very specific set of VCS IDs. But most of us wanted both. We wanted to say "as a developer, I want to specify the version ranges my stuff depends on" while simultaneously saying, "as a release manager, I want absolute assurance of reproducible builds."
After much experimentation (including the ill-fated glide pin commands), we realized that the route that Gems/Bundler, Composer, npm, etc. was the right one. We introduced a lock file.
Now, the glide.yaml file is used for developers to track the direct dependencies of your codebase, and the glide.lock file is used to specify the exact version of all dependencies (immediate and indirect/transient) that a codebase uses.
And just in case there is any ambiguity, we intend that both should be checked into VCS.
Usage
With this new philosophy comes a slightly revised workflow.
Make locks
The glide get and glide up commands will both rebuild the glide.lock file. Use the first to add new things, and the second to update existing things (within the version boundaries you set in glide.yaml)
Use locks
The glide install command can now rebuild a project to the exact version that the lockfile specifies. And because there is no dependency/version resolution, it's blazingly fast.
So your CI/CD system should use glide install to prep the environment. Likewise, when developers first check out an existing project, glide install will get them to a known dependency state post haste.
From there, you only need to glide get or glide up when you are explicitly modifying your dependencies.
How to Write Glide Files
Given this new setup, developers only need to work with the glide.yaml file. And the dependencies specified there can just be your code's direct dependencies. (Though in all honesty we have found cases where we wanted to manage upstream dependencies, too).
And when it comes to the glide.lock file, you can stay completely hands-off. Glide manages that fully.
The March to 1.0 is ON!
The 0.8.0 release marks a huge transition for us. We feel like we've hit the last of our major planned features. From here, we're marching toward a 1.0 release.
That means that what you will see over the next months are changes to stability, speed, and code cleanliness. But the only features we'll be adding are minor administrative features.
And a fond farewell to...
The glide guess and glide pin commands have gone away, their functionality is now totally subsumed by glide up and glide get.
In the next release, glide tree will likely fade away, too. We just haven't found it to be crucial to the core functionality of Glide.
See you in the issue queue!
