Go modules - Practical Go Lessons https://www.practical-go-lessons.

com/chap-17-go-modules

Chapter 17: Go modules

1 What will you learn in this chapter?

• What is a Go module?

• What are a dependency and a dependency graph?

• The Go approach to version selection.

• How to use Semantic Versioning.

• How to perform basic go Modules operations

◦ upgrade all, upgrade one, downgrade one.

1.1 Technical concepts covered

• Dependency

• Version

• Semantic Versioning (SemVer)

• Module

2 Introduction
Modules have been introduced with version 1.11 of Go. Gophers can split their developments into separate code units that can be reused
across other projects.

3 Dependency
Developers reuse code that their peers developed to :

• Reduce the development time

◦ Somebody might have already developed some basic functionalities team. Why taking time to develop it yourself?

◦ What is your added value?

• Reduce the maintenance cost. [@hejderup2018software].

◦ The code that your team write needs to be maintained (ie. solving bugs, fixing vulnerabilities ...)

◦ The code written by a dynamic community might be efficiently maintained.

1 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

▪ An important criterion when choosing a dependency is to check the community’s dynamism around the project.

▪ A trendy project with many contributions might be “safer” than another maintained by a couple of developers.

A program may rely on ten other programs or libraries. The list of programs and libraries that a program uses is called its dependencies. We
say that a program is dependent on another piece of software.

Go gives you the ability to use code written by others with Go modules easily.

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

4 Definition of a Go module
A module is a group of packages (or a single package) that are (is) versioned. This group of go files forms together a module. Modules can
be dependant on other modules.

• A module is identified by a string which is called the “module path”.

• The requirements (if any) of the module are listed in a specific file.

◦ Go will use this file for each installation to build the module.

◦ The file is named go.mod.

• Go will also generate a file named go.sum. We will see later what it is.

A module is a set of packages with a go.mod file and a go.sum file

5 The go.mod file

The go.mod file has the following structure

module gitlab.com/maximilienandile/myAwesomeModule

go 1.15

require (
github.com/PuerkitoBio/goquery v1.6.1
github.com/aws/aws-sdk-go v1.36.31
)

• The first line gives the module path (it can be either a local path or a URI to a repository hosted on a version control system).

module is a reserved keyword. In the example, the module is hosted on gitlab.com on my account. The project is named

2 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

myAwesomeModule.

• The second line will give the version of Go used by the developer

• Then another section define the dependencies that are used by the module :

require (
DEPENDENCY_1_PATH VERSION_OF_DEPENDENCY_1
DEPENDENCY_2_PATH VERSION_OF_DEPENDENCY_2
//...
)

First, the module path and then the desired version :

github.com/PuerkitoBio/goquery v1.6.1

We are loading the module github.com/PuerkitoBio/goquery hosted on GitHub And we require version v1.6.1.

5.1 Initialization with the go command line (on an empty project)

The go.mod file can be created automatically with the go command line :

$ go mod init my/module/path

This will generate the following file :

module my/module/path

go 1.15

For the moment, our go.mod file does not contain any dependency.

We will list dependencies with the require keyword.

For instance :

module gitlab.com/maximilienandile/myawesomemodule

require (
github.com/go-redis/redis/v8 v8.4.10
)

It means that the module gitlab.com/maximilienandile/myawesomemodule require the module github.com/go-redis/redis/v8 at version
v8.4.10

5.2 Initialization on an existing project

You can generate the go.mod file on an existing project with the go command line. Let’s take an example.

We can create a file (main.go) that will import code from the repository gitlab.com/loir42/gomodule :

package main

import "gitlab.com/loir42/gomodule"

func main() {
gomodule.WhatTimeIsIt()
}

The module we have imported has just one exposed function: WhatTimeIsIt (in its package timer ) :

Tree view Package timer

Here is the file content of timer.go :

3 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

package gomodule

import "time"

func WhatTimeIsIt() string {

return time.Now().Format(time.RFC3339)
}

If you run the command go mod init a go.mod file will be created. It will not list your dependency yet:

module go_book/modules/app

Then if you build your project by typing on your terminal go build the go.mod file will be modified, and the dependency is added to the
require section :

module go_book/modules/app

require gitlab.com/loir402/gomodule v0.0.1

We did not specify anything for the required module version. Consequently, the build tool has retrieved the most recent tag from the remote
repository.

5.3 Exclusion
In the go.mod, you can explicitly exclude a version from your build :

exclude gitlab.com/loir402/bluesodium v2.0.1

Note that you can exclude more than one module-version pair :

exclude (
gitlab.com/loir402/bluesodium v2.0.1
gitlab.com/loir402/bluesodium v2.0.0
)

5.4 Replacement
It is possible to replace the code of a module with the code of another module with the directive “replace” :

replace (
gitlab.com/loir402/bluesodium v2.0.1 => gitlab.com/loir402/bluesodium2 v1.0.0
gitlab.com/loir402/corge => ./corgeforked
)

The replaced version is at the left of the arrow; the replacement is at the right.

The replacement module can be :

• Stored on a code sharing website (ex: Github, GitLab .…)

• Stored locally

Some important notes :

• The replacement module should have the same module directive (the first line of the go.mod file).

• Should the replacement specify a version? It depends on the location of the replacement :

◦ Distant (Github, Gitlab) : required

◦ Local: not required

• A specific version of a module can be replaced OR all versions can be replaced.

gitlab.com/loir402/corge => ./corgeforked

will replace all versions of gitlab.com/loir402/corge by a local version

gitlab.com/loir402/corge v0.1.0 => ./corgeforked

4 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

will replace only version v0.1.0 by the local code.

6 Vocabulary: API
API stands for Application Programming Interface. The most important letter is I. An API is an interface.

An interface is like a frontier between two things, a “shared boundary across which two or more separate components of a computer system
exchange information” (Wikipedia).

An interface in computer science is a way for two different things to communicate. So what is an Application Programming Interface? It’s a
set of constructs (constants, variables, functions ...) that are exposed to interact with a piece of software.

With this definition we can say that the go package fmt exposes an API to the go programmer to interact with it. Its API represents the set
of functions that you can use for instance Println . It also covers the set of exported identifiers of the package (constants, variables,
types).

We can also say that the Linux kernel is exposing an API to interact with him, for instance, the function bitmap_find_free_region will tell the
kernel to find a contiguous, aligned memory region.

=> Go Modules expose an API that is composed of all exported identifiers of the package(s) that the module is composed of.

7 Vocabulary: Version
Programs naturally evolve:

• Additional functionalities are added (or removed)

• Bugs are fixed (or created)

• Performance is improved (or decreased)

Developers will add revisions to the original program, making it different (for the better or, the worse). Sometimes, a module’s API will evolve:
for instance, a previously exported function is deleted.

Programs relying on these particular functions will break (because it’s no longer exported).

Developers need to have a way to designate precise versions of a piece of code."1

A version is an unique identifier that designate a program at a specific revision and point in time.

This unique identifier is generated based on a set of well-known and shared rules. The set of rules and the format of the identifier is called a
“versioning scheme”. There are several schemes available. Go use Semantic Versioning. We will detail it in the next section.

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

8 Vocabulary: tag, revision, prerelease, release

• Tagged : it means that a “tag” has been created with a Version Control system.

• A tag is a string (it’s name ) that designate a specific revision of a project maintained by a Version Control System.

◦ Ex : “v1.0.1” is a tag

◦ The string “v1.0.1” is the tag name

◦ It designates the code of a repository at a very specific revision.

◦ Note that a tag can be any string.

◦ "GoIsMyFavLanguage" is a valid tag.

• A tag can designate a released version or a pre-released version of the software

◦ A pre-release version (or release candidate) is considered to be ready. It is made available for last minutes tests.

◦ Yet, it is not considered as a stable release (or just “release”).

◦ Pre-release versions have specific tags with appended characters :

▪ For instance : 1.0.0-alpha, 1.0.0-alpha.1

5 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

▪ Each project has different conventions about that

• An untagged version is a specific state of the program at a given time.

◦ In the Git VCS, it’s a “commit”

◦ The Git VCS will identify each commit with a SHA1 checksum (ex : 409d7a2e37605dc5838794cebace764ea4022388)

9 Semantic Versioning
Semantic Versioning2 is a norm that Tom Preston-Werner wrote. Tom3. This specification defines the way version numbers are formed. It is
widely used in the developer community.

In this section, I will detail some important requirements of Semantic Versioning :

9.1 Version number format

• A version number must have the following format : X.Y.Z

Where :

the major version

the minor version

the patch version (bug fix)

How does it work? X, Y, and Z are positive numbers (without leading zeroes). Those numbers are incremented following a specific norm.

• When you create new features that breaks the existing API of your software, you increment the major version number.

◦ old version : 1.0.0 / new version : 2.0.0

• When you create new features or make performance improvement that do not break the existing API you increment the minor
version number.

◦ old version : 1.0.0 / new version 1.1.0

• When you fix a bug in your code, you just increment the patch version

◦ old version : 1.0.0 / new version 1.0.1

When you create a major version, you set to zero the minor and the patch version number. When you release a new feature, you set to zero
the patch version.

9.2 Major version 0

During the beginning phase of a project, you know that things will change; your public API might not be the same later. During this phase of
development, you can still create a version. The major number is then fixed to 0 but minor and patch version number can still be
incremented. Other developers will know when they integrate your code into their software that your software is not considered as stable
(meaning that the public API might change without notifications)

9.3 A tool to ensure stability

When you increment your major version number, you also communicate to others that you have developed changes that are not compatible
with the previous major version.

Let’s take an example of an API change. A module called gomodule expose a function to the outside world called WhatTimeIsIt :

package gomodule

import "time"

func WhatTimeIsIt() string {

return time.Now().Format(time.RFC3339)
}

The developer has released version 0.0.1 of this code. Later the same developer has received many complaints from others about the format
of time. Fellow developers want to be able to specify a format for the time. RFC3339 is cool, but they want to choose. A new release is
prepared :

6 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

package gomodule

import "time"

func WhatTimeIsIt(format string) string {

return time.Now().Format(format)
}

The signature of the function has changed. As a consequence, code that uses this function will break if they import this new version. That’s
why the developer has to create a new major version : 1.0.0.

9.4 Delete a version?

Once the version is created, we must not change the software. You cannot tag your software to a specific version, then delete the tag and
recreate it with the same version number.

10 Dependency Graph
Handling dependencies is a complicated task, and many approaches exist. But first, I want to define the concept of a dependency graph.

The dependency graph holds in a structured representation the dependencies and the sub-dependencies that are required for an
application to work [@li2003managing].

Dependency Graph[fig:Dependency-Graph]

The figure 1 reprensents the dependency graph of the myApp application. Each package is represented by a node (a circle). Nodes are also
called vertices or leaves. For instance myApp is a node.

An edge is a link between two nodes. Edges are represented with straight lines. In a dependency graph, edges have a direction represented
by an arrow. Direct edges represent a dependency.

• myApp directly depends on two packages: foo and bar

7 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

We say that foo and bar are descendants of myApp.

• The package foo does not depend on anything. Foo has no descendant.

• Bar depends on two other packages baz and qux.

• Qux requires corge to work

• Baz requires also corge to work

11 Dependency solving
The dependency graph represents the dependencies required, but at the end, we need a list of dependencies to install our package. To create
this final list, we need to respect the graph’s requirements. Dependency solving is the process of finding this final list.

In our previous example, the final list will be :

• Foo

• Bar

• Baz

• Qux

• Corge

Note that we only need to include Corge once, even if it is required two times (by Baz and Qux).

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

12 The problem of version selection

In the previous example, we did not consider any version number. In this section, we will add version numbers.

8 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Dependency Graph With versions[fig:Dependency-Graph-version]

In the figure 2 we have represented the version numbers :

• myApp v1.0.0 depends on foo v1.2.0 and on bar v1.3.8 ...

We see that the node corge has been split into two new nodes in the graph. That’s because baz and qux depends on corge, but the two
packages do not depend on exactly the same version.

Which version should we add to our dependency list?

• The v2.0.0 or the version 2.7.0 ?

• Both?

• Just the more recent one, which is in this case v2.7.0?

We will see how Go handle this choice in the next section.

13 The Go approach: Minimal Version Selection

Minimal Version Selection (or MVS) is a set of algorithms4 used under the hood by the go command line to :

• Generate the go.mod file (and the go.sum), that lists all dependencies used by a project.

• Update one or several dependencies

• Downgrade one or several dependencies

9 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

MVS has been theorized by Russ Cox (one of the language developers). Russ wanted to create a system that was “Understandable.
Predictable. Boring”[@minimal-version-cox].

In this section, we will detail how it works by focusing on the main operations we will do every day:

13.1 The two base rules

• Each module will give a list of module requirements

◦ Modules are identified by a module path

◦ Each module required to specify a minimal compatible version

◦ This list is in the go.mod file

• Each module should follow the “import compatibility rule”.

◦ Modules with the same module path should be backward compatible.

13.2 How to add a new dependency

When you add a new dependency to your project (by calling go get) it will :

• Download the module required

• Add the module required to your go.mod file

13.2.0.1 Which version is selected?

Go will choose by order of preference

1. The latest tagged stable release

2. The lastest tagged pre-release

3. The latest untagged version (latest known commit, also called latest pseudo-version)

13.3 Construction of the build list

• The build list is the list of modules necessary to build a Go Program.

• Each item of this list is composed of two things

◦ A module path identifies a module

◦ A revision identifier (which can be a tag or a commit id)

The steps required to create the build list for a given module are5 :

1. Initialize an empty list L

2. Take the list of modules required for the current module (go.mod)

3. For each module required

1. Get the list of modules required by this module (go.mod)

2. append those elements to the list L

3. repeat the operation for elements appended to the list

4. In the end, the list may contain multiple entries for the same module path.

1. If so, for each module path, keep the newest version.

To display the final build list of a module, you can type the command :

$ go list -m all

In the following figure, you can see a mindmap that represents requirements for a module that requires only gitlab.com/loir402
/bluesodium/v2 at version v2.1.1.

• We begin by listing the requirements of bluesodium at the requested version

• It requires goquery

10 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

◦ Which requires itself cascadia

◦ And net

▪ , which requires crypto, sys, and text.

▪ .…

Example of requirements for an application requiring bluesodium

In the end, we have the following list :

• github.com/PuerkitoBio/goquery v1.6.1

• github.com/andybalholm/cascadia v1.1.0

• gitlab.com/loir402/bluesodium/v2 v2.1.1

• golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2

• golang.org/x/net v0.0.0-20200202094626-16171245cfb2

• golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a

• golang.org/x/text v0.3.0

• golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01

The net module is required twice (in bold) :

• golang.org/x/net v0.0.0-20200202094626-16171245cfb2

• golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01

The version golang.org/x/net v0.0.0-20200202094626-16171245cfb2 is prefered because it’s the most recent.

13.4 How to upgrade a dependency to the latest minor or patch

The first time you add a dependency to your project, Go will download a specific revision :

• A tagged version or,

• A tagged prerelease or,

• A specific commit

When you want to upgrade a dependency to its next version, you can type :

go get -u gitlab.com/loir402/bluesodium

The

-u

flag will download the newer minor or patch6

13.5 Upgrade a dependency to a new major version

Your project uses the version v1.0.1 of the module gitlab.com/loir402/bluesodium .

The maintainer released a new major version: tags beginning by v2 will appear on the repository :

11 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Screenshot of tags on GitLab for the module gitlab.com/loir402/bluesodium

A major version is released on a module you use in your program. When you attempt to upgrade it with the command :

go get -u gitlab.com/loir402/bluesodium

It will not download the newest major version (v2.0.1). Why?

• Version 2 of the module have a different path

• This is the direct application of the “import compatibility” rule:

◦ The rule is: “Modules with the same module path should be backward compatible.”

◦ Version 2 of a module introduces breaking changes. Those changes will impact the users of the previous versions.

◦ Therefore, it should have a different module path

Let’s take a look at the go.mod file of the dependency :

module gitlab.com/loir402/bluesodium/v2

go 1.15

The module path has changed; it’s no longer gitlab.com/loir402/bluesodium but it’s gitlab.com/loir402/bluesodium/v2 .

When a module switch from v0 or v1 to v2, it should modify its path to comply with the import compatibility rule.

To specifically require the major version 2, you need to launch the command. :

go get -u gitlab.com/loir402/bluesodium/v2

13.6 Upgrade all modules to the latest version

• Each requirement is read “as if it required the latest module version”[@minimal-version-cox] when the build list is constructed.

Because of the import compatibility rule, no breaking changes should be introduced (this operation only requests new patches and minor
versions).

If new versions are found then, the build list is modified AND also the go.mod file.

Concretely to do so, you will type the following command :

$ go get -u ./...
go: github.com/andybalholm/cascadia upgrade => v1.2.0
go: golang.org/x/net upgrade => v0.0.0-20210119194325-5f4716e94777

12 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

The command will output the upgraded dependencies (here cascadia and net). Let’s take a look at the go.mod

module thisIsATest

go 1.15

require (
github.com/andybalholm/cascadia v1.2.0 // indirect
gitlab.com/loir402/bluesodium/v2 v2.1.1
golang.org/x/net v0.0.0-20210119194325-5f4716e94777 // indirect
)

Here you can see that the upgraded modules have been added to the go.mod and a comment “indirect” has been added. Why? Those lines
are added to ensure that we use those specific upgraded versions when constructing the build list. Otherwise, the build list will stay the same.

13.7 Upgrade one module to a specific newer version

In this case, we want to update only one particular module to its newest version. The algorithm used is the following

• A first build list is constructed as if no upgrades are made.

• A second one is constructed with the requested upgrade.

• Then the two lists are merged.

• If a module is listed twice in the list, then the newest version is selected.

Upgrade One module to a specific new version

13.8 Downgrade one module to a specific lower version

Downgrading a dependency is a common action. It might be necessary if one of the dependency used :

• Introduce a bug

• Introduce a security vulnerability

• Reduce the application performance

• and many other reasons

In that case, the developer needs to be able to use a previous version of the dependency.

Let’s say that we used module E at version v1.1.0 and we want to downgrade the version of E to v1.0.0. Go will take each requirement of the
application :

• The build list is constructed for that particular requirement

• If the build list contains a forbidden version of E (ie. E at version v1.1.0 or above)

13 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

◦ Then the version of that requirement is downgraded

◦ If the downgraded version still contains E at version v1.1.0 (or above) it is downgraded at the previous version

◦ The task is repeated until the forbidden versions are not in the build list.

This operation will potentially remove requirements that are no longer needed.

13.9 Impact of module exclusion

In the previous section, we have seen that the go.mod file can exclude one module at a particular version (with the “exclude” directive).

In that case, Go will find the next higher version (not excluded) of the module. It will search in the list of :

• Releases

• Prereleases

Note that pseudo-versions (commits) are excluded from the selection process.

14 Vocabulary: checksum
• This is a set of characters that are generated by an algorithm (hashing algorithm) that takes as input data (strings, files ...).

• One of the purposes of a checksum is to check the integrity of something quickly that was transmitted over a network.

◦ The transmission of data over a network is not 100% safe, and sometimes you can get a corrupted file (either due to the network
or due to a hacking attempt).

◦ If you compare the checksum that the author of the file generated with the one you generate yourself, you can be almost sure
that you have the same file.

◦ I say “almost” because the hashing algorithm you use can be too weak and generate identical checksum from different files. For
instance, the MD5 algorithm can generate the same checksum for two very different files. We call this hash collisions.

15 Vocabulary: hash vs encoding vs encryption

The hash is the string of characters produced by the hashing algorithm. Hashing is different from encryption and encoding.

• Hash

◦ A hashing algorithm outputs hashes of the same size.

◦ It takes as input a piece of data (file, string, zip, ...)

◦ Theoretically, you CANNOT go back to the hash’s original input.

• Encode

◦ This is the process of converting a piece of data from a format to another format.

◦ From the output, we CAN go back to the input.

• Encrypt

◦ This is the process of taking an input, generally called the plaintext, and convert it to a ciphertext (the output).

◦ The ciphertext is not understandable if you do not have the key.

◦ To get the plaintext from the ciphertext, you need to be authorized.

◦ To get authorized, you need to have the key.

◦ The key can be symmetric or asymmetric

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

16 The go.sum file

The go.sum file will contain cryptographic hashes of the module direct and indirect dependencies.7

14 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

16.1 A sample go.sum

Let’s first generate the go.mod file for the project myApp. In the terminal, we type :

$ cd myApp
$ go mod init

The go.mod file generated is :

module gitlab.com/loir402/myApp

It’s empty! That’s because go mod init do not fill the go.mod file with the required dependencies. To avoid doing it manually, you can run the
command

$ go install

This will update the go.mod file and create the go.sum file also :

// myApp/go.mod
module gitlab.com/loir402/myApp

require (
gitlab.com/loir402/bar v1.0.0
gitlab.com/loir402/foo v1.0.0
)

We have our two direct dependencies listed in the file: foo and bar. The tag that has been selected automatically is the most recent one:
v1.0.0 for both dependencies.

Go install has also generated the following go.sum file :

gitlab.com/loir402/bar v1.0.0 h1:l8z9pDvQfdWLOG4HNaEPHdd1FMaceFfIUv7nucKDr/E=

gitlab.com/loir402/bar v1.0.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=
gitlab.com/loir402/baz v1.0.0 h1:ptLfwX2qCoPihymPx027lWKNlbu/nwLDgLcfGybmC/c=
gitlab.com/loir402/baz v1.0.0/go.mod h1:uUDHCXWc4HmQdep9P0glAYFdIEcenfXwuHmBfAMaEgA=
gitlab.com/loir402/corge v1.0.0 h1:UrSyy1/ZAFz3280Blrrc37rx5TBLwNcJaXKhN358XO8=
gitlab.com/loir402/corge v1.0.0/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=
gitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=
gitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=
gitlab.com/loir402/qux v1.0.0 h1:B1efJPpCgzevbS5THHliTj1owKfOi0Yo7tIaAm65n6w=
gitlab.com/loir402/qux v1.0.0/go.mod h1:QexiArTQZcXRpFC3LLuGhk82aJoknf1n6c4WxlTeWdg=

16.2 Anatomy of the go.sum file :

The go.sum file list all the project’s dependencies, the direct ones and the indirect ones. One dependency = two lines in this file. Let’s focus
on the package foo :

gitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=

gitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=

The two lines have the following anatomy :

Go sum anatomy[fig:Go-sum-anatomy]

Go.sum will record the checksum of the module version used, but also the checksum of the go.mod file of the module; that’s why we have
two lines per module.

The hashing algorithm used is SHA256.

• The first checksum is the hash of all the files of the module.

15 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

• The second checksum is the hash of the go.mod file of the module.

The hash is then converted to base64. The h1 string is fixed. It means that the Hash1 function inside of the go library is used8

• The checksum is here to ensure that the downloaded versions of dependant modules are the same than the first download.

17 Example
To understand better how things work, I have created six projects hosted on GitLab :

• https://gitlab.com/loir402/myApp

• https://gitlab.com/loir402/foo

• https://gitlab.com/loir402/bar

• https://gitlab.com/loir402/baz

• https://gitlab.com/loir402/qux

• https://gitlab.com/loir402/corge

17.1 Project Setup

The first project myApp is our main project that has two direct dependencies : foo and bar. The other projects are indirect dependencies of
myApp.

Here is the code of myApp :

// modules/example/main.go
package main

import (
"fmt"

"gitlab.com/loir402/bar"
"gitlab.com/loir402/foo"
)

func main() {
fmt.Println(foo.Foo())
fmt.Println(bar.Bar())
}

In myApp we use the API of foo and bar. The package foo has no dependencies. Here is its code :

// foo/foo.go
package foo

func Foo() string {

return "Foo"
}

The package bar has two direct dependencies: baz and qux :

// bar/bar.go
package bar

import (
"fmt"

"gitlab.com/loir402/baz"
"gitlab.com/loir402/qux"
)

func Bar() string {

return fmt.Sprintf("Bar %s %s", baz.Baz(), qux.Qux())
}

And here is the package qux :

16 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

// qux/qux.go
package qux

import (
"fmt"

"gitlab.com/loir402/corge"
)

func Qux() string {

return fmt.Sprintf("Qux %s", corge.Corge())
}

The package baz :

// baz/baz.go
package baz

import (
"fmt"

"gitlab.com/loir402/corge"
)

func Baz() string {

return fmt.Sprintf("Baz %s", corge.Corge())
}

And finally our last package corge (which is a dependency of baz and qux) :

// corge/corge.go
package corge

func Corge() string {

return "Corge"
}

Note that this last one has no dependencies.

I have created for each one of those packages a version v1.0.0 on GitLab.

17.2 Upgrade all dependencies to the latest version

To update all the dependencies of your project to the latest version (minor versions and patches) you can run the following command :

$ go get -u ./...

Let’s test it!

We will create a patch for the corge module. If you remember the previous section(about semantic versioning), a patch does not modify the
module’s API (ie. everything that is exported by the module) :

// corge/corge.go
package corge

func Corge() string {

return fmt.Sprintf("Corge")
}

The signature is the same, but we just add a call to fmt.Sprintf .

On GitLab, I created a new tag to materialize that a new version is out! The tag is v1.0.1 (the last digit was incremented).

Then I will run the command go get -u inside the myApp folder. myApp has corge as an indirect dependency :

$ go get -u ./...
go: finding gitlab.com/loir402/corge v1.0.1
go: downloading gitlab.com/loir402/corge v1.0.1

You see that Go has detected that we have a new patch for corge, and he has downloaded it. The go.sum file is now :

17 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

//...
gitlab.com/loir402/corge v1.0.0 h1:UrSyy1/ZAFz3280Blrrc37rx5TBLwNcJaXKhN358XO8=
gitlab.com/loir402/corge v1.0.0/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=
gitlab.com/loir402/corge v1.0.1 h1:F1IcYLNkWk/NiFtvOlFrgii2ixrTWg89QarFKWXPRrs=
gitlab.com/loir402/corge v1.0.1/go.mod h1:xitAqlOH/wLiaSvVxYYkgqaQApnaionLWyrUAj6l2h4=
//...

and the go.mod file is :

module gitlab.com/loir402/myApp

require (
gitlab.com/loir402/bar v1.0.0
gitlab.com/loir402/corge v1.0.1 // indirect
gitlab.com/loir402/foo v1.0.0
)

Let’s note that :

• The previous version (v1.0.0) of corge is still into the go.sum file.

• One line has been added to the go.mod file : “gitlab.com/loir402/corge v1.0.1”

The go.sum file keeps the old version for safety purpose because you might want to downgrade the dependency that you just upgraded
(because of a bug for instance). In that case, you want to be sure to roll back to the same version that worked before the upgrade.

17.3 Upgrade a dependency to the latest version

If you do not want to update every dependency, you can target a specific one with the go get command.

For instance, I have updated the code source of foo, and I have released a new version v1.0.1 (patch) :

// foo/foo.go
// v1.0.1
package foo

func Foo() string {

return fmt.Sprintf("Foo")
}

Then we can run the following command into the terminal (inside the myApp directory) to update only “foo” to the latest version :

$ go get gitlab.com/loir402/foo
go: finding gitlab.com/loir402/foo v1.0.1
go: downloading gitlab.com/loir402/foo v1.0.1

The go.mod file has been modified; foo is now required with the version v1.0.1 :

module gitlab.com/loir402/myApp

require (
gitlab.com/loir402/bar v1.0.0
gitlab.com/loir402/corge v1.0.1 // indirect
gitlab.com/loir402/foo v1.0.1
)

And the go.sum :

gitlab.com/loir402/foo v1.0.0 h1:sIEfKULMonD3L9COiI2GyGN7SdzXLw0rbT5lcW60t84=

gitlab.com/loir402/foo v1.0.0/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=
gitlab.com/loir402/foo v1.0.1 h1:6Dcvy69SCXzrGshVRDZzswqiA5Qm0n6Wt5VLOFtYF5o=
gitlab.com/loir402/foo v1.0.1/go.mod h1:+IP28RhAFM6FlBl5iSYCGAJoG5GtZpUH4Mteu0ekyDY=

The old version is still here (to downgrade safely), and the new version has been added.

17.4 Upgrade a dependency to a specific version

To target a specific version, you can also use the go get command :

18 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

$ go get module_path@X

Where X can be :

• A commit hash

◦ Ex : b822ebd
• A version

◦ v1.0.3

Go will fetch the requested revision of the module and install it locally. Let’s take an example. I have made an evolution to the bar module :

// bar/bar.go
package bar

import (
"fmt"

"gitlab.com/loir402/baz"
"gitlab.com/loir402/qux"
)

func Bar() string {

return fmt.Sprintf("Bar %s %s", baz.Baz(), qux.Qux())
}

func Bar2() string {

return fmt.Sprintf("Bar2 %s %s", baz.Baz(), qux.Qux())
}

I have added to the public API another exposed function Bar2. I have created a new minor version: v1.1.0.

Let’s update myApp to make it use this specific version :

$ go get gitlab.com/loir402/bar@v1.1.0
go: finding gitlab.com/loir402/bar v1.1.0
go: downloading gitlab.com/loir402/bar v1.1.0

Let’s check what’s changed into the go.mod :

module gitlab.com/loir402/myApp

require (
gitlab.com/loir402/bar v1.1.0
gitlab.com/loir402/corge v1.0.1 // indirect
gitlab.com/loir402/foo v1.0.1
)

The version of bar has been updated to v1.1.0 into the go.mod. Let’s check what’s changed into the go.sum file :

gitlab.com/loir402/bar v1.0.0 h1:l8z9pDvQfdWLOG4HNaEPHdd1FMaceFfIUv7nucKDr/E=

gitlab.com/loir402/bar v1.0.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=
gitlab.com/loir402/bar v1.1.0 h1:VntceKGOvGEiCGeyyaik5NwU+4APgyS86IZ5/hm6uEc=
gitlab.com/loir402/bar v1.1.0/go.mod h1:i/AbOUnjwb8HUqxgi4yndsuKTdcZ/ztfO7lLbu5P/2E=

The new version has been added to the list (the old version stays in the list)

17.5 Downgrade a dependency

In some cases, a new version that has been published is not completely stable and occurs bugs in your application. In that case, you might
want to downgrade it to the previous working version.

We will simply run the same command as upgrading :

$ go get gitlab.com/loir402/bar@v1.0.0

This will rollback the local version of bar to v1.0.0.

19 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

18 Cleanup task before a release

Before releasing your app, you want your go.mod and go.sum file to reflect exactly what you use. For the example, we will use the module
myApp. We will make it use a new dependency: grault. We will add it to the project then remove it. The objective is to see what happens to
our go.sum and go.mod file.

The source code of grault is quite the same as the other fake dependencies of our test. It exposes only a method Grault that send back the
string “Grault”. Nothing new here :

// grault/grault.go
// v1.0.0
package grault

import "fmt"

func Grault() string {

return fmt.Sprintf("Grault")
}

Let’s use it into myApp :

// myApp/main.go
package main

import (
//...
"gitlab.com/loir402/grault"
)

func main() {
//...
fmt.Println(grault.Grault())
}

When we run go install a new line has been added to the go.mod file and the go.sum file. Here is the go.mod :

module gitlab.com/loir402/myApp

require (
gitlab.com/loir402/bar v1.0.0
gitlab.com/loir402/corge v1.0.1 // indirect
gitlab.com/loir402/foo v1.0.1
gitlab.com/loir402/grault v1.0.0
)

Now let’s remove the usage of grault and launch again go install. The go.mod file and the go.sum stay the same, but we no longer use
grault. To clean up that we have to launch :

$ go mod tidy -v

unused gitlab.com/loir402/grault

This will:

• remove the mentions to grault in the go.mod and go.sum file

• remove unused versions of dependencies inside the go.sum file

When you delete in go.sum the lines that refer to older versions of your dependencies, you might not get the same version of the module
you used (when downgrading this module). When you downgrade, you want to be sure that the old version is the same as it was before. The
cryptographic checksum stored into the go.sum file is here for that specific purpose.

19 Where are downloaded versions of modules stored?

The different versions of the dependencies that have been downloaded by the go tool have been placed into the $GOPATH/pkg/mod
directory. In the next listing, I have represented the tree of the folder mod :

20 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Tree view of the cache folder

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

20 Is go.sum a lock file?

For gophers that have worked with other languages and their related dependency management system, the go.sum looks like a lock file. A
lock file lists the dependencies that your project use along with a specific revision (a tag or a commit sha1).

Here is an example of a lock file for the Nodejs language (with versioning tool npm) :

{
"name": "nodeLock",
"version": "1.0.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"moment": {
"version": "2.22.2",
"resolved": "https://registry.npmjs.org/moment/-/moment-2.22.2.tgz",
"integrity": "sha1-PCV/mDn8DpP/UxSWMiOeuQeD/2Y="
}
}
}

In this project, I used only one dependency called “moment” in version v2.22.2.

Here is another example of a lock file for a PHP project (using composer as dependency manager) :

21 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

{
//...
"packages": [
{
"name": "psr/log",
"version": "1.0.2",
"source": {
"type": "git",
"url": "https://github.com/php-fig/log.git",
"reference": "4ebe3a8bf773a19edfe0a84b6585ba3d401b724d"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/php-fig/log/zipball/4ebe3a8bf773a19edfe0a84b6585ba3d401b724d",
"reference": "4ebe3a8bf773a19edfe0a84b6585ba3d401b724d",
"shasum": ""
},
"require": {
"php": ">=5.3.0"
},
//....
}

How is it different from the go.sum file?

• The go.sum file stores the versions and the cryptographic sum of direct and indirect modules used by your application

◦ The aim of the go.sum file is to ensure that modules will not be altered at the next download.

◦ Whereas the goal of a lock file is to allow reproducible builds.

• Go use a deterministic approach to version selection.

◦ The build list will remain stable with time (if revisions used are not deleted by maintainer)

◦ It means that the build list generated in January will not be different from the one generated in December.

◦ Thus the introduction of a lock file is not necessary.

• Dependency management systems that have introduced lock files generally don’t have such a deterministic approach

• The lock file is needed to ensure that the builds of the application are reproducible by listing all the dependencies used and at which
version.

21 Go.sum & go.mod : commit them or not ?

You should commit your go.mod and go.sum in your VCS (for instance git), because :

• Others need the go.mod file to construct their build list.

• Others will use the go.sum to ensure that the module downloaded have not been altered.

22 Other commands to know

The go mod command has other interesting commands that you should know. This section will focus only on commands that I find
interesting.

22.1 go mod graph

go mod graph will output a dependency graph of your module to the standard output.

For instance, here is the graph of myApp :

gitlab.com/loir402/myApp gitlab.com/loir402/bar@v1.0.0
gitlab.com/loir402/myApp gitlab.com/loir402/corge@v1.0.1
gitlab.com/loir402/myApp gitlab.com/loir402/foo@v1.0.1
gitlab.com/loir402/bar@v1.0.0 gitlab.com/loir402/baz@v1.0.0
gitlab.com/loir402/bar@v1.0.0 gitlab.com/loir402/qux@v1.0.0
gitlab.com/loir402/qux@v1.0.0 gitlab.com/loir402/corge@v1.0.0
gitlab.com/loir402/baz@v1.0.0 gitlab.com/loir402/corge@v1.0.0

It’s not very visual, but it gives interesting information about myApp application. In the figure 4 I have represented it using arrows and circles.

22 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Each line of the output represents an edge between two packages. In this example, we have seven edges and six modules (foo, bar, baz, qux,
corge, and myApp). Corge appears two times because myApp depends on corge v1.0.1, whereas qux and baz depend on corge v1.0.0.

Go mod graph[fig:Go-mod-graph]

22.2 go mod vendor

The command go mod vendor will :

• create a vendor folder with all the sources of your dependencies.

Here is a tree view of the vendor folder of myApp :

23 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Tree view of vendor directory

• The vendor folder also contains a file : modules.txt that lists the dependencies path along with the version number.

# gitlab.com/loir402/bar v1.0.0
gitlab.com/loir402/bar
# gitlab.com/loir402/baz v1.0.0
gitlab.com/loir402/baz
# gitlab.com/loir402/corge v1.0.1
gitlab.com/loir402/corge
# gitlab.com/loir402/foo v1.0.1
gitlab.com/loir402/foo
# gitlab.com/loir402/qux v1.0.0
gitlab.com/loir402/qux

22.3 go mod verify

As the name indicates it go mod verify will check your locally stored dependencies. Go will check that your locally stored dependencies have
not been changed. This check is very useful to ensure that you are using the correct version of your dependencies and not an altered version.
Those alterations can cause builds to fail.

The command-line tool will check that for each module of the build list that the corresponding files downloaded located in pkg/mod/cache
/download are not altered. Here is how the foder pkg/mod/cache/download is structured

24 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Tree view of cache download directory

You can see that in this directory, Go has stored each version that we used for development. For each version, we have four different files :

• VERSION.info

• VERSION.mod

• VERSION.ziphash

• VERSION.zip

The info file contains the data at which it has been downloaded along with the version number :

{"Version":"v1.0.0","Time":"2018-11-03T19:36:07Z"}

The .mod file is the exact reproduction of the original .mod file of the module. The file .ziphash contains the hash of the .zip file.

I have tried to modify the zipped version of the module and then to run go mod verify. Here is the error message that is displayed by go :

gitlab.com/loir402/foo v1.0.1: zip has been modified (/Users/maximilienandile/go/pkg/mod/cache/download/gitlab.com/loir402

/foo/@v/v1.0.1.zip)

23 Test yourself
23.1 Questions
1. A minor version introduces breaking changes. True or False?

2. What is the command to update a module to its latest version?

3. What is the name of the set of algorithms used in Go to manage dependencies?

4. Write a sentence describing a Module with the following words: packages, source files, go.mod, go.sum.

5. What is the purpose of the go.mod file?

6. What is the purpose of the go.sum file?

7. What is the command to initialize a module?

8. What is the command to display the build list of a module?

9. When the major version 2 is released, the module path is not modified. True or False?

25 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

23.2 Answers
1. A minor version introduces breaking changes. True or False?

1. False

2. Major versions introduce breaking changes

2. What is the command to update a module to its latest version?

$ go get -u path/of/the/module

3. What is the name of the set of algorithms used in Go to manage dependencies?

1. MVS: Minimum Version Selection

4. Create a sentence describing a Module with the following words: packages, go.mod, go.sum.

1. A module is a group of packages and two files, a go.mod & a go.sum.

5. What is the purpose of the go.mod file?

1. The go.mod file will define the module path of the current module

2. The go.mod file lists minimum versions of direct dependencies.

3. It also lists the minimum versions of indirect dependencies

1. It happens when the developer upgraded or removed some dependencies manually

4. Each dependency is identified by a module path.

5. It also gives the expected language version for the module

6. What is the purpose of the go.sum file?

1. The go.sum file is to ensure that the source code of dependencies downloaded are the same as the one downloaded by the
original developer.
7. What is the command to initialize a module?

1. In your module directory :

$ go mod init path/of/the/module

8. What is the command to display the build list of a module?

1. In your module directory :

$ go list -m all

9. When the major version 2 is released, the module path is not modified. True or False?

1. False

2. Because a new major version introduces breaking changes, the module path should change (to respect the import compatibility
rule)

3. The string “v2” should be added to the module path

The paper and the digital edition of this book are available here. ×
I also filmed a video course to build a real world project with Go.

24 Key takeaways
• A Go module is a set of packages that are versioned together with a version control system (for instance, Git)

• Go modules are identified by a module path.

◦ Module paths describe what the module does and where we can find it
• A version is identified by a tag that describes the version changes.

• To describe what changes are added in a version, we usually use a versioning scheme which is a set of rules enforced by developers

• Semantic Versioning is a versioning scheme that Go uses

26 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

◦ In this scheme, a version number is a string formatted this way => “vX.Y.Z” (v is optional)

▪ X, Y, Z are unsigned integers.

▪ X is the major version number. When you increase this number, it means that you introduce breaking changes

▪ Y is the minor number. This number should be increased when new non-breaking features are added.

▪ Z is the patch number. This number is increased when a patch is created (a bug fix, for instance)

• When a module publish a new major version greater or equal than 2, it should add a major version suffix to the module path

◦ gitlab.com/loir402/bluesodium becomes gitlab.com/loir402/bluesodium/v2

• We can initialize a Go module in an existing project by executing the go mod init my/new/module/path command.

• To add a new direct dependency to a program, use the go get command:

$ go get my/new/module/to/add

• To upgrade all your dependencies, use the go get command:

$ go get -u ./...

• To upgrade one dependency, use the go get command:

$ go get -u the/module/path/you/want/to/upgrade

• In the go.mod file, you can replace the code of a module by another one (stored on a code-sharing website or locally)

replace gitlab.com/loir402/corge => ./corgeforked

• You can also exclude a specific version from your builds.

exclude gitlab.com/loir402/bluesodium v2.0.1

1. https://en.wikipedia.org/wiki/Software_versioning↩

2. https://semver.org/↩

3. Tom is one of the cofounders of Github↩

4. In this section, we will see how MVS works. It is a transcription of the excellent Russ Cox’s article [@minimal-version-cox].↩

5. Please note that the actual implementation of this algorithm is not as described. The actual implementation is based on graph traversal,
which is more efficient than this approach. Take a look at the Russ Cox article to know more about that!↩

6. Source: go help get↩

7. https://golang.org/ref/mod section go.sum↩

8. if you are curious, here is the source code that generated the checksum: https://github.com/golang/go/blob/master/src/cmd
/go/internal/dirhash/hash.go↩

Bibliography
• [hejderup2018software] Hejderup, Joseph, Arie van Deursen, and Georgios Gousios. 2018. “Software Ecosystem Call Graph for
Dependency Management.” In 2018 IEEE/ACM 40th International Conference on Software Engineering: New Ideas and Emerging
Technologies Results (ICSE-NIER), 101–4. IEEE.
• [li2003managing] Li, Bixin. 2003. “Managing Dependencies in Component-Based Systems Based on Matrix Model.” In Proc. Of Net.
Object. Days, 2003:22–25. Citeseer.
• [minimal-version-cox] Cox, Russ. 2018. “Minimal Version Selection.” https://research.swtch.com/vgo-mvs.pdf.
• [minimal-version-cox] Cox, Russ. 2018. “Minimal Version Selection.” https://research.swtch.com/vgo-mvs.pdf.
• [minimal-version-cox] Cox, Russ. 2018. “Minimal Version Selection.” https://research.swtch.com/vgo-mvs.pdf.

Previous Next

Interfaces Go Module Proxies

Table of contents

27 of 28 02/01/2023, 02:09
Go modules - Practical Go Lessons https://www.practical-go-lessons.com/chap-17-go-modules

Did you spot an error ? Want to give me feedback ? Here is the feedback page! ×

Newsletter:
Like what you read ? Subscribe to the newsletter.

I will keep you informed about the book updates.

@ my@email.com

Practical Go Lessons
By Maximilien Andile
Copyright (c) 2023
Follow me Contents
Posts
Book
Support the author Video Tutorial

About
The author
Legal Notice
Feedback
Buy paper or digital copy
Terms and Conditions

28 of 28 02/01/2023, 02:09