The canonical name of a Go module is its path. For third party modules, that
means an URI. It is no secret that GitHub is currently the dominant code hosting
platform, hence the fact that most Go modules are named
github.com/<account>/<repository>. While I currently use GitHub for most of my
repositories, I also want the ability to move out of it in the future.
Fortunately the Go module system allows package authors to control how the
package is accessed. One can use it to provide Go modules with their own domain
and total control on module paths. Let us see how to do that.
How packages are downloaded
When the Go client downloads a module (either directly with GOPROXY=direct or
when a proxy does it), it fetches the module URI with an HTTP request and a
go-get=1 query parameter. The server includes a <meta> HTML element named
go-import in the response, its value indicating how to actually download the
module.
Let us check ourselves with my go-uuid module:
$ curl -s https://github.com/galdor/go-uuid\?go-get\=1 | grep go-import
<meta name="go-import" content="github.com/galdor/go-uuid git https://github.com/galdor/go-uuid.git">
GitHub indicates that to download the github.com/galdor/go-uuid module, the
client must clone the Git repository at https://github.com/galdor/go-uuid.git.
So for a module to be named example.com/foo, all you need is an HTTP server
responding to requests at https://example.com/foo?go-get=1.
Using GitHub Pages
I wanted all my packages to be available at go.n16f.net/<name>. It is short,
it looks good, and it opens the possibility to move the repositories out of
GitHub in the future.
While I could have used my own HTTP server, I was worried about what would
happen if someone needed to download a module when the server is down. Since I
already use GitHub to host most repositories, I decided to use GitHub Pages:
after all, what I need is a way to host HTML pages with the right <meta> tag
at the right URI.
Generating repository pages
I created a GitHub repository for these pages. It was tempting to use a simple shell script to generate pages but I really did not want to use M4 and Ruby lets me use ERB templates, so Ruby it is.
All my Go modules are listed in a text
file, and a
small Ruby
script
generates both an index (not required but why not?) and a page for each module.
The <meta> tag of each module page indicates that go.n16f.net/<name> can be
accessed by cloning the Git repository at https://github.com/galdor/go-<name>.
It is that simple.
Configuring the domain
Using your own domain means configuring it to point to GitHub pages.
First head to your domain name registrar website and configure your domain to
point to GitHub pages. In my case, that means creating a CNAME DNS record
named go.n16f.net. pointing to galdor.github.io.. You will notice that the
GitHub Pages address is global to your account and not specific to each
repository: GitHub takes care of routing, based on which domain is associated
with each repository.
Then go to the GitHub Pages settings page
and add your domain. You will have to create a TXT DNS record with a specific
name and value to prove ownership of the domain.
Once done, go to the GitHub Pages settings page of the repository. Select the “GitHub Actions” source, and add the domain to “Custom domain”. Make sure to tick “Enforce HTTPS” while you are here.
Publishing pages
To generate and publish pages, we use a small GitHub Actions
workflow.
When new commits are pushed to the master branch, we run the script (the
ubuntu-latest image already contains Ruby, so there is nothing to setup) and
use official GitHub actions to upload and deploy the pages we just generated.
A quick check shows that the new URIs work as expected:
$ curl -s https://go.n16f.net/uuid\?go-get\=1 | grep go-import
<meta name="go-import" content="go.n16f.net/uuid git https://github.com/galdor/go-uuid.git">
Updating modules
You can now update your Go modules: update the path in the go.mod file, and
all import paths in source files. Yes this is annoying, and I would love a way
to declare the path of each dependencies at project level, but
Sed makes the task trivial.
Updating dependencies is a bit trickier. If your project depends on
github.com/example/foo which you just renamed to example.com/foo, update all
module paths in source files then run:
GOPROXY=direct go get example.com/foo@latest && go mod tidy
This will force Go to fetch the module directly from its source and not from the global proxy which may not be up-to-date yet.
Result
For a surprisingly small amount of work, I now have all my Go modules available at a path I fully control and I can add new modules by modifying a text file and letting the GitHub workflow handles the rest. And while the current setup is still dependent on GitHub, I can move it somewhere else without any change to module paths.
What is your excuse for not doing the same thing?