Nicolas Martyanoff – Brain dump About

Gmail backups using ISync

Like so many people, I use Gmail. There are not a lot of options, and Google Workspace provides me additional useful services and lets me use my own domain name. However there has been a lot of horror stories about users losing access to their Gmail account without any recourse. It is not clear if it can happen with Google Workspace, but the risk is too high to ignore.

I already backup my passwords; now is the time to backup my emails. While Google Workspace offers an export system, I wanted something scriptable. I went with IMAP access since I already use it to read emails with Gnus.

Authentication

Gmail IMAP access requires an application password. You can refer to the Google help center for more information about how to create them.

We will store credentials in the semi-standard netrc file. For Gmail, add the following entry to ~/.netrc:

machine imap.gmail.com port 993 login <email-address> <password>

Make sure that your netrc file is only user readable. Its permissions should be set to 0600.

ISync

We will export emails using the synchronization features of ISync. Note that ISync is the name of the project while mbsync is the name of the executable.

The configuration of the program is stored in ~/.mbsyncrc and is based on two main concepts: stores and channels. Stores designate location containing emails; channels describe a link between two stores: the “far” one an the “near” one.

Let us first define the store for the Gmail account:

IMAPStore gmail
Host imap.gmail.com
Port 993
SSLType IMAPS
User <email-address>
PassCmd "perl -ne '/^machine imap\\.gmail\\.com .*?login <email-address> .*?password (\\S+)/ && print \"$1\\n\"' ~/.netrc"
PipelineDepth 1

While you can use simple use Pass <password>, we will use PassCmd to instruct mbsync to extract the password from our netrc file. I first tried to use AWK for this purpose, but it does not support regexp capture, so I went with Perl. Note that the login, i.e. the email address for Gmail, has to be properly escaped in the regular expression. For example, bob@example.com must be included as bob\\@example\\.com.

PipelineDepth controls the number of IMAP commands send in a row before reading responses. It is recommended to use 1 due to Gmail rate limiting.

We then define the local store, i.e. a Maildir directory on our machine:

MaildirStore local
Path ~/mail/
Inbox ~/mail/INBOX
Subfolders Verbatim

Nothing special here, we specify the path of the directory (careful, the final / is important), and indicate that IMAP directories are stored in Maildir directories named the same way. At this point we can create the local directory:

mkdir -m 700 ~/mail

Finally we define the channel:

Channel backup
Far :gmail:
Near :local:
Patterns *
Create Near
Remove Near
Expunge Near

The Patterns instruction is used to indicate that we want to synchronize all IMAP directories. We also indicate that directory creation, directory removal and mail deletion is only performed on the local store.

To test that everything is correct, run mbsync -l backup. If everything is correct, mbsync will list all IMAP directories which will be synchronized.

To actually copy all emails, run mbsync --pull backup. Since I have a gigabit optic fiber connection, I was only limited by Google servers. Exporting more than 20 thousands messages (1.5GB) took roughly one hour and a half.

Systemd timer

At this point, you could simply run mbsync manually, but I am way too lazy for that. I use Archlinux, meaning that my init system is the controversial Systemd. While not everyone likes Systemd, it makes it easy to run periodic tasks.

First write the service file at ~/.config/systemd/user/mbsync.service:

[Unit]
Description=Backup emails
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/bin/mbsync --pull backup

[Install]
WantedBy=default.target

Then the timer itself at ~/.config/systemd/user/mbsync.timer:

[Unit]
Description=Backup emails

[Timer]
OnStartupSec=5min
OnUnitActiveSec=1h

[Install]
WantedBy=timers.target

We can now use systemctl to enable the service and the timer and execute it a first time:

systemctl --user enable mbsync.service
systemctl --user enable mbsync.timer
systemctl --user start mbsync

The local Maildir will now be updated 5 minutes after the user session starts, and every hour after that. You can check the status of the task with systemctl --user status mbsync.

Share the word!

Liked my article? Follow me on Twitter or on Mastodon to see what I'm up to.