blog.shukriadams.com

Game devops and other things

Run a single-user Mastodon server with Docker

Mastodon is a federated Twitter-like alternative which I'm sure you've already heard about. You can join one of many public Mastodon servers, but it's also fairly simple to host your own.

Why self-hosted?

Why would you want to host your Mastodon server?

Well, some people just like hosting their own things, and "because I like to" is the best reason of all. Beyond that, there are sound technical and social reasons though. While you can migrate Mastodon profiles between servers, you can't take your posts with you. It therefore makes sense to stay in one place, and if you own that place, all the better.

Mastodon currente has a weath of public servers you can join, but many of them are focused around a specific theme, and it might be that you cannot find a good fit. And while being part of a community might work well for you, you might also be the kind of person who'd prefer not to be at the whims of server administrators. Servers can also shut down because the people running it no longer want to.

The biggest downside with being all by your lonesome is discoverability - you're not going to get randomly discovered by your neighbors if you're the only one on the block, but discoverability on Mastodon is a bigger issue than which server you're on, given that its user base is fragmented by design.

Here be technical stuff

Should you decide to go it alone though, here is a guide for setting up Mastodon on a small cloud-based virtual private server (VPS). It uses Docker containers and for ease-of-use it also includes an Nginx and Let's Encrypt container in a single docker-compose file that does almost all the work for you. Containerizing Nginx means it's more complex to host other public HTTP services on this VPS, as they'll all be connected to your Mastodon docker-compose cluster. Seeing as Mastodon can be quite a CPU hog at times though, you're probably better off giving it full run of the VPS.

This guide assumes that you understand the basics of Docker, like installing it and using compose. This guide also relies on several files that live in a github repo I made for the guide. Clone this to your machine and use the files directly, or copy/paste file contents (links are provided as needed in the text below). The compose file I'm using is based on the official Mastodon version, with the following changes:

  • I added an Nginx container for reverse proxying
  • I added a Let's Encrypt container that generates certificates automatically
  • ElasticSearch is enabled by default, this is pretty handy
  • I changed the Docker network to run in bridged mode

This guide also includes scripts to backup and restore your Mastodon server. This is pretty important if you're self-hosting as you're responsible for your own data. I strongly suggest you do a test backup and restore before committing to this setup, just so you verify that the process runs end-to-end for you.

Requirements

  • A Linux machine with 1 CPU core, 1 Gb of RAM, and a few gigs of disk space - it's only you on this instance after all. A basic Linode, Digitalocean or any other VPS will do. I use Ubuntu server, but this should work for any Linux distro.
  • A relatively recent version of docker and docker-compose. I run Docker 25.0.3-1, older versions may necessitate running containers in privileged mode.
  • Port 80 and 443 available, so you can't have Nginx, Apache or any other standard web apps using those ports. On a clean VPS this won't be an issue.
  • A domain with a DNS etc. This will be the public address people see your server at. This guide assumes you know how to manage those.
  • Access to an SMTP server to send email. Mastodon suggests Mailgun, I used Amazon SES because it's dirt cheap.

Setup

  • Create and navigate to a directory somewhere on your VPS to host Mastodon, f.egs

    mkdir /srv/mastodon
    cd /srv/mastodon
    
  • Create a text file

    nano docker-compose.yml
    

    Copy/paste the contents of this docker-compose file into this. Find LETSENCRYPT_HOST and replace that the domain (or subdomain) your Mastodon server will be accessible at. Find and replace LETSENCRYPT_EMAIL with an email address you can be contacted at. These two fields are used by Let's Encypt to autogenerate SSL certificates for you. Save and exit your text editor.

  • create an empty settings file, the Mastodon setup wizard requires this.

    touch .env.production
    
  • Start the Mastodon setup wizard. It's a bit clunky, and if you do something wrong you need to start from scratch, etc etc, but oh well

    docker-compose run --rm web bundle exec rake mastodon:setup
    

    Note that the wizard doesn't remember your choices from previous attempts, nore does it use the contents of .env.production, it will always run through the same steps. I've briefly summarized each step below

    Your Mastodon domain : f.egs mastodon.yourdomain.com
    Single user mode : Yes 
    Run in Docker : yes
    All Postgres questions : select default answers (if you get an error, you're likely on an outdated version of docker)
    All Redis questions : select default answers (if you get an error, you're likely on an outdated version of docker)
    Store uploaded files on the cloud : no (all media will be saved to your VPS disk)
    Send emails from localhost: no (we want to use external SMTP)
    All SMTP questions: enter whatever fits your SMTP service
    Emails from : notifications@<some-sender-address-your-smtp-service-allows>
    Send test email : yes (always good to confirm your service works)
    Periodic checks : no (this guide pins containers so updates must be planned anyway)
    Save config : YES. 
    

    The wizard will print your config to screen, copy/paste and save this to .env.production in another terminal window, then

    Prepare db : yes
    

    Mastodon will populate your DB, and this is where things can get hairy.

    Create admin user : yes
    Name : <user name for your account> (you'll be logging in with this)
    Email: email address to send password resets to
    

    Mastodon will attempt to set up your user, and due to a common bug could fail when contacting its Redis container. If it does, don't panic, your user will likely have been created. If it does't fail, you'll be shown a random password to login in with. Save this. If you get the Redis error, proceed anyway.

  • You should be back at the command prompt now. Nginx needs to know what your server is called so it can reverse proxy to it. In your current directory, you should now see a /proxy/conf.d directory where the Nginx container stores its config.

    nano ./proxy/conf.d/mastodon-nginx.conf
    

    Copy/paste the contents of this nginx config file into it. Find the two occurences of YOURDOMAIN.COM and replace with your Mastodon domain address. Save and exit.

  • Finally, we need to set some permissions on the ./public directory that Mastodon saves files to. For this we need the ID of the Mastodon user in the main web container, and we need to make sure its up. Do a hard restart

    docker-compose down
    docker-compose up -d
    

    then

    docker exec mastodon-web id -u mastodon
    

    this returns a number, for me it's 991. Then run

    chown 991 -R ./public
    

    Replace 991 with whatever number you got.

  • Restart everything one last time

    docker-compose restart
    

    All containers should now start. If your domain DNS is pointed to your VPS' IP number, you should be able to access your instance in a browser. Note that Mastodon can take a few minutes to start first time - a few long minutes. If you managed to get a password out of the wizard, you can log in with that and set a real password. If not, request a password reset via email. If Mastodon doesn't send an email but you were able to receive a test email during the setup phase, it's likely that your user failed to create and you'll either need to start from scratch or troubleshoot.

  • Once in Mastodon, confirm write permissions work by setting your profile image under Preferences > Public Profile > Profile Image and ensure that it can save. If you get an error, you haven't set the permissions on the ./public directory properly.

Troubleshooting

I found setting Mastodon up quite simple … when it worked. When it didn't it was difficult to figure out why, and in the end it came down to either trying again or ignoring the problem. If your install attempt fails and you want to start from scratch be sure to

  • take all containers down with docker-compose down
  • delete whatever database got created by deleting the ./postgres14 directory - the Mastodon setup wizard doesn't like encountering an already-populated database.

Note that Mastodon has sporadic CPU spikes, if it's slow or unresponsive, give it a few minutes to settle.

Backup

There is a simple backup script in my repo, recreate it in your Mastodon directory and run with sh ./backup.sh (sudo likely required). This will create a backup.zip file with today's date in the name, containing your entire Postgres database, as well as everything in ./public, .env.production, your compose file, etc. This is all your data in a single zip. Ideally, you'd generate backups, frequently, with a cronjob maybe, and store your backups on another machine.

Restore

To restore a backup

  • create a new directory to run your restored Mastodon instance

  • Place a backup zip in that directory, and unzip with

    unzip <yourbackup>.zip -d .
    
  • You need to explicitly repopulate your Mastodon database first (you can't use the setup wizard for this). To repopulate, you need the database container to be up. You can do this by starting all your containers with your regular compose file , but that's messy, so I added a lightweight database-only compose. Create a docker-compose-dbonly.yml file in your directory, paste the contents into that, and start

    docker-compose -f docker-compose-dbonly.yml up -d
    

    Also create a restore.sh file and paste in restore-db.sh. Then run

    sh ./restore-db.sh

    You should get a bunch of output from Postgres as it populates itself. Take down the container

    docker-compose -f docker-compose-dbonly.yml down

    And restart everthing with

    docker-compose up -d

    Your instance should be up and ready, rewound to the backup point.

Conclusion

It's still early days for me on this. I don't know if Mastodon is what I'm looking for in a microblogging platform - in terms of feature and especially community, but I'm glad it's here, and I'm especially glad it's self-hostable. Like many open source projects I feel it could put a bit more work into its ease of deployment, specifically with weird bugs, but I've definitely seen worse.