Roll Your Own Cloud

 Posted on July 30, 2021

Cloud has to be the most over-used and under-understood term of the last 10 years.

The problem is that “cloud” can refer to many things, since at its core a cloud service is simply a remote service.

I’ll use a slightly more expansive definition, at least for the purpose of this post. I’ll refer to a cloud as some service that allows you to securely share, store, and manage data and files between users.

Nextcloud

The most popular open source cloud solution is Nextcloud. Nextcloud was introduced in 2016 after forking from the OwnCloud project. Since they share a common codebase, they are quite similar but diverged on ideological grounds. The political and philosophical points are not relevant here, so please explore both and make your own choice. I like Nextcloud, so that’s what I’ll base this on.

Among the wonderful things Nextcloud offers is a Docker Image, the ability to self host, and several 1st and 3rd party app integrations that work with the platform.

Here are a few closely integrated apps:

  • Videoconferencing
  • Calendar
  • Mail
  • Messaging
  • File Sharing (Individual & Group)
  • Task Management

… and more!

Browse the list of available apps HERE.

Setting up Nextcloud

The wonderful thing about the Docker image is that (properly configured) it will install everything automatically, prepopulating the database with an admin user and some smart default settings.

As usual we’ll be working on the command line in the /docker directory, so head over there and follow along with me.

devil@linodevm:$ sudo su -
devil@linodevm:# mkdir /docker/nextcloud
devil@linodevm:# cd /docker/nextcloud

Then create the following docker-compose.yml using nano or vim —

version: "2"

services:
  app:
    image: nextcloud:stable
    container_name: nextcloud
    restart: unless-stopped
    environment:
      - MYSQL_PASSWORD=nextcloudPass
      - MYSQL_DATABASE=nextcloudDB
      - MYSQL_USER=nextcloudUser
      - MYSQL_HOST=db
      - NEXTCLOUD_TRUSTED_DOMAINS=nextcloud.bowtieddevil.com
      - NEXTCLOUD_ADMIN_USER=admin
      - NEXTCLOUD_ADMIN_PASSWORD=someRandomlyGeneratedPassword
      - TRUSTED_PROXIES=traefik
      - REDIS_HOST=nextcloud-redis
      - REDIS_HOST_PASSWORD=nextcloudRedisPass
      - SMTP_HOST=smtp.mailgun.org
      - SMTP_SECURE=ssl
      - SMTP_PORT=587
      - SMTP_NAME=postmaster@mail.bowtieddevil.com
      - SMTP_PASSWORD=[redacted]
      - MAIL_FROM_ADDRESS=nextcloud@bowtieddevil.com
    volumes:
      - app:/var/www/html
    depends_on:
      - cron
      - db
      - redis
    labels:
      - traefik.enable=true
      - traefik.docker.network=nextcloud
      - traefik.http.routers.nextcloud.entrypoints=websecure
      - traefik.http.routers.nextcloud.rule=Host(`nextcloud.bowtieddevil.com`)
      - traefik.http.routers.nextcloud.middlewares=nextcloud-dav,nextcloud-header
      - traefik.http.middlewares.nextcloud-dav.replacepathregex.regex=^/.well-known/(card|cal)dav
      - traefik.http.middlewares.nextcloud-dav.replacepathregex.replacement=/remote.php/dav/
      - traefik.http.middlewares.nextcloud-header.headers.forceSTSHeader=true
      - traefik.http.middlewares.nextcloud-header.headers.stsSeconds=15552000
      - traefik.http.services.nextcloud.loadbalancer.server.port=80

  cron:
    image: nextcloud:stable
    container_name: nextcloud-cron
    restart: unless-stopped
    volumes:
      - app:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db

  db:
    image: mariadb:10.5
    container_name: nextcloud-db
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    restart: unless-stopped
    environment:
      - MYSQL_DATABASE=nextcloudDB
      - MYSQL_USER=nextcloudUser
      - MYSQL_PASSWORD=nextcloudPass
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
    volumes:
      - db:/var/lib/mysql

  redis:
    image: redis:6
    container_name: nextcloud-redis
    command: redis-server --requirepass nextcloudRedisPass
    restart: unless-stopped
    networks:
      - backend
    volumes:
      - redis:/data

volumes:
  app:
  db:
  redis:

networks:
  default:
    name: nextcloud

A few notes for this one:

There are some variables above that reference SMTP servers and Linode DNS. You should have created a Mailgun account and credentials in the Sending Email post, and a Linode server in the Project: Virtual Private Server post, so refer back to those if you need to catch up or need a refresher. Change these variables if you have another email server that you’d rather use. And please change the [redacted] settings to match your API token and email address.

This docker-compose.yml file assumes that you have a Traefik reverse proxy running elsewhere. That Traefik container needs access to the nextcloud network, which you can set in the appropriate docker-compose.yml file in /docker/traefik.

My Traefik docker-compose.yml is repeated below, please note the formatting of the networks section. If you choose to run everything inside a single docker-compose.yml, you can skip the networks stuff entirely since all containers will be created with access to one another.

services:
  traefik:
    image: traefik:v2.4
    container_name: traefik
    hostname: traefik
    restart: unless-stopped
    environment:
      - LINODE_TOKEN=[redacted]
    command:
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --certificatesresolvers.letsencrypt.acme.dnschallenge=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=linode
      - --certificatesresolvers.letsencrypt.acme.email=[redacted]
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.websecure.address=:443
      - --entrypoints.websecure.forwardedHeaders.insecure=true
      - --entrypoints.websecure.http.tls=true
      - --entrypoints.websecure.http.tls.certResolver=letsencrypt
      - --entrypoints.websecure.http.tls.domains[0].main=bowtieddevil.com
      - --entrypoints.websecure.http.tls.domains[0].sans=*.bowtieddevil.com
    ports:
      - "80:80/tcp"
      - "443:443/tcp"
    networks:
      - bowtieddevil
      - nextcloud
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - certs:/letsencrypt

volumes:
  certs:

networks:
  bowtieddevil:
    external: true
  nextcloud:
    external: true

When that’s all created, bring the stack up with docker-compose up and watch the output. You should see the db container start and report that it’s ready to accept connections, and then the app container will display an output similar to this:

nextcloud | Initializing finished
nextcloud | New nextcloud instance
nextcloud | Installing with MySQL database
nextcloud | starting nextcloud installation
nextcloud | Nextcloud was successfully installed
nextcloud | setting trusted domains…
nextcloud | System config value trusted_domains => 1 set to string nextcloud.bowtieddevil.com
nextcloud | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.3. Set the 'ServerName' directive globally to suppress this message
nextcloud | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.3. Set the 'ServerName' directive globally to suppress this message
nextcloud | [Sat Jul 31 05:04:39.402153 2021] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.4.22 configured -- resuming normal operations
nextcloud | [Sat Jul 31 05:04:39.402664 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

If that happens, navigate to your configured URL and log in with user admin and password someRandomlyGeneratedPassword. Once you’re in, change the admin password to something more robust. You can leave or delete that environment variable in the docker-compose.yml, since it’s only used at initial container creation.

The Nextcloud image will try to auto-configure your instance only one time, so if your first attempt failed, bring the stack down with docker-compose down && docker volume prune to start from a blank slate.

BUG NOTE: If you attempt to log in and see only a spinning button, you’ll need to make a small edit to the Nextcloud configuration file. Edit /var/lib/docker/volumes/nextcloud_app/_data/config/config.php using nano or vim, and add this line:

'overwriteprotocol' => 'https',

Don’t forget the closing comma! Nextcloud’s installer gets confused if it’s initiated behind an SSL-terminating reverse proxy, so it keeps trying to send you to the http:// version instead of https://. Adding this override fixes this behavior, and it’s permanent with no need to restart the container. Just refresh the page or log in again.

Using Nextcloud

Once you’ve logged into your account, navigate to the Apps menu in the top-right corner. I recommend installing the following:

  • Calendar
  • Contacts
  • Deck
  • Mail
  • Notes
  • Talk
  • Tasks

As you install these, the top bar will begin to fill with these new apps. Check them out and explore!

Download the Nextcloud app for your phone:

Then you can invite your family and friends to use your new cloud server, exchange messages directly, videoconference, send mail, and share files.

Good stuff, and Google doesn’t get to make money off you.

Newsletter

self hosted 

See also