HomeLab initial setup

7 minute read

Installing Docker

There is no point in explaining in details how to install docker on your machine as the documentation would be deprecated the moment I finish this sentence.

However you can find how to install Docker on Ubuntu right here.

Installing Docker-compose

Now that you have installed Docker, you can install docker-compose, a magnificent tool to configure your container with a simple text file ! The instructions can be found on docker’s official documentation.

Setting up the docker network

We want each of our docker container to have statics MAC and IP addresses. To do so, we use the macvlan network mode.

To be able to use this mode, we first need to create the actual macvlan. Let’s create a script called host-macvlan-net.sh containing the command and its explanation :

vim host-macvlan-net.sh

Here is the script content:

#!/bin/zsh
# This script creates the network host-macvlan-net used by the docker containers on this server
# Every container has an address in 10.0.6.x
# Note that it is impossible for the current server to ping these containers
# However an external computer, or another docker container will be able to do so
docker network create -d macvlan --subnet=10.0.0.0/20 --gateway=10.0.0.1 --ip-range=10.0.6.0/24 -o parent=eno1 host-macvlan-net

# Use `docker network ls` to list the docker networks
# Use `docker network inspect host-macvlan-net` to print the network details
# User `docker network rm host-macvlan-net` to remove the network

The command is

docker network create -d macvlan --subnet=10.0.0.0/20 --gateway=10.0.0.1 --ip-range=10.0.6.0/24 -o parent=eno1 host-macvlan-net
  • -d macvlan : uses the network mode macvlan
  • --subnet=10.0.0.0/20 : defines the subnet for the network
  • --gateway=10.0.0.1 : defines the gateway for the network
  • --ip-range=10.0.6.0/24 : defines the ip-range for the network (useful only if you don’t want to set static IPs in the docker-compose file)
  • -o parent=eno1 : defines the parent interface (here it is my ethernet interface)
  • host-macvlan-net: the name I gave to the network.

We can now use docker network ls to list the docker networks to verify that it was properly created.

Setting up an application

Since most applications will be set up the same way, I decided I would show in details how a simple one is done here to save some effort for the next ones.

In this sample application setup we install pyload.

The folder

I like to organize my applications in folders :

braincoke@nuc$  ls -1
Firefly
Gitlab
Jdownloader2
MayanEdms
Monica
OpenHab
Organizr
Portainer
Pyload
Rancher
Torrent

Each folder here will contain :

  • the docker-compose.yml file describing the application
  • the directories that will be mounted in the container

The network

Since I decided at the beginning of my project that each container will have its own reserved IP address, I will have to manually reserve the IP address in my router configuration panel.

If I decide that the IP address of the container should be 10.0.6.11, I know that the MAC address will be 02:42:0A:00:06:0B. With that information I can set my DHCP to reserve the IP 10.0.6.11 to the MAC address 02:42:0A:00:06:0B.

Docker MAC addresses are predictable for static IPs. It will follow the format : 02:42:<IPv4 address in hexa>. For instance 10 in hexadecimal is 0A, 11 is 0B and 6 is 06.

The docker-compose file

Once we have our folder (that was hard), we have to create our docker-compose.yml file.

cd Pyload
touch docker-compose.yml

Our docker-compose will look like this :

version: '2.2'
services:
  web:
    container_name: 'pyload'
    image: 'writl/pyload:latest' 
    hostname: 'pyload'
    restart: unless-stopped 
    environment:
      - PUID=1000
      - PGID=1004
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - '../NAS/Downloads:/opt/pyload/Downloads'
      - './config:/opt/pyload/pyload-config'
    networks:
      host-macvlan-net:
        ipv4_address: 10.0.6.11
networks:
  host-macvlan-net:
    external:
      name: host-macvlan-net

As you can see from the line image: 'writl/pyload:latest', we are downloading a container named pyload with the tag latest from user writl from hub.docker.io.

Be very careful what container you download from hub.docker.io. It is as dangerous as downloading an unkwnown binary and executing it on your system. Take 10 minutes to overview the Dockerfile and see if there is something shady going on when the image is being built.

Here is a brief explanation of most of the configuration lines :

  • version: '2.2' : the docker-compose file version used (expands or reduces the scope of options that can be used in the file)
  • services : we tell the file that we are about to describe containers
    • web : this is our first service, we call it “web”
      • container_name : the name we want our container to have. Used by the docker daemon.
      • hostname: the name our container will give itself on the network.
      • restart: define when to restart the container
      • environment: define environment variables
      • ports: define ports you want to export
      • volumes: define volume you want to mount from the system to the container
      • networks: tell the file we are about to define the network configuration for the container
        • host-macvlan-net : tell the file we are about to define the configuration for the network named host-macvlan-net
          • ipv4_address : the IP address we want our container to have
    • networks: describe the networks we will use for our container

When using the volumes option, remember that the syntax is <path/to/local/dir>:<path/to/dir/inside/container>.

To summarize all of this, our docker file is saying that :

  • our container comes from the docker hub
  • our container will be named pyload
  • the hostname of the container will also be pyload
  • its IP address will be 10.0.6.11
  • the PUID and GUID environment variables will be set to our predefined values in the container
  • some volumes will be mounted as described in the file

Mount points

Creating the directories

As we have seen from our docker-compose file, we have specified that :

  • the directory ../NAS/Downloads should be mounted to /opt/pyload/Downloads inside the container
  • the directory ./config should be mounted to /opt/pyload/pyload-config inside the container

We don’t have to create the directories inside the containers, docker will handle that (if they don’t already exist). However we do need to create them on our system. So be sure that ~/NAS/Downloads and ~/Pyload/config exist before starting the container !

You will often see a mount point ./config. This is a really usefull way to save the application config. Docker containers are stateless, when a container is removed, all the data it contained is destroyed… unless it was stored in a mounted directory ! By mounting the application config folder, you can save the state of your docker container and back it up regularly if you ever have a hardware issue with your homelab server.

Docker volumes

You can also obtain data persistence by creating docker volumes which is the recommended way to mount volumes.

This creates a data blob that you can reuse in other containers and is managed by docker. It adds a layer of security since the container does not have direct access to the host filesystem.

Using a NAS

In my setup I mostly use a NAS to store :

  • the backups of the configuration directories
  • the container downloads

To do so I mount the NAS directory directly on the host. For instance if I need to mount the NAS directory //nas/Downloads to the host ~/mounted/Downloads/ I will need to :

  1. Create a user downloaduser with the correct rights on the NAS.
  2. Create a read-only file /etc/nas-downloaduser-credentials containing the downloaduser credentials.
  3. Add the following line to /etc/fstab: 
    //nas/Download    /home/braincoke/mounted/Downloads cifs user,rw,credentials=/etc/nas-downloaduser-credentials,dir_mode=0777,file_mode=0777,iocharset=utf8,x-systemd.automount,vers=3.0 0 0

The file /etc/nas-downloaduser-credentials looks like this :

username=downloaduser
password=horse-staple-battery-correct-vintage

We change its permission to read-only to the user with chmod 400 /etc/nas-downloaduser-credentials.

Since the credentials are not protected on the host, it is important to reduce its privileges on the NAS to the strict minimum.

Managing your containers

Once you have your docker-compose.yml file, you can start the container with :

docker-compose up -d

-d starts the process as a daemon. Also, you must do it in the directory containing the file or use the appropriate option to specify which docker-compose file should be used.

If you ever need to see the container logs, just run :

docker-compose logs -f

To stop all the containers started by the file:

docker-compose stop

Replace stop by restart to restart the containers. And replace stop by rm to delete the containers.

You can list the running containers with docker ps and all the containers (including the stopped ones) wit docker ps -a.

Backup script

I always end up my configurations by creating a backup script and testing my backup.

If you have never tested your backup, consider you have none.

I store my backup scripts in ~/scripts/backup and the backup themselves in ~/backup which is a mounted volume from my NAS. That way, my backup are stored on another device.

The purpose of the backup script is to back up the state of the container. To do so, I copy and tar every volume the container mounts.

Conclusion

This post exposed the basics of how I manage my docker-compose files, volumes, and backups. The next posts will explain briefly how to install specific applications and how to configure them.

You May Also Enjoy