One of the first things that will be missing after creating a swarm and delegating nodes is that there will be no way to execute commands with crontab such that the command will be agnostic to all nodes.
Of course, there are various software solutions that are able to execute commands on a timer similar to cron that are designed for docker but ultimately the commands that are needed to be run will still have to be stored somewhere centrally accessible to all nodes.
Since cron has been around for ages, one simple solution is to create a fat cron docker container that will be able to execute commands and also be deployed to various nodes in the swarm. The container will be an open-ended build and will require commands to be added to the docker build file and then the container has to rebuilt.
The docker build file is as follows:
FROM debian:latest # Install cron and required utilities. RUN apt-get update RUN apt-get -y install \ build-essential \ openssl \ git \ subversion \ gnupg \ rsyslog \ procps \ cron \ cron-daemon-common \ curl \ jq \ gawk \ sed \ grep \ sqlite3 \ less \ bc \ wget \ rclone # Install node-js RUN curl -fsSL https://deb.nodesource.com/setup_current.x | bash - && \ apt-get install -y nodejs RUN node --version && npm --version # Install selenium for web automation. RUN npm install -g selenium-side-runner RUN npm install -g chromedriver RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \ echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list && \ apt-get update && apt-get -y install google-chrome-stable # Install YouTube downloader. RUN curl -fsSL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp && \ chmod +x /usr/local/bin/yt-dlp && \ yt-dlp -U # Create an auxiliary user. RUN groupadd -r system -g 1000 && \ useradd -u 1000 -r -g system -s /usr/bin/bash -c "Docker image user" system # Run the command on container startup. CMD /usr/sbin/rsyslogd -iNONE && cron && tail -f /var/log/syslog
where only the cron
and cron-daemon-common
packages are needed whilst the rest of the packages and installations are optional.
In this example, the container will have: rclone
(a remote cloud synchronization tool), node.js
, yt-dlp
(the YouTube downloader), selenium (for web-automation) and various other smaller tools such as jq
(the JSON command-line parser).
This file is saved to a file called Dockerfile
and then built with:
docker build -t docker.repo:5000/cron:latest .
where:
docker.repo:5000
is an internal docker repositoryThe compose file to run the build after completion, is the following YAML file:
version: '3.7' services: cron: image: docker.repo:5000/cron:latest volumes: - /mnt/docker/data/cron/hourly:/etc/cron.hourly - /mnt/docker/data/cron/daily:/etc/cron.daily - /mnt/docker/data/cron/monthly:/etc/cron.monthly - /mnt/docker/data/cron/weekly:/etc/cron.weekly - /mnt/docker/data/cron/yearly:/etc/cron.yearly - /mnt/docker/data/cron/scratch:/scratch - /mnt/storage:/mnt/storage
where:
scratch
is a place where some configuration files can be stored when writing scripts,/mnt/storage
is mapped into the container and typically it would represent some NAS storageWhen the container is run, it will start executing files from the "hourly", "daily", "monthly", "weekly" and "yearly" folders such that scripts should be placed within those folders.
Of course, when another command-line tool is needed, the tool must be installed using the Docker build file, the container rebuilt, published and then ran again.