Table of Contents

About

One of the first things that will be missing after creating a swarm and delegating nodes is that there will be no way to execute commands with crontab such that the command will be agnostic to all nodes.

Of course, there are various software solutions that are able to execute commands on a timer similar to cron that are designed for docker but ultimately the commands that are needed to be run will still have to be stored somewhere centrally accessible to all nodes.

Since cron has been around for ages, one simple solution is to create a fat cron docker container that will be able to execute commands and also be deployed to various nodes in the swarm. The container will be an open-ended build and will require commands to be added to the docker build file and then the container has to rebuilt.

Setup

The docker build file is as follows:

FROM debian:latest

# Install cron and required utilities.
RUN apt-get update
RUN apt-get -y install \
    build-essential \
    openssl \
    git \
    subversion \
    gnupg \
    rsyslog \
    procps \
    cron \
    cron-daemon-common \
    curl \
    jq \
    gawk \
    sed \
    grep \
    sqlite3 \
    less \
    bc \
    wget \
    rclone

# Install node-js
RUN curl -fsSL https://deb.nodesource.com/setup_current.x | bash - && \
    apt-get install -y nodejs
RUN node --version && npm --version

# Install selenium for web automation.
RUN npm install -g selenium-side-runner
RUN npm install -g chromedriver
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
    echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list && \ 
    apt-get update && apt-get -y install google-chrome-stable

# Install YouTube downloader.
RUN curl -fsSL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp && \
    chmod +x /usr/local/bin/yt-dlp && \
    yt-dlp -U

# Create an auxiliary user.
RUN groupadd -r system -g 1000 && \
    useradd -u 1000 -r -g system -s /usr/bin/bash -c "Docker image user" system

# Run the command on container startup.
CMD /usr/sbin/rsyslogd -iNONE && cron && tail -f /var/log/syslog

where only the cron and cron-daemon-common packages are needed whilst the rest of the packages and installations are optional.

In this example, the container will have: rclone (a remote cloud synchronization tool), node.js, yt-dlp (the YouTube downloader), selenium (for web-automation) and various other smaller tools such as jq (the JSON command-line parser).

This file is saved to a file called Dockerfile and then built with:

docker build -t docker.repo:5000/cron:latest .

where:

  • docker.repo:5000 is an internal docker repository

The compose file to run the build after completion, is the following YAML file:

version: '3.7'

services:
  cron:
    image: docker.repo:5000/cron:latest
    volumes:
      - /mnt/docker/data/cron/hourly:/etc/cron.hourly
      - /mnt/docker/data/cron/daily:/etc/cron.daily
      - /mnt/docker/data/cron/monthly:/etc/cron.monthly
      - /mnt/docker/data/cron/weekly:/etc/cron.weekly
      - /mnt/docker/data/cron/yearly:/etc/cron.yearly
      - /mnt/docker/data/cron/scratch:/scratch
      - /mnt/storage:/mnt/storage
 

where:

  • all the "hourly", "daily", "monthly", "weekly", "yearly" paths represent a cron folder that is mounted on the local host and then exported,
  • scratch is a place where some configuration files can be stored when writing scripts,
  • /mnt/storage is mapped into the container and typically it would represent some NAS storage

Strategy

When the container is run, it will start executing files from the "hourly", "daily", "monthly", "weekly" and "yearly" folders such that scripts should be placed within those folders.

Of course, when another command-line tool is needed, the tool must be installed using the Docker build file, the container rebuilt, published and then ran again.


docker/creating_a_fat_cron_container_for_a_swarm.txt ยท Last modified: 2024/04/26 22:30 by office

Access website using Tor Access website using i2p Wizardry and Steamworks PGP Key


For the contact, copyright, license, warranty and privacy terms for the usage of this website please see the contact, license, privacy, copyright.