The write-up on the web FUSS attempts to settle the dispute between sub-path and domain reverse-proxying.

The only way to install aCaddy modules is to use a tool named xcaddy that will automatically recompile caddy and additionally also build modules if they are specified on the command line because "modules" as installable packages like everyone else, nginx, apache, etc, is just too old-fashioned, breaking the Linux FSH is awesome, and "compiling" makes you look cool to your friends that won't know what you're talking about.

Assuming that golang is installed (Google Go), that xcaddy is installed from (a separate) caddy repository, and as an example, to install the replace_response module that does content re-writing, the following command is issued:

xcaddy build --with github.com/caddyserver/replace-response

this will compile caddy and excrete the binary into the current directory. Of course, because you do not want to end up with an unmanageable operating system and due to Caddy not providing these modules as separate packages, more than likely you would want to copy the caddy binary to /usr/local/bin instead of just replacing the system-installed binary, or worse, just tossing it into /usr/bin/ like a pro.

After copying the caddy binary to /usr/local/bin, the caddy service file usually placed at /lib/systemd/system/caddy.service would have to be copied into /etc/systemd/system/ and then loaded in order to mask the old service file as well as load caddy from /usr/local/bin instead of /usr/bin.

On Debian, the provided SystemD file is copied from /lib/systemd/system/caddy.service to /etc/systemd/system/caddy.service and then the file is modified to just change the patch to match the new, enhanced, caddy binary residing in /usr/local/bin:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/local/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

while all the rest of the file can remain the same.

Caddy has a built-in client for ZeroDNS and LetsEncrypt such that Caddy can be configured to generate certificates automatically for domains that it reverse proxies to backend servers. Furthermore, wildcard certificates are supported through LetsEncrypt using a CloudFlare plugin that sets a TXT record on a wildcard domain such that LetsEncrypt can pass validation and issue a wildcard certificate.

The first step is to re-compile xcaddy with the cloudflare plugin:

xcaddy build --with github.com/caddy-dns/cloudflare

Note that initially the compile failed due to the golang package being too old on Debian stable, but fortunately golang does not have dependencies that must be upgraded such that the packages could be switched to unstable/ testing just to re-compile caddy.

After caddy compiles, the file /etc/caddy/Caddyfile has to be changed in order to add a domain and potentially a wildcard domain for which certificates should be automatically generated. Here is grimore.org, as an example that can be directly added to /etc/caddy/Caddyfile:

*.grimore.org, grimore.org {
        tls {
                dns cloudflare API_KEY
                resolvers 1.1.1.1
        }

        reverse_proxy {
                to grimore.lan
        }
}

where API_KEY is a Cloudflare API key that must be created from the CloudFlare interface. For the record, the API key created must only have permissions to edit the DNS records of the domain for which certificates have to be generated, such that all other permissions are not necessary.

Note that in the past, we have written scripts that interact with the CloudFlare API, where a global API key has been used; however for the CloudFlare plugin for caddy, a global API key cannot be used and a separate API key must be manually configured by following the menu My Profile→API Tokens→Create Token.

For domains that do not need a wildcard certificate for subdomains, for example domain.tld, with the following caddy configuration:

caddy {
    tls valid@email
    
    redir /sync /sync/
    handle_path /sync/* {
        reverse_proxy backend:8888 {
            header_up Host {host}
            header_up X-Real-IP {remote}
            header_up X-Forwarded-Proto {scheme}
        }
    }
}

the configuration tls valid@email where valid@email should be an E-Mail (that will not be verified, and must just be syntactically valid) is sufficient to make caddy generate certificates.

Here are some configuration templates for various services reverse-proxied by caddy.

Typically, a set definition such as:

        @skip_auth {
                not path_regexp ^\/login$
                not remote_ip 192.168.1.0/24
        }

can be used to group conditions together in order to generate exceptions to rules and will match any request where:

• the URL path does not start and end in /login,
• the remote IP address from where the request originated is not within the 192.168.1.0/24 address range.

One interesting feature that caddy supports is to additionally match request headers. For example, the following set definition:

        @skip_auth {
                not path_regexp ^\/login$
                not remote_ip 192.168.1.0/24
                not header X-Authorize "TOKEN"
        }

will add an exception to the @skip_auth match set in case the HTTP request contains the header X-Authorize with the value TOKEN.

This scheme is secure but it needs to be combined with the tls caddy feature to ensure that the header is not sent to caddy over a plaintext connection. One particularly useful application is a situation where caddy is configured to authenticate requests but if the request contains a certain header and token then the request can go through for applications that cannot be modified to include a full authentication process.

One of the cool built-in features of caddy is forms-based authorization that can be used to gate services that are not supposed to be accessed publicly. Given that caddy is a reverse-proxy, the authentication can take place directly on caddy without even having to reach backend services, as well as allowing for separation of concerns between caddy and the gated services.

The following is a caddy configuration that defines a form-based authenticator in the global section for the domain home.tld, as well as two custom services service1 and service2, both services with their designated subdomains service1.home.tld, respectively service2.home.tld.

{
...

        order authenticate before respond
        order authorize before reverse_proxy

        security {
                local identity store localdb {
                        realm local
                        #path /tmp/users.json
                        path /etc/caddy/auth/local/users.json
                }
                authentication portal gate {
                        enable identity store localdb
                        cookie domain home.tld
                        cookie lifetime 86400
                        ui {
                                theme basic
                                links {
                                        "Services" /services
                                }
                        }
                }
                authorization policy admin_policy {
                        set auth url https://auth.home.tld
                        allow roles authp/user authp/admin
                }
        }

        order replace after encode
        
...
}

auth.home.tld {
        tls me@home.tld
        
        authenticate with gate
}

service1.home.tld {
         tls me@home.tld
         
         authorize with admin_policy
         
         reverse_proxy service1 {
                  header_up Host {host}
                  ...
         }
}

service2.home.tld {
         tls me@home.tld
         
         @skip_auth {
                  not remote_ip 192.168.1.0/24
         }
         
         handle @skip_auth {
                   authorize with admin_policy
         }
}

In order to generate the password file automatically, the line:

#path /tmp/users.json

is commented out and caddy started. This will lead to creating the file /tmp/users.json that will then have to be moved to /etc/caddy/auth/local/users.json. The password inside the file must be generated using bcrypt using any hashing tool with support for bcrypt.

The effect of the configuration is that once service1.home.tld or service2.home.tld is accessed, in case the request has not previously been authenticated, the request gets redirected to auth.home.tld where caddy will prompt for an username and password matching the file /etc/caddy/auth/local/users.json. After authentication, caddy will redirect the user back to the respective service, either service1.home.tld or service2.home.tld.

The difference between service1.home.tld and service2.home.tld is that service2.home.tld contains a set definition that will only require the request to authenticate if it does not originate from the IP subnet 192.168.1.0/24. The exclusion is performed by first defining a set of exclusions @skip_auth and then only requiring authentication if the @skip_auth set matches (in this case, a single rule stating that the request should not be from the 192.168.1.0/24 subnet).

One problem with dynamic DNS providers like DuckDNS is that if a large number of subdomains are registered then caddy will have a hard time generating certificates for all domains due to the throttles imposed by upstream services such as ACME and ZeroSSL. The immediate solution is to use a wildcard certificate that will match all subdomains and then handle each sub-domain of the top TLD by matching the HTTP Host header and reverse-proxying to the correct machine.

Here is a skelleton configuration for various services configured for the domain.duckdns.org top TLD:

*.domain.duckdns.org, domain.duckdns.org {
    tls {
        dns duckdns ...
    }

    @media host media.domain.duckdns.org
    handle @media {
        reverse_proxy server1:8000 {
            trusted_proxies 192.168.1.0/24
        }
    }
    
    @games host games.domain.duckdns.org
    handle @games {
        reverse_proxy server2:8001 {
            trusted_proxies 192.168.1.0/24
        }
    }
}

where:

• media.domain.duckdns.org and games.domain.duckdns.org are subdomains of domain.duckdns.org,
• server1, 8000 and server2, 8001 are the real machine hostnames, respectively ports, behind caddy

For this to work, the duckdns TLS provider is used to authenticate to ACME and ZeroSSL using DNS TXT entries and it requires caddy to be compiled with the duckdns DNS provider. A docker build file is provided that shall accomplish that.