Transfer Directory using Tape Archive Binding, Compression, Encryption and Raw Network Connections

A straight up example that has many variations will be to bind the contents of a directory to transfer using tape archives.

Requirements

  • lzma, either packaged by its own name or as lzma-utils, the utility being used being of the same name lzma,
  • openssl, or an SSL suite variant, mentions include wolfSSL for embedded systems, but access to a command-line tool that is able to perform symmetric encryption

Command

On the system that has to send a directory:

tar -cvf - -C /mnt/data . | \
    lzma -z | \
    openssl enc -aes128 -k "password" | \
    nc -l -p 10000

where:

  • /mnt/data is the directory whose contents must be transferred, it does this by using -C /mnt/data which makes tar to change the current directory internally to /mnt/data and the act on the files in the current directory represented by the last full stop .
    • -c stands for creating the archive,
    • -v means to be verbose (the output from tar will be the only text being displayed and monitoring the files as they are transferred is useful),
    • -f - means to output the archive to stdout,
  • openssl is used for encryption,
    • will use 128 bit AES and symmetrically encrypt the data -aes128,
    • using the unsalted password password via -k "password"
  • netcat is then actually used to listen for a connection instead of connecting to remote machine

and on the system that is meant to receive the directory:

nc SEND.TLD 10000 | \
    openssl enc -d -aes128 -k "password" \
    lzma -d | \
    tar -xvf -

where:

  • SERVER.TLD and 10000 is the IP endpoint of the machine sending the contents of the directory,
  • openssl with decrypt the data being sent using an unsalted password password:
    • -d stands for "decrypt",
    • AES 128 bit encryption is used via -aes128 and,
    • the unsalted password is specified via -k,
  • lzma -d will decompress the stream,
  • tar -xvf - will:
    • -x, extract,
    • -v be verbose and,
    • -f - use standard input contents as the data to work on

Remarks

Note that there is some pedantic issue that can be a counter-mnemonic if the user understands theory because for a symmetric cipher the parameters should be symmetrically identical. In other words, applying the algorithm once, would result in a ciphertext and then applying the same algorithm twice without any modifications should result in the plaintext thereby completing the round-trip. However, for the openssl command-line tool, decrypting the data requires the parameter -d and omitting the -d parameter results openssl producing garbage (which it shouldn't for a symmetric cipher!) that is difficult to trace down to something. This is more than likely not a bug but rather some implementation detail or design choice of openssl that makes its usage counter-intuitive.

In other words, to symmetrically encrypt, use:

openssl enc -aes128 -k "password"

but to decrypt, use:

openssl enc -d -aes128 -k "password"

where:

  • the extra -d stands for decrypt (even though it should not be a requirement for symmetric ciphers)

Sometimes the openssl call can be found in the wild with the additional -base64 parameter which instructs openssl to armor the encrypted output by encoding it to Base64. In principle, the idea would be great in case one would be dealing with stuff like HTML entities where the character set is restricted to "readable characters" that could influence some syntax, but in this case raw socket connections via netcat are used such that armoring to Base64 is literally a very cute way to waste time.


sh/quickly_transfer_files_between_machines/transfer_directory_using_tape_archive_binding.txt ยท Last modified: 2025/07/12 21:59 by office

Wizardry and Steamworks

© 2025 Wizardry and Steamworks

Access website using Tor Access website using i2p Wizardry and Steamworks PGP Key


For the contact, copyright, license, warranty and privacy terms for the usage of this website please see the contact, license, privacy, copyright.