Differences

This shows you the differences between two versions of the page.

--- fuss:linux [2020/05/08 21:53] – [Automatically Add all RNDIS Devices to a Bridge] office
+++ fuss:linux [2021/04/16 19:21] – [Self-Delete Shell Script] office
@@ Line 1142: / Line 1142: @@
 One usage case for this rule is to connect a bunch of RNDIS devices to an USB hub and have them join the network automatically as they are hotplugged; for instance, Raspberry Pis [[fuss/raspberry_pi#logging-in_via_ssh_over_usb|can be configured as USB gadgets]] and then connected to an USB hub.
+====== Scraping a Site Automatically using SystemD ======
+FTP sites can be scraped elegantly by using systemd and tmux on Linux. By starting a ''tmux'' detached terminal, ''wget'' can run in the background and download a website entirely whilst also allowing the user to check up on the progress by manually attaching and detaching from ''tmux''.
+The following script contains a few parameters underneath the ''Configuration'' comment and up to ''Internals'' in order to set:
+  * the download path (''DOWNLOAD_DIRECTORY''),
+  * the ''wget'' download URL (all protocols supported by ''wget'' such as FTP or HTTP) (''DOWNLOAD_URL''),
+  * a descriptive name for the ''tmux'' session (''TMUX_SESSION_NAME'')
+<code bash>
+[Unit]
+Description=Scrape FTP Site
+Requires=network.target local-fs.target remote-fs.target
+After=network.target local-fs.target remote-fs.target
+[Install]
+WantedBy=multi-user.target
+[Service]
+# Configuration
+Environment=DOWNLOAD_DIRECTORY="/path/to/directory"
+Environment=DOWNLOAD_URL="ftp://somesite.tld/somedirectory"
+Environment=TMUX_SESSION_NAME="somesite.tld-download"
+# Internals
+Type=oneshot
+KillMode=none
+User=root
+ExecStartPre = -/bin/mkdir -p \""$DOWNLOAD_DIRECTORY"\"
+ExecStart=/usr/bin/tmux new-session -d -c "\"$DOWNLOAD_DIRECTORY\"" -s "$TMUX_SESSION_NAME" -n "$TMUX_SESSION_NAME" "/usr/bin/wget -c -r \"$DOWNLOAD_URL\" -P \"$DOWNLOAD_DIRECTORY\""
+ExecStop=/usr/bin/tmux send-keys -t "$TMUX_SESSION_NAME" C-c
+RemainAfterExit=yes
+</code>
+The file should be placed under ''/etc/systemd/system'', then systemd has to be reloaded by issuing ''systemctl daemon-reload'', the service should then be loaded with ''systemctl enable SERVICE_FILE_NAME'' where ''SERVICE_FILE_NAME'' is the name of the file copied into ''/etc/systemd/system'' and finally started by issuing ''systemctl start SERVICE_FILE_NAME''.
+Upon every reboot, the service file will create a detached tmux terminal and start scraping files from the URL.
+====== Access Directory Underneath Mountpoint ======
+In order to access a directory underneath a mountpoint without unmounting, create a bind mount of the root filesystem to a directory and then access the content via the bind mount.
+Ie, to access the contents of the directory ''/mnt/usb'' on top of which a filesystem has been mounted, create a directory:
+<code bash>
+mkdir /mnt/root
+</code>
+and create a bind mount:
+<code bash>
+mount -o bind / /mnt/root
+</code>
+Finally access the original underlying content via the path ''/mnt/root/mnt/usb2''.
+====== Self-Delete Shell Script ======
+<code>
+rm -rf -- "$@"
+</code>
+====== Resize Last Partition and Filesystem in Image File ======
+Assuming that an image file is available and named, for example, ''raspios.img'' then the following procedure will extend the last partition and the filesystem by a given size.
+First, extend the image file ''raspios.img'' itself with zeroes:
+<code bash>
+dd if=/dev/zero bs=1M count=500 >> raspios.img
+</code>
+where:
+  * ''1M'' is the block size,
+  * ''500'' is the amount of blocks to copy over
+In this case, the image file ''raspios.img'' is extended by $500MiB$. Alternatively, ''qemu-img'' can be used to extend the image file:
+<code bash>
+qemu-img resize raspios.img +500M
+</code>
+The next step is to run ''parted'' and extend the last partition inside the image file. Open the image file with ''parted'':
+<code bash>
+parted raspios.img
+</code>
+and then resize the partition, for example:
+<code bash>
+(parted) resizepart 2 100%
+</code>
+where:
+  * ''2'' is the partition number
+The ''parted'' command will resize the second partition to fill 100% of the available space (in this example, it will extend the second partition by $500MiB$).
+The final step is to enlarge the filesystem within the second partition that has just been extended by $500MiB$. ''kpartx'' will create mapped devices for each of the partitions contained within the ''raspios.img'' image file:
+<code bash>
+kpartx -avs raspios.img
+</code>
+First, the existing filesystem has to be checked:
+<code bash>
+e2fsck -f /dev/mapper/loop0p2
+</code>
+where:
+  * ''/dev/mapper/loop0p2'' is the last partition reported by ''kpartx''
+and then finally the filesystem is extended to its maximum size:
+<code bash>
+resize2fs /dev/mapper/loop0p2
+</code>