Glob
is a fast shell-based equivalent of regular expressions.
Glob | Explanation |
---|---|
* | Matches any string, of any length |
foo* | Matches any string beginning with foo |
*x* | Matches any string containing an x (beginning, middle or end) |
*.tar.gz | Matches any string ending with .tar.gz |
foo? | Matches foot or foo$ but not fools |
Glob | Explanation |
---|---|
[abcd] | Matches a , b , c and d |
[a-d] | The same as above, if your locale is C or POSIX . Otherwise, implementation-defined. |
[!aeiouAEIOU] | Matches any character except a , e , i , o , u and their uppercase counterparts |
[[:alnum:]] | Matches any alphanumeric character in the current locale (letter or number) |
[[:space:]] | Matches any whitespace character |
[![:space:]] | Matches any character that is not whitespace |
[[:digit:]_.] | Matches any digit, or _ or . |
array[$[${#array[@]}+1]] = 3
is homologous to:
array[elements_in(array)+1] = 3
Input
:
T="test.svg" echo ${T/svg/png}
Output
:
test.png
CHECK_STRING="google" if [[ "$CHECK_STRING" != *o* ]]; then echo "character o is in the string" fi
The following code:
IFS=' ' read -ra VAR_ARRAY <<< "$input"
where:
IFS
will switch the separator (the character to split on) to a space.read
will read-in the inputVAR_ARRAY
is the array that will be filled with the values from the input split by the separator$input
is the input to splitwill split the input to an array.
In order to iterate over the elements of the array, one would write:
for e in "${VAR_ARRAY[@]}"; do echo $e done
which will print out all the elements in the array.
To get both the index and the value while iterating over the array, one would write:
for i in "${!VAR_ARRAY[@]}"; do echo "$i ${VAR_ARRAY[i]}" done
where on every iteration $i
is the index and ${VAR_ARRAY[i]}
will expand to the array element at i
.
To get the number of elements in the array, one would write:
echo "${VAR_ARRAY[@]}"
If you use coretemp
to retrieve the CPU sensors temperature, a bunch of files are created in /sys/devices/platform/coretemp.*/hwmon/hwmon*/
such as temp1_label
, temp2_label
, etc… Each of them represent the temperature of one of the cores but there is no guarantee that the numbering will start at 1
. The following bash function reads the files in /sys/devices/platform/coretemp.*/hwmon/hwmon*/
and returns the file corresponding to a certain core file:
########################################################################### ## Copyright (C) Wizardry and Steamworks 2016 - License: GNU GPLv3 ## ########################################################################### function getCoreTemperatureFile { for i in /sys/devices/platform/coretemp.*/hwmon/hwmon*/*_label; do IFS=' ' read -ra CORE <<< `cat $i` if [ "${CORE[0]}" == "Core" ] && [ "${CORE[1]}" == $1 ]; then echo "${i//label/input}" fi done }
an example call, would then be:
CPU0_TEMPERATURE_FILE=$(getCoreTemperatureFile 0) echo $CPU0_TEMPERATURE_FILE
in order to retrieve the temperature of the first core (0
).
Via a function listFiles
:
# By: https://stackoverflow.com/users/1773798/renaud-pacalet listFiles() { local -n _OUTPUT_="$1" local -n _DIRECTORY_="$2" local -n _TYPES_="$3" local _FILTER_ _FILTER_="${_TYPES_[@]/#/ -o -name *.}" _FILTER_="${_FILTER_# -o }" while read -d $'\0' FILE; do _OUTPUT_+=( "$FILE" ) done < <( find "$_DIRECTORY_" -type f \( $_FILTER_ \) -print0 ) }
Example call:
DIR_PATH=/mnt/Movies TYPES=(mkv mp4 avi) listFiles FILES DIR_PATH TYPES
As the title would imply, the following script will recursively convert video files to conform to the following parameters:
Its main design was meant to convert recorded TV shows where a set quality should not necessarily exceed the mentioned parameters for the sake of storage conservation.
#!/bin/bash ########################################################################### ## Copyright (C) Wizardry and Steamworks 2021 - License: GNU GPLv3 ## ########################################################################### # The script is a way of converting all video files recursively in a # # directory tree in order to conform to the following parameters: # # * MPEG-4 (MP4) # # * 480p or less # # * AVC (H.264) # # * AAC # # # # Notes: # # * There are some added optimizations such as switching to just # # converting to MPEG-4 in case no more than a container change is # # necessary (ie: MKV to MP4). # # * This is a long-running script and best ran within a detachable # # terminal such as tmux or screen. # # * This is a bash script that will not run on other shells. # ########################################################################### ########################################################################### ## CONFIGURATION ## ########################################################################### # The path to the ffmpeg binary. FFMPEG='/usr/bin/ffmpeg' # The ffmpeg flags to use when converting. # veryfast vs. veryslow https://write.corbpie.com/ffmpeg-preset-comparison-x264-2019-encode-speed-and-file-size/ FFMPEG_OPTIONS=`tr -d '\012' <<'EOF' -c:v libx264 \ -crf 23 \ -level 3.1 \ -preset veryfast \ -tune film \ -sws_flags lanczos \ -sn \ -c:a libfdk_aac \ -vbr 5 \ -q:a 2 \ -ac 2 \ -b:a 128k \ -c:s mov_text \ -map 0:v:0 \ -map 0:a:0 \ -map_metadata -1 \ -movflags +faststart \ -strict -2 EOF` # The maximal height of the video files. FFMPEG_HEIGHT=480 # The maximal integral CPU percentage that will be allowed to be used. FFMPEG_CPU=600 # The file types to convert. TYPES=(mp4 mkv avi) ########################################################################### ## INTERNALS ## ########################################################################### if [ -z "$1" ] || [ ! -d "$1" ]; then echo "SYNTAX: $0 <DIRECTORY>" exit 1 fi DIR_PATH="$1" # By: https://stackoverflow.com/users/1773798/renaud-pacalet listFiles() { local -n _OUTPUT_="$1" local -n _DIRECTORY_="$2" local -n _TYPES_="$3" local _FILTER_ _FILTER_="${_TYPES_[@]/#/ -o -name *.}" _FILTER_="${_FILTER_# -o }" while read -d $'\0' FILE; do _OUTPUT_+=( "$FILE" ) done < <( find "$_DIRECTORY_" -type f \( $_FILTER_ \) -print0 ) } TEMP_FILE="" # Cleanup on file termination. trap '{ rm -rf "$TEMP_FILE" 2>&1 2>/dev/null exit 0 }' KILL QUIT TERM EXIT INT HUP # Iterate over all matching files. FILES=() listFiles FILES DIR_PATH TYPES for FILE in "${FILES[@]}"; do FILE_NAME=`basename "$FILE"` DIR_NAME=`dirname "$FILE"` FILE_EXTENSION=`echo "$FILE" | awk -F'.' '{ print $NF }'` TEMP_FILE="/tmp/${FILE_NAME/$FILE_EXTENSION/mp4}" INFO=`mediainfo "$FILE" | awk -F':' '{ print $1":"$2 }' | awk '{$1=$1};1'` HAS_AVC=`echo "$INFO" | grep 'Format : AVC'` HAS_AAC=`echo "$INFO" | grep 'Format : AAC LC'` HAS_MP4=`echo "$INFO" | grep 'Format : MPEG-4'` HAS_2CH=`echo "$INFO" | grep 'Channel(s) : 2'` HAS_TXT=`mediainfo $FILE | grep "^Text"` HEIGHT=`mediainfo --Inform="Video;%Height%" "$FILE"` # If all criteria matches then proceed to the next file. if [[ $HEIGHT -le $FFMPEG_HEIGHT ]] && \ [[ ! -z $HAS_AVC ]] && \ [[ ! -z $HAS_AAC ]] && \ [[ ! -z $HAS_MP4 ]] && \ [[ ! -z $HAS_2CH ]] && \ [[ -z $HAS_TXT ]] then echo "Skipping $FILE_NAME..." continue fi FFMPEG_ACTION=$FFMPEG_OPTIONS # Resize down to $FFMPEG_HEIGHT or less depending on the height of the file. if [[ $HEIGHT -gt $FFMPEG_HEIGHT ]]; then FFMPEG_SIZE="-vf scale=-2:$FFMPEG_HEIGHT" fi # Start the conversion. echo -n "Converting $FILE_NAME to $TEMP_FILE: " # -loglevel quiet "$FFMPEG" -hide_banner -loglevel quiet -i "$FILE" $FFMPEG_ACTION $FFMPEG_SIZE "$TEMP_FILE" </dev/null 2>/dev/null 2>&1 & FFMPEG_PID=$! cpulimit -q -z -e ffmpeg -l $FFMPEG_CPU >/dev/null 2>/dev/null 2>&1 & T1=`date +%s` wait $FFMPEG_PID T2=`date +%s` RUNTIME=$(( T2 - T1 )) T_H=$((RUNTIME / 3600)) T_M=$((RUNTIME % 3600 / 60 )) T_S=$(((RUNTIME % 3600) % 60 )) if [[ "$?" -eq 0 ]]; then echo "Done ($T_H:$T_M:$T_S)." # Permute the files on successful conversion. rm "$FILE" cp "$TEMP_FILE" "$DIR_NAME" if [ $? -ne 0 ]; then echo "Failed to copy converted file." continue fi rm "$TEMP_FILE" else echo "Failed." fi done
The following code allows a user to print out a list of established connections without having to resort to using the command-line tool netstat
by reading and processing /proc/net/tcp
. There are other versions out there that use awk
extensively in order to convert the hex values from /proc/net/tcp
and even smaller than the provided script albeit at the cost of readability.
#!/usr/bin/env bash ########################################################################### ## Copyright (C) Wizardry and Steamworks 2022 - License: GNU GPLv3 ## ## Please see: http://www.gnu.org/licenses/gpl.html for legal details, ## ## rights of fair usage, the disclaimer and warranty conditions. ## ########################################################################### # # # This script processes the TCP connections from /proc/net/tcp and will # # output a list of locally listening TCP IP addresses and ports (sockets) # # mapped to a list of remotely connecting TCP IP addresses and ports. # # # # The script is written with a minimalistic environment in mind with just # # a few tools available that should be present within most busyboxes. # # # # For best results, disable IPv6 on the executing machine (via the kernel # # command line) because some TCP connections are processed through IPv6 # # even if they are IPv4 such that /proc/net/tcp might not include them. # # # ########################################################################### decode () { DUO=$1 LEN=`echo -n $DUO | wc -c` OUT="" for X in `seq 0 2 $(($LEN-2))`; do OUT=$((16#${DUO:$X:2}))"."$OUT done OUT=${OUT::-1} echo $OUT } tcp_state() { OUT="" for SET in $1; do IP_HEX=`echo $SET | awk -F':' '{ print $1 }'` PO_HEX=`echo $SET | awk -F':' '{ print $2 }'` IP=`decode $IP_HEX` PO=$((16#$PO_HEX)) OUT=$OUT" "$IP:$PO done echo $OUT } TMP_STATE=/tmp/tcp_state trap '{ rm -rf $TMP_STATE; }' KILL QUIT TERM EXIT INT HUP cat /proc/net/tcp >$TMP_STATE TCP_STATE_LEN=`cat $TMP_STATE | wc -l` LISTEN_LOCAL=`cat $TMP_STATE | \ awk 'NR==1 { for (i=1;i<=NF;++i) if ($i=="local_address") { n=i; break }} { print $n }' | \ tail +2` REMOTE_ADDRS=`cat $TMP_STATE | \ awk 'NR==1 { for (i=1;i<=NF;++i) if ($i=="rem_address") { n=i; break }} { print $n }' | tail +2` LISTEN=`tcp_state "$LISTEN_LOCAL"` REMOTE=`tcp_state "$REMOTE_ADDRS"` for I in `seq 1 1 $(($TCP_STATE_LEN-1))`; do A=`echo $LISTEN | awk -v var=$I '{ print $var }'` B=`echo $REMOTE | awk -v var=$I '{ print $var }'` echo $A" - "$B done
The following script is meant to run in the background and poll /proc/net/tcp
for incoming TCP connections on a configurable port and then publish a JSON message to an MQTT broker with the following structure:
server -> FQDN hostname of the machine that the script is running on, this -> the listening IP address followed by the port separated by a colon that -> the connecting IP address followed by the port separated by a colon
#!/usr/bin/env bash ########################################################################### ## Copyright (C) Wizardry and Steamworks 2022 - License: GNU GPLv3 ## ## Please see: http://www.gnu.org/licenses/gpl.html for legal details, ## ## rights of fair usage, the disclaimer and warranty conditions. ## ########################################################################### # # # This script publishes the state of TCP connections to the machine over # # a specified port to an MQTT broker. The script leverages polling # # /proc/net/tcp for TCP connections instead of using netstat and uses # # just the most basic commands making it suitable for various cases. # # # ########################################################################### ########################################################################### # CONFIGURATION # ########################################################################### # path to the mosquitto publishing tool MOSQUITTO_PUBLISHER=/storage/.local/usr/local/bin/mosquitto_pub # the host name of the MQTT broker MOSQUITTO_HOST=iot.internal # the MQTT topic to publish on MOSQUITTO_TOPIC=arcade # the TCP port to monitor for connections MONITOR_PORT=55435 # the time to sleep between polling SLEEP_TIME=1 ########################################################################### # INTERNALS # ########################################################################### decode () { DUO=$1 LEN=`echo -n $DUO | wc -c` OUT="" for X in `seq 0 2 $(($LEN-2))`; do OUT=$((16#${DUO:$X:2}))"."$OUT done OUT=${OUT::-1} echo $OUT } tcp_state() { OUT="" for SET in $1; do IP_HEX=`echo $SET | awk -F':' '{ print $1 }'` PO_HEX=`echo $SET | awk -F':' '{ print $2 }'` IP=`decode $IP_HEX` PO=$((16#$PO_HEX)) OUT=$OUT" "$IP:$PO done echo $OUT } TMP_STATE=/dev/shm/tcp_state RUN=1 trap '{ rm -rf $TMP_STATE; RUN=0; }' KILL QUIT TERM EXIT INT HUP SIGUSR1 while [ $RUN -ne 0 ]; do cat /proc/net/tcp >$TMP_STATE TCP_STATE_LEN=`cat $TMP_STATE | wc -l` LISTEN_LOCAL=`cat $TMP_STATE | \ awk 'NR==1 { for (i=1;i<=NF;++i) if ($i=="local_address") { n=i; break }} { print $n }' | \ tail +2` REMOTE_ADDRS=`cat $TMP_STATE | \ awk 'NR==1 { for (i=1;i<=NF;++i) if ($i=="rem_address") { n=i; break }} { print $n }' | tail +2` LISTEN=`tcp_state "$LISTEN_LOCAL"` REMOTE=`tcp_state "$REMOTE_ADDRS"` for I in `seq 1 1 $(($TCP_STATE_LEN-1))`; do X=`echo $LISTEN | awk -v var=$I '{ print $var }'` Y=`echo $REMOTE | awk -v var=$I '{ print $var }'` NETPLAY=`echo $X $Y | grep $MONITOR_PORT | grep -v 0.0.0.0` if [ ! -z "$NETPLAY" ]; then THIS=`echo $NETPLAY | awk '{ print $1 }'` THAT=`echo $NETPLAY | awk '{ print $2 }'` $MOSQUITTO_PUBLISHER \ -h $MOSQUITTO_HOST \ -t $MOSQUITTO_TOPIC \ -m "{ \"origin\": \"`hostname -f`\", \"this\": \"$THIS\", \"that\": \"$THAT\" }" fi done sleep $SLEEP_TIME done
In order to use the text activity spinners in bash, here is an example script:
#!/usr/bin/env bash ASCII_SPINNER=( ".oOo" "oOo." "Oo.o" "o.oO" ) while [ 1 = 1 ]; do CAR="${ASCII_SPINNER[0]}" CDR=("${ASCII_SPINNER[@]:1:${#ASCII_SPINNER[@]}}") ASCII_SPINNER=(${CDR[@]} ${CAR}) echo -n -e "\r${CAR}" done
The logic is pretty straight-forward:
\r
in order to make sure that the next element in the spinner array will overwrite the old element