Differences

This shows you the differences between two versions of the page.

@@ Line 1: / Line 1: @@
+====== Distribution Timeline ======
+{{fuss:fuss_linux_distribution_timeline.png?512}}
+====== Initd Skeleton Script ======
+<code bash>
+#!/bin/bash
+case "$1" in
+  start)
+    ;;
+  stop)
+    PID=`ps -ax | grep '[D]AEMON' | awk '{print $1}'`
+    ;;
+  *)
+    echo "Usage: $0 {start|stop}"
+    exit 1
+    ;;
+esac
+exit 0
+</code>
+where ''DAEMON'' is the name of the daemon the script is supposed to manage.
+====== Basic Firewall ======
+<code bash>
+#!/bin/sh
+LOCAL_IF="eth1"
+NET_IF="eth0"
+iptables -F
+iptables -t nat -F
+iptables -X
+iptables -P INPUT DROP
+# Accept local network
+iptables -A INPUT -i $LOCAL_IF -j ACCEPT
+# and loopback.
+iptables -A INPUT -i lo -j ACCEPT
+# accept established, related
+iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
+# masquerade
+iptables -t nat -A POSTROUTING -o $NET_IF -j MASQUERADE
+# ip-forwarding
+echo "1" >/proc/sys/net/ipv4/ip_forward
+</code>
+====== Stop Udev from Renaming Interfaces ======
+On Debian-like Linux systems, including Ubuntu, Udev by default keeps track of the MAC address of network interfaces. If you happen to replace a network card, the operating system increments the interface number instead of reporting just the cards that it finds in the computer at that time. To stop this behavior, the following Udev ruleset can be eliminated:
+<code bash>
+echo "" > /etc/udev/rules.d/70-persistent-net.rules
+</code>
+After a reboot, Udev will stop renaming the interfaces (as it should have done from the start).
+A different way to stop Linux from changing the interface names is to append:
+<code>
+net.ifnames=0
+</code>
+to the kernel command line (for grub, by editing ''/etc/default/grub'' and adding it to ''GRUB_CMDLINE_LINUX_DEFAULT'').
+====== Crontab Diagram ======
+<code>
+*  *  *  *  * command to execute
+|  |  |  |  |
+|  |  |  |  +-- day of week (0-7) (Sunday=0 or 7)
+|  |  |  +----- month (1-12)
+|  |  +-------- day of month (1-31)
+|  +----------- hour (0-23)
++-------------- minute (0-59)
+</code>
+====== Get IP of Interface ======
+The following command will return the IP address of the interface ''eth0'':
+<code bash>
+/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'
+</code>
+====== Routing Packets Out of the Same Interface ======
+A common problem on linux is that packets coming in from an interface do not necessarily get a reply from a server out of the same interface that they came in from. In order to fix this, we have to set-up a few routing tables by editing ''/etc/iproute2/rt_tables'' and adding, for example, two tables:
+<code>
+     table_a
+     table_b
+</code>
+then, we can route the packets out of the same interface that they came in from using:
+<code bash>
+ip route add default via $GATEWAY_A dev $INTERFACE_A src $IP_A table table_a
+ip rule add from $IP_A table table_b
+</code>
+where:
+  * ''$GATEWAY_A'' is the gateway IP for an interface.
+  * ''$INTERFACE_A'' is the interface that the packets come in from.
+  * ''$IP_A'' is the IP address assigned to the interface.
+====== Prelink ======
+To prelink binaries, using the ''prelink'' tool:
+<code bash>
+prelink -amR
+</code>
+To restore:
+<code bash>
+prelink -au
+</code>
+====== Enable Directory Indexing ======
+<code bash>
+tune2fs  -O dir_index /dev/sda2
+</code>
+Where ''/dev/sda2'' contains an Ext3 filesystem.
+====== Get Top 10 CPU Consuming Processes ======
+<code bash>
+ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
+</code>
+====== Recover Linux Password ======
+Made by [[http://www.youtube.com/watch?v=8RCBw3phhAY|Blagovest ILIEV]] and adapted to GIF animation.
+{{linux_recover_password.gif}}
+====== Recompile Custom Kernel for Debian ======
+After downloading the source, applying the necessary patches, issue:
+<code bash>
+make menuconfig
+</code>
+and configure the kernel. After that issue:
+<code bash>
+make-kpkg --initrd kernel_image
+</code>
+to make a ''.deb'' package which will be placed one level up from the kernel source directory.
+Note that building an ''initrd'' image is essential because it contains the necessary drivers to bootstrap the boot system. If you recompile manually, the old way, Debian will not boot.
+====== Tripwire Regenerate Configuration Files ======
+After modifying the database configuration at ''/etc/tripwire/twpol.txt'', the following script can be used to regenerate the database:
+<code bash>
+#!/bin/sh -e
+twadmin -m P -S site.key twpol.txt
+twadmin -m F -S site.key twcfg.txt
+tripwire -m i
+</code>
+====== Create UDEV Symlink ======
+For example to link any ''/dev/grsec'' device to ''/dev/grsec2'', add a file at ''/etc/udev/rules.d/60-grsec-compatiblity.rules'' with the following contents:
+<code>
+KERNEL=="grsec*", SYMLINK+="grsec2"
+</code>
+====== Inherit Group Ownership ======
+Suppose you have a parent directory ''upper'', and that the directory ''upper'' is group-owned by a group called ''maintain''.
+You want that all new directories and files under that parent directory ''upper'', regardless by whom they are created (ie: ''root'') to be group-owned by ''maintain''.
+This can be accomplished by setting the set-guid flag on the parent directory ''upper'':
+<code bash>
+chmod g+s upper
+</code>
+====== Guess Module Configuration for Compiling Kernels ======
+''localmodconfig'' can be used to auto-detect the necessary modules for the kernel.
+<code bash>
+make localmodconfig
+</code>
+====== Create Bootable USB ======
+For a disk with the following partition layout:
+<ditaa>
+  +-----------+
+  | /dev/sdc  |
+  |           |
+  |-----------|
+  | /dev/sdc1 |
+  +-----------+
+</ditaa>
+first install ''syslinux'' and then issue:
+<code bash>
+mkdosfs -F32 /dev/sdc1
+</code>
+to format ''/dev/sdc1'' to MS-DOS.
+Now copy the ''MBR'' file to the drive:
+<code bash>
+dd if=/usr/share/syslinux/mbr.bin of=/dev/sdc
+</code>
+Finally, install ''syslinux'':
+<code bash>
+syslinux /dev/sdc1
+</code>
+Next step is to make the disk bootable with ''fdisk'' (run fdisk /dev/sdc and press ''a'' to toggle the bootable flag).
+====== Get Page Size ======
+<code bash>
+getconf PAGE_SIZE
+</code>
+====== Measuring Performance ======
+This can be accomplished with:
+<code bash>
+dstat -t -c 5 500
+</code>
+where ''t'' indicates time-based output and ''c'' stands for ''CPU''.
+The output is:
+<code>
+----system---- ----total-cpu-usage----
+     time     |usr sys idl wai hiq siq
+-02 18:33:24|  3   1  95   0   0   0
+-02 18:33:29| 14   3  83   0   0   0
+</code>
+Other options are also available:
+^ Flag ^ Meaning ^
+| ''c'' | ''CPU''    |
+| ''d'' | disk (read, write) |
+| ''g'' | page stats (in, out) |
+| ''i'' | interrupts |
+| ''l'' | load (1min, 5min, 15min) |
+| ''m'' | memory (used, buffers, cache, free) |
+| ''n'' | network (receive, send) |
+| ''p'' | process stats (runnable, uninterruptible, new) |
+| ''r'' | ''I/O'' request stats (read, write) |
+| ''s'' | swap stats (used, free) |
+| ''y'' | system stats (interrupts, context switches) |
+| ''aio'' | asynchronous ''I/O'' |
+| ''fs'' | filesystem (open files, inodes) |
+| ''ipc'' | IPC stats (queues, semaphores, shared memory) |
+| ''lock'' | file locks (posix, flock, read, write) |
+| ''raw'' | raw sockets |
+| ''socket'' | sockets (total, tcp, udp, raw, ip-fragments) |
+| ''tcp'' | tcp stats (listen, established, syn, time_wait, close) |
+| ''udp'' | udp stats (listen, active) |
+| ''unix'' | unix stats (datagram, stream, listen, active) |
+| ''vm'' | vm stats (hard pagefaults, soft pagefaults, allocated, free) |
+====== Renumber Partitions ======
+To renumber partitions we first dump the table using ''sfdisk'':
+<code bash>
+sfdisk -d /dev/sda > sda.table
+</code>
+then, we edit ''sda.table'' to edit the partitions:
+<code>
+# partition table of /dev/sda
+unit: sectors
+/dev/sda1 : start=      2048, size=  4194304, Id=82
+/dev/sda2 : start=         0, size=        0, Id= 0
+/dev/sda3 : start=   4196352, size= 47747546, Id=83, bootable
+</code>
+In this case, we will delete the line starting with ''/dev/sda2'' and rename ''/dev/sda3'' to ''/dev/sda2'':
+<code>
+# partition table of /dev/sda
+unit: sectors
+/dev/sda1 : start=      2048, size=  4194304, Id=82
+/dev/sda2 : start=   4196352, size= 47747546, Id=83, bootable
+</code>
+Next, we restore the modified table:
+<code bash>
+sfdisk /dev/sda < sda.table
+</code>
+====== Show Socket State Counters ======
+<code bash>
+netstat -an | awk '/^tcp/ {A[$(NF)]++} END {for (I in A) {printf "%5d %s\n", A[I], I}}'
+</code>
+====== Scrolling Virtual Terminal ======
+To scroll the virtual terminal up and down use the keys <key>Shift-PageUp</key> and <key>Shift-PageDown</key>. In case you are using a Mac keyboard without <key>PageUp</key> or <key>PageDown</key>, then the keys <key>Shift-Fn-Up</key> and <key>Shift-Fn-Down</key> should achieve the scrolling.
+====== Set Date and Time ======
+''date'' can be used to set the system clock, however ''hwclock'' has to also be used to set the hardware clock. First we set a date using ''date'':
+<code bash>
+date -s "1 MAY 2013 10:15:00"
+</code>
+or in two commands using formatting characters; first the date:
+<code bash>
+date +%Y%m%d -s "20130501"
+</code>
+then the time:
+<code bash>
+date +%T -s "10:15:00"
+</code>
+After that, the hardware clock has to be set (the hardware clock runs independent of the Linux time and of other hardware, powered by a battery). To set the hardware clock to the system clock (since we have already done that above), issue:
+<code bash>
+hwclock --systohc
+</code>
+Or, as an independent command, to set the hardware clock to local time:
+<code bash>
+hwclock --set --date="2013-05-01 10:15:00" --localtime
+</code>
+of for ''UTC'':
+<code bash>
+hwclock --set --date="2013-05-01 10:15:00" --utc
+</code>
+====== Load Average ======
+The load-average is included in the ''uptime'' command:
+<code>
+:48:35 up 8 days,  7:03,  5 users,  load average: 0.24, 0.28, 0.25
+</code>
+The load average numbers are scaled up to the number of ''CPU''s. For example, on a quad-core CPU, the maximal load-average (when all 4 cores are busy) would be ''4''. The numbers thus represent only a fraction of the total CPU power that is currently being utilised.
+====== Override DHCP Client Nameservers ======
+''dhclient'' is responsible in most Linux distributions for acquiring the ''DHCP'' parameters from upstream ''DHCP'' servers. The configuration can be altered to not pull name-servers and instead prepend some static name-servers. The configuration has to changed such that the domains a prepended:
+<code>
+prepend domain-name-servers 1.1.1.1, 2.2.2.2;
+</code>
+where ''1.1.1.1'' and ''2.2.2.2'' represent name-servers ''IP'' addresses.
+Next, the ''domain-name-servers'' and ''domain-search'' directives should be removed from the ''request'' clause, the result looking like:
+<code>
+request subnet-mask, broadcast-address, time-offset, routers,
+        domain-name, host-name,
+        dhcp6.name-servers, dhcp6.domain-search,
+        netbios-name-servers, netbios-scope, interface-mtu,
+        rfc3442-classless-static-routes, ntp-servers;
+</code>
+After a restart ''dhclient'' will prepend the specified name-servers and place them in ''/etc/resolv.conf'' as well as ignoring the ''DHCP'''s settings for the ''domain-name-servers'' and ''domain-search'' directives.
+====== Using the Temporary Memory Filesystem ======
+The temporary memory filesystem (''tmpfs'') can be used when you want to temporary store files that will be deleted on the next reboot. This is helpful, for example, when you want to store log-files that are not important over reboots and want to reduce the pressure on the hard-drive.
+Adding this entry to ''/etc/fstab'' will, for example, mount ''polipo'''s cache directory in ''RAM'':
+<code>
+tmpfs        /var/cache/polipo       tmpfs nodev,noexec,nodiratime,noatime,nosuid,size=5G,async      0       0
+</code>
+using a slab of ''5G''.
+====== Dynamically Limiting a Processes CPU on Network Idling ======
+This function works together with ''iptables'' and the ''IDLETIMER'' module in order to limit the ''CPU'' consumption of a process (commonly a daemon) when the process does not generate incoming traffic.
+<code bash>
+#!/bin/bash
+###########################################################################
+##  Copyright (C) Wizardry and Steamworks 2014 - License: GNU GPLv3      ##
+##  Please see: http://www.gnu.org/licenses/gpl.html for legal details,  ##
+##  rights of fair usage, the disclaimer and warranty conditions.        ##
+###########################################################################
+# The function suspends or resumes the named process passed as parameter to
+# the fuction, provided that iptables has been set-up to create an idle
+# timer for the named process passed as the parameter to this function.
+#
+# For this function to work properly, you should issue:
+# iptables -A INPUT -p tcp --dport 8085 -j IDLETIMER \
+#	--timeout 60 --label $process_name
+# where $process_name is the parameter passed to this function
+#
+# This script is best called via crontab to periodically check whether a
+# proccess's network traffic is stale and to suspend the process if it is.
+function idlecpulimit {
+    # path to the cpulimit daemon
+    local cpulimit=/usr/bin/cpulimit
+    # percent to throttle to accounting for multiple CPUs
+    # effective throttle = (CPUs available) x throttle
+    local throttle=1
+    # get the car and cdr of the daemon
+    local car=`echo $1 | cut -c 1`
+    local cdr=`echo $1 | cut -c 2-`
+    # get the daemon if it is running
+    local daemon=`ps ax | grep "[$car]$cdr" | awk '{ print $1 }'`
+    if [ -z $daemon ]; then
+    	# just bail if it is not running
+    	return;
+    fi
+    # get the PID of the cpulimit daemon for the process
+    local cpulimit_PID=`ps ax | grep '[c]pulimit' | grep $daemon | awk '{ print $1 }'`
+    case `cat /sys/class/xt_idletimer/timers/$1` in
+)
+            # suspend
+            if [ -z $cpulimit_PID ]; then
+                $cpulimit -l $throttle -p $daemon -b >/dev/null 2>&1
+            fi
+            ;;
+        *)
+            # resume
+            if [ ! -z $cpulimit_PID ]; then
+                kill -s TERM $cpulimit_PID >/dev/null 2>&1
+            fi
+            ;;
+    esac
+}
+</code>
+As an example, suppose you had a daemon named ''mangosd'' and, as a daemon, it is active when it has inbound connections on port ''8085''. In that case, you would first add a firewall rule:
+<code bash>
+iptables -A INPUT -p tcp --dport 8085 -j IDLETIMER --timeout 60 --label mangosd
+</code>
+which will start a countdown timer in ''/sys/class/xt_idletimer/timers/mangosd'' when the connection is idle.
+After that, you would create a script containing the function above and call it in your script:
+<code bash>
+function idlecpulimit {
+    ...
+}
+idlecpulimit mangosd
+</code>
+The script will then be placed in ''/etc/cron.d/cron.minutely'' and will limit or release the ''CPU'' limitation when the daemon receives traffic.
+====== Rescue Mount ======
+Suppose that you have made a configuration error and you need to boot from a ''LiveCD'' and ''chroot'' to the filesystem in order to repair the damage. In that case, you will find that you will need the ''proc'', ''dev'' and ''sys'' filesystems. These can be mounted by using the ''bind'' option of ''mount'':
+<code bash>
+mount -o bind /dev /mnt/chroot/dev
+mount -o bind /sys /mnt/chroot/sys
+mount -o bind /proc /mnt/chroot/proc
+</code>
+Considering that the damaged filesystem is mounted on ''/mnt/chroot''. After the filesystems are mounted, you can ''chroot'' to the filesystem and run commands such as ''update-grub'':
+<code bash>
+chroot /mnt/chroot
+</code>
+====== Get Communicating MAC Addresses ======
+<code bash>
+tcpdump -i eth0 -s 30 -e | cut -f1 -d','
+</code>
+where ''eth0'' is the interface.
+====== Kernel Stack Traceback ======
+For hung processes, the stack traceback can show where the processes are waiting. The ''CONFIG_MAGIC_SYSRQ'' must be enabled in the kernel to enable stack tracebacks. If ''kernel.sysrq'' is not set to ''1'' with ''sysctl'', then run:
+<code bash>
+echo 1 > /proc/sys/kernel/sysrq
+</code>
+Next, trigger the stack traceback by issuing:
+<code bash>
+echo t > /proc/sysrq-trigger
+</code>
+The results can be found on the console or in ''/var/log/messages''.
+====== Check Processes Listening on IPv6 Addresses ======
+<code bash>
+netstat -tunlp |grep p6
+</code>
+====== Disable IPv6 ======
+First edit ''/etc/hosts'' to comment out any IPv6 addresses:
+<code>
+# The following lines are desirable for IPv6 capable hosts
+#::1     ip6-localhost ip6-loopback
+#fe00::0 ip6-localnet
+#ff00::0 ip6-mcastprefix
+#ff02::1 ip6-allnodes
+#ff02::2 ip6-allrouters
+</code>
+After that, if you are using grub, edit ''/etc/default/grub'' and add:
+<code>
+ipv6.disable=1
+</code>
+to the list following ''GRUB_CMDLINE_LINUX_DEFAULT''.
+In case you use lilo, edit ''/etc/lilo.conf'' instead and modify the ''append'' line to include ''ipv6.disable=1''.
+Issue ''update-grub'' or ''lilo'' to make those changes.
+You can also add a sysctl setting:
+<code>
+net.ipv6.conf.all.disable_ipv6 = 1
+</code>
+to ''/etc/sysctl.d/local.conf''.
+Additionally, in case you are running a system with a bundled MTA such as exim, you should probably keep it from binding to IPv6 addresses.
+For exim, edit ''/etc/exim4/update-exim4.conf.conf'' and change the ''dc_local_interfaces'' to listen only on IPv4:
+<code>
+dc_local_interfaces='127.0.0.1'
+</code>
+and then add:
+<code>
+# Disable IPv6
+disable_ipv6 = true
+</code>
+in both ''/etc/exim4/exim4.conf.template'' and ''/etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs'' and run ''update-exim4.conf'' followed by a restart of the service.
+Otherwise you might receive the error: ''ALERT: exim paniclog /var/log/exim4/paniclog has non-zero size, mail system possibly broken failed!''.
+====== Clear Semaphores ======
+''ipcs'' can be used to display all semaphores:
+<code bash>
+ipcs -s
+</code>
+to remove a semaphore by id, issue:
+<code bash>
+ipcrm sem 2123561
+</code>
+To clear all semaphores for a user, for example, for apache (as user ''www-data'' on Debian):
+<code bash>
+ipcs -s | grep www-data | awk '{ print $2 }' | while read i; do ipcrm sem $i; done
+</code>
+====== WatchDog Error Messages ======
+Before the watchdog restarts the system, it fires off an email indicating the problem, for example:
+<code>
+Message from watchdog:
+The system will be rebooted because of error -3!
+</code>
+The error codes can be found in the man page, here is a list of reasons:
+  * ''-1'' The system will reboot. Does not indicate an error.
+  * ''-2'' The system will reboot. Does not indicate an error.
+  * ''-3'' The load average has reached its maximum specified value.
+  * ''-4'' Temperature too high.
+  * ''-5'' ''/proc/loadavg'' contains no data or not enough data.
+  * ''-6'' The given file was not changed in the given interval.
+  * ''-7'' ''/proc/meminfo'' content.
+  * ''-8'' Child process was killed by a signal.
+  * ''-9'' Child process did not return in time.
+  * ''-10'' User specified.
+====== Codel over Wondershaper ======
+On recent Linux distributions, Codel can be enabled which is better than wondershaper. This can be done by editing the sysctl configuration file (''/etc/sysctl.d/local.conf'') and adding the line:
+<code>
+net.core.default_qdisc = fq_codel
+</code>
+for general-purpose routers including virtual machine hosts and:
+<code>
+net.core.default_qdisc = fq
+</code>
+for fat servers.
+====== Granting Users Permissions to Files ======
+Using POSIX ACLs, it is possible to modify permissions to files (even recursively) such that it is no longer necessary to fiddle with the limited Linux user and group permissions. For example, suppose you wanted to allow a user access to a directory without adding them to a group and then separately modifying all the file permissions to allow that group access.
+In that case, you would write:
+<code bash>
+setfacl -R -m u:bob:rwX Share
+</code>
+where:
+  * ''-R'' means to recursively change the permissions
+  * ''-m'' means modify (and ''-x'' means to remove)
+  * ''u:'' stands for user (and ''g:'' for group)
+  * ''bob'' is the user that we want to grant access to
+  * ''rwX'' means read (''r''), write (''w'') and ''X'' (note the capital case) means to only grant execute permissions in case the file already had execute permissions
+  * ''Share'' is the directory (or file) to set the permissions on
+The command will thus recursively grant permissions on the file or folder named ''Share'' to the user ''bob'' allowing ''bob'' to read, write and execute the files but only if the file was executable in the first place.
+====== Change the Default Text Editor ======
+The following command will let you pick the default editor:
+<code bash>
+update-alternatives --config editor
+</code>
+====== Print all Open Files Sorted By Number of File Handles ======
+<code bash>
+find /proc/*/fd -xtype f -printf "%l\n" | grep -P '^/(?!dev|proc|sys)' | sort | uniq -c | sort -n
+</code>
+====== Reboot Hanging Machine ======
+In case the machine is hanging and Magic SysRq is enabled in the kernel (enabled by default), then issuing the following combination will reboot the machine more or less gracefully:
+<key>A-PrtScrn-R-s-E-I-U-B</key>
+which will perform, in order:
+  - <key>R</key> give control back to the keyboard
+  - <key>s</key> sync
+  - <key>E</key> sends all processes but ''init'' the ''TERM'' signal
+  - <key>I</key> sends all processes but the ''init'' the ''KILL'' signal
+  - <key>U</key> mounts all filesystems to read-only to prevent ''fsck'' at boot
+  - <key>B</key> reboots the system
+====== Check Solid State Drive for TRIM ======
+To check whether an attached SSD currently has TRIM enabled, first mount the drive and change directory to the drive:
+<code bash>
+cd /mnt/ssd
+</code>
+Now create a file:
+<code bash>
+dd if=/dev/urandom of=tempfile count=100 bs=512k oflag=direct
+</code>
+and check the fib-map:
+<code bash>
+hdparm --fibmap tempfile
+</code>
+which will output something like:
+<code>
+tempfile:
+ filesystem blocksize 4096, begins at LBA 2048; assuming 512 byte sectors.
+ byte_offset  begin_LBA    end_LBA    sectors
+  383099904  383202303     102400
+</code>
+Now, note the number under ''begin_LBA'' (''383099904'' in this example) and run:
+<code bash>
+hdparm --read-sector 383099904 /dev/sdc
+</code>
+where:
+  * ''383099904'' is the number obtained previously
+  * ''/dev/sdc'' is the device for the SSD drive
+The last command should output a long string of characters for those sectors.
+Now, issue:
+<code bash>
+rm tempfile
+sync
+</code>
+and repeat the previous ''hdparm'' command:
+<code bash>
+hdparm --read-sector 383099904 /dev/sdc
+</code>
+if now the output consists of only zeroes then automatic TRIM is in place otherwise, wait for a while and run the last ''hdparm'' again.
+====== Automatically Mount Filesystems on Demand ======
+On distributions based on ''systemd'', filesystems can be mounted on demand instead of using ''/etc/fstab'' in order to let the main system boot while all the requests to the ''systemd'' managed filesystems can buffer-up.
+Suppose you have a ''/home'' partition that you want mounted on demand with ''systemd''. In that case, you can modify the ''/etc/fstab'' options to read:
+<code bash>
+noauto,x-systemd.automount
+</code>
+where ''noauto'' prevents Linux from mounting the partition on boot and ''x-systemd.automount'' will use ''systemd'' to auto-mount the partition on demand.
+Additionally, the parameter ''x-systemd.device-timeout=1min'' can be added to the mount options which will allow for 1 minute before giving up trying to mount the resource which can be useful for network-mounted devices.
+====== Automatically Reboot after Kernel Panic ======
+In order to have Linux automatically reboot after a kernel panic, add a setting to sysctl - on Debian systems, you will have to edit the file ''/etc/sysctl.d/local.conf'':
+<code>
+kernel.panic = 30
+kernel.panic_on_oops = 30
+</code>
+which will make the machine restart in ''30'' seconds.
+====== List Top Memory Consuming Processes ======
+<code bash>
+ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | less
+</code>
+====== Get the Most Frequently Used Commands ======
+The following snippet pipes the second field from the ''history'' command and counts the number of time it appears:
+<code bash>
+history | awk '{ a[$2]++ } END { for(i in a) { print a[i] " " i } }' | sort -urn | head -n 20
+</code>
+which then gets sorted and the top most ''20'' results are displayed.
+====== Force a Filesystem Check on Reboot ======
+You can add: ''fsck.mode=force'' and ''fsck.repair=preen'' to the grub parameter line on reboot in order to trigger a filesystem recheck. Alternatively, if you feel bold, you can add ''fsck.repair=yes'' instead of ''fsck.repair=preen'' in order to have Linux automatically fix the errors. This is especially useful to recover from a damaged root filesystem.
+====== Enable Multi-Queue Block IO Queuing Mechanism ======
+Edit ''/etc/default/grub'' and add:
+<code>
+scsi_mod.use_blk_mq=1
+</code>
+to the kernel command line parameters.
+====== Export Linux Passwords ======
+This helper script can be useful in case you wish to export a bunch of "real" users by scanning the home directory and extracting only users that have a folder inside that directory.
+<file bash exportusers.sh>
+###########################################################################
+##  Copyright (C) Wizardry and Steamworks 2016 - License: GNU GPLv3      ##
+###########################################################################
+HOMES="/home"
+FILES="/etc/passwd /etc/passwd- /etc/shadow /etc/shadow-"
+ls -l $HOMES | awk '{ print $3 }' | sort -u | while read u; do
+    for file in $FILES; do
+        cat $file | while read p; do
+            ENTRY=`echo $p | awk -F':' '{ print $1 }'`
+            if [ "$ENTRY" == "$u" ]; then
+	        echo $p >> `basename $file`
+            fi
+        done
+    done
+done
+</file>
+When the script runs, it will scan all folders under the ''/home'' directory, grab the users that the folders belong to and scan the Linux password files (''/etc/passwd'', ''/etc/passwd-'', ''/etc/shadow'' and ''/etc/shadow-'') for entries for that user. It will then generate Linux password files from the matching home directories that can be later inserted into the Linux password files of a different machine.
+====== Create Sparse Image of Device ======
+''dd'' dumps an entire device but has no options that are aware of the number of zeroes on the device. The following command:
+<code bash>
+dd if=/dev/sda | cp --sparse=always /dev/stdin image.img
+</code>
+will create an image named ''image.img'' of the device ''/dev/sda'' such that the image file will not contain any zeroes.
+To check that the image was created successfully, you can then issue:
+<code bash>
+md5sum image.img
+</code>
+and
+<code bash>
+md5sum /dev/sda
+</code>
+and check that the hashes are identical.
+====== Bind to Reserved Ports as Non-Root User ======
+Binding to reserved ports (ports under ''1024'') can be done under Linux by issuing:
+<code bash>
+setcap 'cap_net_bind_service=+ep' /path/to/binary
+</code>
+====== Mount Apple Images ======
+DMG files are usually compressed; in fact, if you issue in a terminal:
+<code bash>
+file someimage.dmg
+</code>
+you may get output such as:
+<code>
+someimage.dmg: bzip2 compressed data, block size = 100k
+</code>
+indicating a bzip2 compressed file, or:
+<code>
+someimage.dmg: zlib compressed data
+</code>
+You can then uncompress the DMG image under Linux by issing:
+<code bash>
+bzip -dc someimage.dmg > someimage.dmg.uncompressed
+</code>
+Now, if you inspect the uncompressed image (in this example ''someimage.dmg.uncompressed''):
+<code bash>
+file someimage.dmg.uncompressed
+</code>
+you will get some interesting info such as:
+<code bash>
+someimage.dmg.uncompressed: Apple Driver Map, blocksize 512, blockcount 821112, devtype 0, devid 0, descriptors 0, contains[@0x200]: Apple Partition Map, map block count 3, start block 1, block count 63, name Apple, type Apple_partition_map, contains[@0x400]: Apple Partition Map, map block count 3, start block 64, block count 861104, name disk image, type Apple_HFS, contains[@0x600]: Apple Partition Map, map block count 3, start block 811148, block count 4, type Apple_Free
+</code>
+indicating an uncompressed image.
+To convert the DMG into an image that can be mounted, you can use the tool''dmg2img'':
+<code bash>
+dmg2img someimage.dmg someimage.dmg.uncompressed
+</code>
+You can now mount the image using the HFS+ filesystem:
+<code bash>
+mount -t hfsplus -o ro someimage.dmg.uncompressed /mnt/media
+</code>
+====== Purge all E-Mails from Command-Line ======
+To purge all inbox e-mails on Linux from the command line, you can use the ''mail'' command with the following sequence of instructions:
+<code>
+d *
+q
+</code>
+where:
+  * ''mail'' is the mail reader program,
+  * ''d *'' instructs ''mail'' to delete all messages,
+  * ''q'' tells ''mail'' to quit
+====== Remount Root Filesystem as Read-Write ======
+There are cases where a Linux system boots with the root ''/'' mounted as read-only. This can occur for various reasons but the standard way of recovering is to issue:
+<code bash>
+mount -o remount,rw /
+</code>
+which should mount the root filesystem in read-write mode.
+However, assuming that you have bad options in ''/etc/fstab'', that will not work and you will get errors in ''dmesg'' along the lines of:
+<code>
+Unrecognized mount option ... or missing value
+</code>
+this is due to ''mount'' reading ''/etc/fstab'' when you do not specify the source and the target. To work around the problem, you can mount the root manually by specifying both:
+<code bash>
+mount -t ext4 /dev/vda1 / -o remount,rw
+</code>
+which should give you enough leverage to adjust the entries in your ''/etc/fstab'' file.
+====== Enable Metadata Checksumming on EXT4 ======
+Metadata checksumming provides better data safety protection - you will need e2fsprogs version ''1.43'' or beyond. On Debian you can check your current e2fsprogs with ''apt-cache policy e2fsprogs'' and upgrade to ''unstable'' or ''testing'' if needed.
+On new systems, to enable metadata checksumming at format time, you would issue:
+<code bash>
+mkfs.ext4 -O metadata_csum /dev/sda1
+</code>
+where:
+  * ''/dev/sda1'' is the path to the device to be formatted as EXT4
+On existing systems, the filesystem must be unmounted first (using a LiveCD, for instance). With the filesystem unmounted and assuming that ''/dev/sda1'' contains the EXT4 filesystem for which metadata checksumming is to be enabled, issue:
+<code bash>
+e2fsck -Df /dev/sda1
+</code>
+in order to optimise the filesystem; followed by:
+<code bash>
+resize2fs -b /dev/sda1
+</code>
+to convert the filesystem to 64bit and finally:
+<code bash>
+tune2fs -O metadata_csum /dev/sda1
+</code>
+to enable metadata checksumming.
+If you want, you can see the result by issuing:
+<code bash>
+dumpe2fs -h /dev/sda1
+</code>
+Now that metadata checksumming is enabled, you may have some performance gain by adding the a module to initrd called ''crypto-crc32c'' that will enable hardware acceleration for the CRC routines. On Debian, adding the ''crypto-crc32c'' module to initrd is a matter of [[fuss/debian#add_modules_to_initd|editing a file and rebuilding the initramfs]].
+====== Get a List of IMAP Users from Logs ======
+The following command will read in ''/var/log/mail.log'' and compile a list of unique IMAP users.
+<code bash>
+cat /var/log/mail.log | \
+  grep imap-login:\ Login | \
+  sed -e 's/.*Login: user=<\(.*\)>, method=.*/\1/g' | sort | uniq
+</code>
+====== Disable Console Blanking ======
+To disable the linux console blanking (turning off), the following methods can be mentioned:
+  * append ''consoleblank=0'' to the linux kernel parameters (ie: edit ''/etc/default/grub'' on Debian),
+  * issue ''setterm -blank 0 -powerdown 0'' on the console to turn off blanking on,
+  * issue ''echo -ne "\033[9;0]" >/dev/ttyX''; where ''X'' is the console number to turn off blanking for,
+  * issue ''echo -ne "\033[9;0]" >/etc/issue'' to turn off blanking (''/etc/issue'' is loaded on console boot).
+Note that ''setterm -blank 0'' and ''echo -ne "\033[9;0]"'' are equivalent such that you can redirect both their output to a ''tty'' device.
+====== Manipulating Linux Console ======
+Most console-oriented commands that are meant to work on virtual terminals expect a proper terminal to be set up and to be executed on a virtual terminal. The ''openvt'' command can be used to execute a program on the Linux virtual terminal. For instance, to force the screen to blank whilst being logged-in via an SSH session (''/dev/pts''), issue:
+<code bash>
+TERM=linux openvt -c 1 -f -- setterm -blank force
+</code>
+where:
+  * ''TERM=linux'' sets the terminal type to ''linux'' otherwise the terminal type of the terminal used for the SSH session is going to be assumed,
+  * ''openvt'' makes the command run on a virtual terminal,
+    * ''1'' refers to ''/dev/tty1'',
+    * ''-f'' forces the command to run even if the virtual terminal is occupied (this is by default the case for login terminals),
+  * ''--'' is the separator between ''openvt'' parameters and the command to be executed,
+  * ''setterm'' is the command to execute and,
+  * ''-blank force'' instructs the terminal to blank.
+====== Resetting USB Device from Command Line ======
+[[http://www.usb.org/developers/wusb/|Wireless USB]], by consequence, has brought to Linux the capability of simulating an USB disconnect and reconnect - this is particularly useful if the device is connected on the inside of the machine such that the device cannot be removed (even logically) because it cannot be replugged.
+The first step is to identify the device you want to reset by issuing:
+<code bash>
+lsusb
+</code>
+and checking the column with the device ID. For instance, you would want to reset the device:
+<code>
+Bus 001 Device 007: ID 05b7:11aa Canon, Inc.
+</code>
+such that the relevant bit to retain is the vendor ID ''05b7'' and the product id ''11aa''.
+Next, locate the device on the USB HUB by issuing:
+<code bash>
+find -L /sys/bus/usb/devices/ -maxdepth 2 -name id* -print -exec cat '{}' \; | xargs -L 4
+</code>
+and then locate the ''/sys'' path to the device you would like to reset. In this case, the line matching the vendor and product ID would be:
+<code>
+/sys/bus/usb/devices/1-8/idProduct 11aa /sys/bus/usb/devices/1-8/idVendor 05b7
+</code>
+Finally deauthorize the device by issuing:
+<code bash>
+echo 0 >/sys/bus/usb/devices/1-8/authorized
+</code>
+and re-authorize the device by issuing:
+<code bash>
+echo 1 >/sys/bus/usb/devices/1-8/authorized
+</code>
+The above is sufficient to trigger and udev hotplug event - in case you are debugging udev scripts.
+====== Set CPU Governor for all CPUs ======
+The following command will set the CPU governor to ''powersave'' for all CPUs installed in the system:
+<code bash>
+for i in `find /sys/devices/system/cpu/cpu[0-9]* -type d | awk -F'/' '{ print $6 }' | sort -g -k1.4,1 -u | cut -c 4-`; do cpufreq-set -c $i -g powersave; done
+</code>
+====== Enable Persistent Journal Logging ======
+From man (8) ''systemd-journald'':
+<code bash>
+mkdir -p /var/log/journal
+systemd-tmpfiles --create --prefix /var/log/journal
+</code>
+====== Home Folder Permissions ======
+  * ''711'' if you dont want to add groups as well or
+  * ''751'' so that public can't read your home directory
+====== Correct /etc/hosts Setup ======
+In the event that Linux decides to answer with an IPv6 address when pinging ''localhost'', for example:
+<code>
+PING localhost(localhost (::1)) 56 data bytes
+bytes from localhost (::1): icmp_seq=1 ttl=64 time=0.226 ms
+bytes from localhost (::1): icmp_seq=2 ttl=64 time=0.291 ms
+bytes from localhost (::1): icmp_seq=3 ttl=64 time=0.355 ms
+bytes from localhost (::1): icmp_seq=4 ttl=64 time=0.353 ms
+</code>
+then the issue is an incorrect setup of ''/etc/hosts'' - notably, the IPv6 addresses are not setup correctly and Linux answers with the IPv6 equivalent address of ''localhost''.
+Open ''/etc/hosts'' and modify the IPv6 section to contain the following:
+<code>
+# The following lines are desirable for IPv6 capable hosts
+::1 ip6-localhost ip6-loopback
+fe00::0 ip6-localnet
+ff00::0 ip6-mcastprefix
+ff02::1 ip6-allnodes
+ff02::2 ip6-allrouters
+ff02::3 ip6-allhosts
+</code>
+and all services should start working properly again.
+====== Disk Dump with Progress ======
+On newer Linux systems, the command:
+<code bash>
+dd if=/dev/xxx of=/dev/yyy bs=8M status=progress
+</code>
+will display progress status whilst copying. Unfortunately, that does not include a convenient progress bar to check for completion.
+Alternatively, the corresponding command:
+<code bash>
+pv -tpreb /dev/xxx | dd of=/dev/yyy bs=8M
+</code>
+will use ''pv'' and display a progress bar.
+====== Disable Spectre and Meltdown Patches ======
+Add to the command line in ''/etc/default/grub'', the kernel parameters:
+<code>
+nopti noibrs noibpb nospectre_v2
+</code>
+and execute:
+<code bash>
+update-grub
+</code>
+After a reboot, the patches should be disabled and the performance will be back!
+====== Allow Binding Privileged Ports =====
+<code bash>
+setcap 'cap_net_bind_service=+ep' /path/to/program
+</code>
+where:
+  * ''program'' is an _executable_ - not a script.
+====== Determining the Last Power On Method ======
+''dmidecode'' can be used to retrieve BIOS information and, amongst which, can also tell what the last power on method has been:
+<code bash>
+dmidecode -t system | grep 'Wake-Up Type'
+</code>
+will print the last wake-up type.
+====== Issues with Stuck Cores ======
+It may happen that logs fill up with messages indicating that some power management policy cannot be enforced on a given CPU core:
+<code>
+cpufreqd: cpufreqd_loop            : Cannot set policy, Rule unchanged ("none").
+cpufreqd: cpufreqd_set_profile     : Couldn't set profile "Performance High" set for cpu4 (100-100-performance)
+</code>
+It may be that the CPU core is simply stuck and may need replugging. The following two commands will take the CPU offline and the next one will start the CPU back up:
+<code bash>
+echo "0" > /sys/devices/system/cpu/cpu4/cpufreq/online
+echo "1" > /sys/devices/system/cpu/cpu4/cpufreq/online
+</code>
+In doing so, the power management issue seems to be resolved.
+====== Automatically Add all RNDIS Devices to a Bridge ======
+Edit or create the file at ''/etc/udev/rules.d/70-persistent-net.rules'' with the following contents:
+<code>
+SUBSYSTEM=="net", ACTION=="add", ATTRS{idProduct}=="a4a2", ATTRS{idVendor}=="0525", RUN+="/bin/sh -c '/sbin/ip link set dev %k up && /sbin/brctl addif br0 %k'"
+</code>
+where:
+  * ''br0'' is the interface name of the bridge that the RNDIS devices will be added to.
+followed by the command:
+<code bash>
+udevadm control --reload
+</code>
+to reload all udev rules.
+The reason this works is due to ''a4a2'' and ''0525'' respectively being the identifiers for the RNDIS driver and not for the device itself. For instance, by issuing:
+<code bash>
+udevadm info -a /sys/class/net/usb0
+</code>
+will show at the top the RNDIS device without any identifiers whereas the parent ''RNDIS/Ethernet Gadget'' matches the identifiers.
+One usage case for this rule is to connect a bunch of RNDIS devices to an USB hub and have them join the network automatically as they are hotplugged; for instance, Raspberry Pis [[fuss/raspberry_pi#logging-in_via_ssh_over_usb|can be configured as USB gadgets]] and then connected to an USB hub.
+====== Scraping a Site Automatically using SystemD ======
+FTP sites can be scraped elegantly by using systemd and tmux on Linux. By starting a ''tmux'' detached terminal, ''wget'' can run in the background and download a website entirely whilst also allowing the user to check up on the progress by manually attaching and detaching from ''tmux''.
+The following script contains a few parameters underneath the ''Configuration'' comment and up to ''Internals'' in order to set:
+  * the download path (''DOWNLOAD_DIRECTORY''),
+  * the ''wget'' download URL (all protocols supported by ''wget'' such as FTP or HTTP) (''DOWNLOAD_URL''),
+  * a descriptive name for the ''tmux'' session (''TMUX_SESSION_NAME'')
+<code bash>
+[Unit]
+Description=Scrape FTP Site
+Requires=network.target local-fs.target remote-fs.target
+After=network.target local-fs.target remote-fs.target
+[Install]
+WantedBy=multi-user.target
+[Service]
+# Configuration
+Environment=DOWNLOAD_DIRECTORY="/path/to/directory"
+Environment=DOWNLOAD_URL="ftp://somesite.tld/somedirectory"
+Environment=TMUX_SESSION_NAME="somesite.tld-download"
+# Internals
+Type=oneshot
+KillMode=none
+User=root
+ExecStartPre = -/bin/mkdir -p \""$DOWNLOAD_DIRECTORY"\"
+ExecStart=/usr/bin/tmux new-session -d -c "\"$DOWNLOAD_DIRECTORY\"" -s "$TMUX_SESSION_NAME" -n "$TMUX_SESSION_NAME" "/usr/bin/wget -c -r \"$DOWNLOAD_URL\" -P \"$DOWNLOAD_DIRECTORY\""
+ExecStop=/usr/bin/tmux send-keys -t "$TMUX_SESSION_NAME" C-c
+RemainAfterExit=yes
+</code>
+The file should be placed under ''/etc/systemd/system'', then systemd has to be reloaded by issuing ''systemctl daemon-reload'', the service should then be loaded with ''systemctl enable SERVICE_FILE_NAME'' where ''SERVICE_FILE_NAME'' is the name of the file copied into ''/etc/systemd/system'' and finally started by issuing ''systemctl start SERVICE_FILE_NAME''.
+Upon every reboot, the service file will create a detached tmux terminal and start scraping files from the URL.
+====== Access Directory Underneath Mountpoint ======
+In order to access a directory underneath a mountpoint without unmounting, create a bind mount of the root filesystem to a directory and then access the content via the bind mount.
+Ie, to access the contents of the directory ''/mnt/usb'' on top of which a filesystem has been mounted, create a directory:
+<code bash>
+mkdir /mnt/root
+</code>
+and create a bind mount:
+<code bash>
+mount -o bind / /mnt/root
+</code>
+Finally access the original underlying content via the path ''/mnt/root/mnt/usb2''.