Linux command reference

Notes and references on various commands on Linux systems.

Konsole

apt

apt-key is deprecated, how to add repos and keys

This (now) familiar error:

$ sudo apt update
W: GPG error: http://download.example.com/debian bookworm InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 1140AF8F639E0C39
E: The repository 'http://download.example.com/debian bookworm InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

The path where keys are stored has changed, and you need a signed-by attribute in the repo .list file referencing the key's path.

TLDR

  • New path for key files, use /etc/apt/keyrings (or /usr/share/keyrings if its your repo)
  • Add a signed-by to the .list file, referencing the filename: 
    deb [arch=amd64 signed-by=/etc/apt/keyrings/key.gpg] http://download.example.com/debian bookworm main
  • Remove old keys from apt-key

De-armoring the key

Convert the file from the ASCII armor to binary format (the pipe is important, haven't found the right args to gpg to do it without stdin).

$ file key.asc
key.asc: OpenPGP Public Key Version 4, Created Mon Nov  9 06:59:32 2020, RSA (Encrypt or Sign, 4096 bits); User ID; Signature; OpenPGP Certificate

$ cat key.asc| gpg --dearmor > key.gpg

$ file key.gpg
key.gpg: OpenPGP Public Key Version 4, Created Mon Nov  9 06:59:32 2020, RSA (Encrypt or Sign, 4096 bits); User ID; Signature; OpenPGP Certificate

This step isn't really needed, Debian just recommended it for compatibility reasons

Add the repo

Move the de-armored public key into place and ensure correct ownership:

$ sudo cp key.gpg /etc/apt/keyrings/
$ sudo chown root:root /etc/apt/keyrings/key.gpg
$ sudo chown 644 /etc/apt/key.gpg

Add the repos .list file with ansible:

- name: add apt repo
  apt_repository:
    repo: "deb [arch=amd64 signed-by=/etc/apt/keyrings/key.gpg] http://download.example.com/{{ ansible_distribution | lower }} {{ ansible_distribution_release }} main"
    #repo: "deb [arch=amd64] http://download.proxmox.com/{{ ansible_lsb.id | lower }}/pve {{ ansible_lsb.codename | lower}} pve-no-subscription"
    state: present
    update_cache: false
    filename: /etc/apt/sources.list.d/example

Which should look something like this

$ cat /etc/apt/sources.list.d/example
deb [arch=amd64 signed-by=/etc/apt/keyrings/key.gpg] http://download.example.com/debian bookworm main

With signed-by referencing where the file exists on your file system.

Note

Alternatively use the deb822_repository module:

- name: add your repo
  deb822_repository:
    name: example
    types: deb
    uris: http://download.example.com/{{ ansible_distribution | lower }}
    suites: "{{ ansible_distribution_release }}"
    components: stable
    architectures: amd64
    signed_by: /etc/apt/keyrings/key.gpg

Or even more alternatively instead of .list file, create a .sources file like /etc/apt/sources.list.d/example.sources:

Types: deb
URIs: http://download.example.com
Suites: {{ ansible_distribution | lower }}
Components: main
Signed-By: /etc/apt/keyrings/key.gpg

Cleanup

Then clean up the key from apt-key if it was there already. List existing keys with

$ sudo apt-get list

Debian/Ubuntu keys are still there for compatibility reasons, so grep them out:

$ sudo apt-get list | grep uid | grep -vi debian

If that turns up any keys, delete them:

$ sudo apt-key del support@example.com.

Now you can apt update and apt install and etc.

Repository changed its 'Suite' value. This must be accepted explicitly

If you find yourself on some old Debian-based system you'd forgotten about and try to apt install something and get this error message:

$ sudo apt update

Reading package lists... Done
E: Repository 'http://deb.debian.org/debian buster InRelease' changed its 'Suite' value from 'stable' to 'oldoldstable'
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.

The man page has a lot of dense info, so here's the command:

$ sudo apt-get --allow-releaseinfo-change update

Now you can update your system (and hopefully not forget about it again :).

git

XKCD 1597

XKCD #1597: Git

Submodules

By default they are added at fixed commits. Now git can set submodules to track branches1 (It's still fiddly and I'm not sure its working correctly for me.)

Adding a new submodule tracking $branch2:

$ branch=main
$ git submodule add -b $branch $url;
$ git submodule update --remote

Change an existing submodule to track $branch3:

$ submodule=foo
$ branch=main

$ # change the submodule defintion in the parent repo
$ git config -f .gitmodules submodule.${submodule}.branch $branch

$ # and make sure the submodule itself is actually at that branch
$ cd $submodule
$ git checkout $branch
$ git branch -u origin/$branch $branch

This how I have currently used it in ben/builds, but I'm not sure its correct. Also not sure that repo is a good idea or if I should go back to separate repos.

Erase file from history

Use a relative path to the file you need to erase.

It's a good idea to take a note of the commit hashes the file occurs in:

$ git log -S"${filename}" --oneline

Erase $filename from the Git history:

$ git filter-branch --force --index-filter \
    "git rm --cached --ignore-unmatch ${filename}" \
    --prune-empty --tag-name-filter cat -- --all

It's smart to search the commit history again for the file afterwards. When you are done, push to all remote branches (on all remotes):

$ git push origin --force --all

Don't forget remotes other than origin (if you have them). Note that this rewrites the Git history, altering commit hashes.

References

rsync

Sync and preserve file attributes

This preserves file attributes, and mirrors the source and destination:

$ rsync -rahS \
    --numeric-ids \
    --info=progress2 \
    --delete-after \
    --exclude="lost+found" \
    --exclude="*.tmp" \
    $source $destination

Arguments5:

  • -h/--human-readable: Output numbers in human-readable format.
  • -S/--sparse: Turn sequences of null bytes into sparse blocks
  • -a: Archive mode6, equivalent to-rlptgoD:
    • -r/--recursive: Recurse into directories.
    • -l/--links: Preserve symlinks (copy them as symlinks).
    • -p/--perms: Preserve permissions.
    • -t/--times: Preserve modification times (file attributes).
    • -g/--group: Preserve group ownership.
    • -o/--owner: Preserver user ownership (only root can chown files).
    • -D: Same as --devices --specails:
      • --devices: Preserve device files (requires root).
      • --specials: Preserve specxial files.
  • --numeric-ids: Don't map uid/gid values by names of users/groups, preserve them as-is.
  • --info=progress2: Outputs statistics based on the whole transfer.
  • --delete-after: receiver deletes after transfer complets (not during).

Sometimes useful:

  • --remove-source-files: Sender removes synchronized files (use with caution).

References

5

man rsync

Resize images and remove metadata

Inspect and strip metadata with exiftool:

$ sudo dnf install perl-Image-ExifTool
$ exiftool ${imagefile}
$ exiftool -all= -overwrite_original ${imagefile}

Strip metadata with mogrify, works for most formats (provided by ImageMagick):

$ sudo dnf install imagemagick
$ mogrify -strip ${imagefile}

Resize a .jpg image with mogrify:

$ mogrify -format jpg -resize 422x316 image.jpg
$ mogrify -format jpg -resize "50%" ${imagefile}

Arguments to mogrify:

  • -strip: remove metadata
  • -format $format: output format
  • -resize $geometry/-resize "N%: output size

Optimizing .png file sizes and remove metadata with optipng:

$ sudo dnf install optipng
$ optipng -strip all image.png

Arguments for pngoptim:

  • -strip all: remove metadata
  • -o: optimization level, defaults to 2.

Optimizing .jpg files and remove metadata with jpegoptim:

$ sudo dnf install jpegoptim
$ jpegoptim --strip-all image.jpg
$ jpegoptim --strip-all --max 85 ./*.jpg

Arguments for jpegoptim:

  • --strip-all/-s: strip all metadata
  • --max/-m: quality factor, 0-100.

References

OpenSSL

TCP

OpenSSL can read a certificate over SSL (does the TLS handshake and obtains the certificate):

$ openssl s_client -connect ${host}:${port} < /dev/null 2>/dev/null

This can then be piped into openssl with -in /dev/stdin.

Fingerprints

Get the fingerprint of a certificate in a .pem/.crt file8:

$ openssl x509 -in ${path} -noout -sha256 -fingerprint

To see everything in the certificate:

$ openssl x509 -in ${path} -noout -text

Get fingerprint of an SSL certificate over TCP9, use s_client and pipe that to x509 like above:

$ openssl s_client -connect ${host}:${port} < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin

Fetch the server certificate from MariaDB/MySQL9:

$ openssl s_client -starttls mysql -connect ${host}:3306

MariaDB/MySQL support requires OpenSSL >=1.1.1 (released in 2018).

References

Basic certbot usage

Create (request) a new cert for ${name}:

$ certbot certonly -d ${name}

The new cert exists in the certbot-managed dir /etc/letsencrypt/live:

$ ls -d /etc/letsencrypt/live/${name}
/etc/letsencrypt/live/${name}

If you need to delete/revoke a cert for ${name}:

$ certbot delete --cert-name ${name}

Stream USB camera with VLC without transcoding

$ cvlc v4l2:///dev/video0 --sout '#standard{access=http,mux=ts,dst=:8080}'

Clean desktop on Linux systems

Clean desktop in KDE

Create an empty dir and make it unwritable:

$ mkdir ~/.empty
$ chattr -fR +i ~/.empty
$ lsattr -d ~/.empty
----i---------e------- ~/.empty

Change XDG_DESKTOP_DIR in ~/.config/user-dirs.dirs, and KDE Plasma you can also set ~/.config/plasma-org.kde.plasma.desktop-appletsrc:

url=/home/${USER}/.empty/

Now the desktop won't clutter

Clean up old versions from snap

The snap packages can take up a lot of space

LANG=C snap list --all | awk '/disabled/{print $1, $3}' |
    while read snapname revision; do
        snap remove "$snapname" --revision="$revision"
    done

This removes old versions of installed "packages".

Close an LVM

$ sudo lvchange -an /dev/${vol_group_name}/${vol_name}

Closes a LVM (logical) volume, for example so that a USB drive can be luksClosed.

Trigger VueScan to scan

$ xdotool windowactivate --sync $(xdotool search --name "^VueScan.*${titlebar_part}") && sleep 1 && xdotool key KP_Enter

Super hacky way to trigger VueScan, as long as the regex ^VueScan.*${titlebar_part} is in the titlebar. Works for other programs as well, it's just matching the titlebar and making a keypress with xdotool.

GPG passphrase prompt in terminal

export GPG_TTY=$(tty)

This will cause GPG to prompt for passphrase with an ncurses-interface in the terminal, instead of trying to use KDE/GNOME keychain, making it work over ssh as well.

Reboot into UEFI/BIOS/firmware settings

$ sudo systemctl reboot --firmware-setup

haproxy

Various summary info12:

$ echo "show info;show stat;show table" | socat /var/lib/haproxy/stats stdio

Cleanly print stats for a few columns:

$ echo "show stat" | socat /var/lib/haproxy/stats stdio | awk 'BEGIN{FS=","} {sub(/^#\ */, "", $0); print $1,$2,$9,$10}' | column -t

List all available column names:

$ echo "show stat" | socat /var/lib/haproxy/stats stdio | awk 'BEGIN {FS=","} NR == 1 {sub(/^#\ */, "", $0); for (i=1; i < NF; i++) { print "$"i" = "$i }}'

More in HAProxy documentaion.

Networking

Handy output for tcpdump12:

$ tcpdump -nlei $interface

veth interfaces

See what interface is paired with a veth interface12:

$ ethtool -S $vethDevice
NIC statistics:
     peer_ifindex: 255
     rx_queue_0_xdp_packets: 0
     rx_queue_0_xdp_bytes: 0
     rx_queue_0_drops: 0
     rx_queue_0_xdp_redirect: 0
     rx_queue_0_xdp_drops: 0
     rx_queue_0_xdp_tx: 0
     rx_queue_0_xdp_tx_errors: 0
     tx_queue_0_xdp_xmit: 0
     tx_queue_0_xdp_xmit_errors: 0

$ ethtool -S vethc31a9a9 | grep "peer_ifindex" | awk -F': ' '{ print $2 }'
255

The peer_ifindex (peer interface index) should match the entry number in the output of ip link or ip addr.

Namespaces

Network namepaces are also a file12:

$ ls /var/run/netns

So finding the inodes of a process's network namespace is easy:

$ readlink /proc/{0..9}*/ns/net  | sort | uniq
$ stat /var/run/netns/*

Sorting ps output

Sort by memory13:

$ ps au --sort=-%mem
$ ps auk-%mem

Other parameters for --sort can also be used.