Category: Linux

Mount External, Encrypted, LMV drives

Posted on by rchapin

First decrypt the drive. When I plugged it in GNOME displayed a dialog and asked for the key. I’m guessing that cryptsetup luksOpen /dev/device device_crypt would work as well.

Run lvdisplay to get the LV Path for the logical volume that you want to mount.

Then, activate the volumes , there is a way to activate a specific one volume, but the following activates them all.

vgchange -ay

Then mount it

mount <lv-path> /mnt

PDF Commands Cheat Sheet

Posted on by rchapin

Following are a number of common PDF processing commands on a Linux box

Convert a series of images to a PDF

convert image-1 image-2 ... images.pdf

convert-im6.q16: attempt to perform an operation not allowed by the security policy `PDF’ @ error/constitute.c/IsCoderAuthorized/426.

ImageMagick, specifically version 6 (im6), is preventing the conversion of PDF files due to a security policy. To address this

  1. As root, edit /etc/ImageMagick-6/policy.xml
  2. Search for the line <policy domain="coder" rights="none" pattern="PDF" />
  3. Update rights="none" to rights="read|write"
  4. Save the xml file

Debian Package Management Cheat Sheet

Posted on by rchapin
  • List version of an installed package: dpkg -s docker-ce | grep Version
  • Show all available versions of a package: apt-cache madison <package>

Packages being held back because of phasing

Run the following command replacing the package in question

apt-get --with-new-pkgs upgrade <package-name>

The following packages have been kept back

Try the following, in order. Each is less risky than the next. Stop as soon as you have fixed the problem.

Solution 1

apt-get --with-new-pkgs upgrade <list of packages kept back>

Solution 2

apt-get install <list of packages kept back>

Failed to fetch . . . Hash Sum mismatch

If you see an error similar to the following

Reading package lists... Done
E: Failed to fetch https://packages.cloud.google.com/apt/dists/cloud-sdk/main/binary-all/Packages  Hash Sum mismatch
   Hashes of expected file:
    - Filesize:1677314 [weak]
    - SHA256:09c3377dd2740e63c831dea4a86ccb1253ab6a41ec71eaa1cd4c7e52ee8c9255
    - MD5Sum:16ab83da58155f40c9a5a18a2a8c6587 [weak]
   Hashes of received file:
    - SHA256:ee6eabdfda398ab36e6fa8cd777d5d02b45dbac892dacdf6f1be1ca5ad0cbb65
    - MD5Sum:6dd0c5ac95bc55b7864e66986b71b8fa [weak]
    - Filesize:1677314 [weak]
   Last modification reported: Fri, 04 Apr 2025 01:54:07 +0000
   Release file created at: Tue, 01 Apr 2025 12:57:58 +0000
E: Failed to fetch https://packages.cloud.google.com/apt/dists/cloud-sdk/main/binary-amd64/Packages
E: Some index files failed to download. They have been ignored, or old ones used instead.

Try removing all of the apt list data

rm -rf /var/lib/apt/lists/* && apt-get update

Setting up software RAID on Debian with mdadm

Posted on by rchapin

Software RAID has come a long way. Unless you have some very high-rate, high-volume, I/O workloads with SLAs that will otherwise cost you money, for the most part a software RAID will perform just fine. One of the primary benefits of using software RAID is the portability of your disks/data. If a box on which you have a software RAID dies somehow and at least some (depending on your RAID configuration) of drives survive, you can easily resurrect the RAID on another machine without having a duplicate RAID card on hand.

Following is an example of how to setup a RAID 1 array using mdadm on Debian. I have two 2TB drives, /dev/sdb and /dev/sdc.

Install mdadm

apt-get install mdadm

Create the array

mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc

Once you create the RAID device, you can check on the mirroring process (Not sure what it is syncing with a blank disk…)

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc[1] sdb[0]
1953382464 blocks super 1.2 [2/2] [UU]
[>....................] resync = 1.0% (20516736/1953382464) finish=190.1min speed=169401K/sec
bitmap: 15/15 pages [60KB], 65536KB chunk

Create a filesystem

mkfs.xfs /dev/md0

Create a mount point and mount the filesystem

mkdir /data
mount /dev/md0 /data

Ensure that the array is reassembled on boot by adding it to /etc/mdadm/mdadm.conf. We do this by scanning the active array and appending it’s details to the config file.

mdadm --detail --scan /dev/md0 >> /etc/mdadm/mdadm.conf

Upate the initramfs so that the array will be available on boot

update-initramfs -u

Add an entry to /etc/fstab

/dev/md0 /data               xfs     defaults,nofail,discard  0 0

Do a test reboot and make sure that it is mounted and the data accessible.

[SOLVED] debsig-verify for Failed verification error, “signatures using the SHA1 algorithm are rejected” and “Can’t check signature: Invalid digest algorithm”

Posted on by rchapin

If you are using debsig-verify for the verification of a downloaded .deb file and are unable to verify it, run it with the -d option to get more information. If you see the following two lines

gpg: Note: signatures using the SHA1 algorithm are rejected
gpg: Can't check signature: Invalid digest algorithm

It is likely that the PGP signature used to sign the package uses the SHA1 algorithm which has been deprecated in most of the recent Linux distros. If you can generate another PGP key with a different algorithm. If you are a consumer of this deb package and cannot get the maintainer to update their public key you can add a gpg configuration that will enable gpg to use the PGP signature

echo "allow-weak-digest-algos" >> /etc/gnupg/gpg.conf

And then retry with debsig-verify.

LVM Resize – reduce the size of one logical volume to enable expanding another

Posted on by rchapin

I’m running an Ubuntu workstation and when setting it up simply went the “next-next-next” route when setting up the encrypted disk via LVM. The default is to create a 1G swap partition which is just not enough when you attempt to run too many things and locks up and/or crashes the machine.

My goal was to reduce my /root partition and then use that space to extend my swap partition.

Ensure that you back up your data first! There is no guarantee when executing the following operations, even correctly, that you will not lose your data.

Decreasing the size of an LVM partition

Because I need to modify the /root partition I first needed to boot to a live image or rescue image. I happened to have a Debian 12 iso on a flash drive and after sorting out how to get my laptop to boot from it chose the rescue option when booting.

One of the unexpected options during boot to the rescue image was to decrypt the filesystem so I simply entered the passphase to decrypt it. If you are using a rescue image that does not include that feature checkout this post for details on how to decrypt it.

An overview of the steps is as follows

  1. Run a filesystem check on your filesystem
  2. Resize the filesystem contained in the logical volume
  3. Resize the logical volume

Run a filesystem check

Since you have booted from a rescue disk you first need to activate the volume group to work with it

vgchange -ay

Enter the following to the details for the volume group

root@marge:~# vgdisplay
  --- Volume group ---
  VG Name               marge-vg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <465.27 GiB
  PE Size               4.00 MiB
  Total PE              119108
  Alloc PE / Size       119108 / <465.27 GiB
  Free  PE / Size       0 / 0   
  VG UUID               m06QqY-lQ3N-Y0be-j7nw-V1jo-hHwK-1OrE6U

Then use lvdisplay to see the details of all of the logical volumes contained within that group

root@marge:~# lvdisplay
  --- Logical volume ---
  LV Path                /dev/marge-vg/root
  LV Name                root
  VG Name                marge-vg
  LV UUID                820Zm6-zOcV-9gMu-efCr-jnVv-nmzX-T04zFq
  LV Write Access        read/write
  LV Creation host, time marge, 2022-12-15 14:58:30 -0500
  LV Status              available
  # open                 1
  LV Size                <464.31 GiB
  Current LE             118863
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
   
  --- Logical volume ---
  LV Path                /dev/marge-vg/swap_1
  LV Name                swap_1
  VG Name                marge-vg
  LV UUID                MHyNcx-ASX7-bsDd-Wto3-k9u8-QjZA-Z85yIR
  LV Write Access        read/write
  LV Creation host, time marge, 2022-12-15 14:58:30 -0500
  LV Status              available
  # open                 2
  LV Size                980.00 MiB
  Current LE             245
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

Run the filesystem check as you cannot resize a filesystem that in a bad state. The LV Path is the path to the underlying file system.

e2fsck -fy /dev/marge-vg/root

Shrink the filesystem

Just to be safe, we will shrink the filesystem to be smaller than the target logical volume size. If we accidentally shrink the logical volume to be smaller than the filesystem that it contains this can result in corruption of your data. Once the operation is complete, we can reclaim that “lost” space. The last argument in the command specifies the desired target size of the filesystem.

resize2fs /dev/marge-vg/root 430G

Shrink the logical volume

Once the filesystem is shrunk we can now shrink the logical volume. Again, we will specify a specific target size for the logical volume of 434G. You will get a warning that the operation could result in data loss. If we have performed the previous steps successfully we are (relatively) safe to proceed.

lvreduce -L 434G /dev/marge-vg/root

lvdisplay should now show the smaller logical volume

root@marge:~# lvdisplay
  --- Logical volume ---
  LV Path                /dev/marge-vg/root
  LV Name                root
  VG Name                marge-vg
  LV UUID                820Zm6-zOcV-9gMu-efCr-jnVv-nmzX-T04zFq
  LV Write Access        read/write
  LV Creation host, time marge, 2022-12-15 14:58:30 -0500
  LV Status              available
  # open                 1
  LV Size                <434.00 GiB  <-- New LV Size
  Current LE             118863
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1

Now that the logical volume has been reduced we can reclaim the additional space (the difference between the reduced filesystem size and the new logical volume size) by extending the filesystem to use all available space in the logical volume.

resize2fs /dev/marge-vg/root

Extend the other logical volume

Verify the amount of free space now available in the volume group

root@marge:~# vgdisplay
  --- Volume group ---
  VG Name               marge-vg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <465.27 GiB
  PE Size               4.00 MiB
  Total PE              119108
  Alloc PE / Size       111427 / <435.27 GiB
  Free  PE / Size       7618 / 30.00 GiB  <-- Free space
  VG UUID               m06QqY-lQ3N-Y0be-j7nw-V1jo-hHwK-1OrE6U

Run the following command to extend the logical volume. In this case, we will extend the swap volume by 30 G

lvextend -L +30G /dev/marge-vg/swap_1

Resize the filesystem for the extended logical volume

We have only resized the logical volume. The filesystem contained therein has not yet been resized to use the additional space

Because we are resizing a swap partition we need to do things a little differently. Run the following to recreate the swap partition in this logical volume

mkswap /dev/marge-vg/swap_1

For a “normal” partition you would run the following to extend the filesystem to the size of the logical volume.

resize2fs /dev/<vol-group>/<lv>

From here you should be able to reboot your system with your new LVM configuration.

Use printf to join an array in Bash

Posted on by rchapin

If you would like to join an array of elements with a defined delimiter in Bash there is an easy way to go about it by using printf. Following is an example

#!/bin/bash

declare -a arr=()

for i in `seq 1 5`
do
  arr=("${arr[@]}" $i)
done

# Generate a single string joined by a comma.  The printf string can contain
# any arbitrary delimiter.
printf -v joined '%s,' "${arr[@]}"

# Print out the string minus the trailing comma
echo "${joined%,}"

Diffing the output of two commands

Posted on by rchapin

The GNU diff command on most Linux and UNIX systems will diff the contents of two files. With Bash, you can, using process substitution, take the output of any arbitrary command and process its input, or output, as a file descriptor. In this way, you can then use diff against the output of two commands as follows

diff <(cmd1) <(cmd2)

Both cmd1 and cmd2 will appear as a file name/file descriptor. The < character indicates that the file descriptor should be read to obtain the output.

How to check if a file is sourced in Bash

Posted on by rchapin

Sometimes you will want to ensure that a file is sourced instead of executed. This ensures, among other things, that any environment variables that the script defines remain in your current shell after the script completes.

To do so, use the following to check whether the file was sourced or run in a sub-shell

(return 0 2>/dev/null) && sourced=1 || sourced=0
echo "sourced=$sourced"

Bash allows return statements only from functions and in a scripts top level scope IF it is sourced. If you try to use a return statement outside of a function in a non-sourced context it returns an error.

© 2005 Ryan Chapin