Configuring rsyslog to rotate log files from log messages streamed to it from a Systemd service

In general, I have moved to writing all of my applications to write their log output to STDOUT. This makes running them on the command line, in an IDE, on a bare metal box, VM, or in a container completely decoupled from how you store and view the logs. No more having multiple logging configs for each flavor of deployment.

In this particular case, I am running an application in a container (but it isn’t necessary that it is in → Continue reading “Configuring rsyslog to rotate log files from log messages streamed to it from a Systemd service”

Looping Through a List of Files with Spaces in the File Name with Bash

If you have a list of files that you want to operate on in a loop in bash and some of them have spaces in the file name the default IFS (Internal Field Separator) will match with the space and tokenize the file.

The simple approach is to temporarily set the IFS as follows.  This can be done in a shell script, but the following example is directly on the command line for ‘one-liner’ usage.

OIFS="$IFS"

IFS=$'\n' 

for i in 
Continue reading “Looping Through a List of Files with Spaces in the File Name with Bash”

How To Remove the Byte Order Mark (BOM) from UTF-8 Encoded Text Files

The easiest way that I have seen so far for doing so is to use tail and simply read everything except the first three bytes (start reading at the 4th byte), as follows:

tail --bytes=+4 text_file.txt text_file-wo-bom.txt
Continue reading “How To Remove the Byte Order Mark (BOM) from UTF-8 Encoded Text Files”

Adding the Contents of a Source File to the Beginning of a Target File

Following is *nix a command that you can use to add the contents of a source text file to the start of another text file (the source file).

echo -e '0r <source_file_name\nw' | ed -s <target_file_name
Continue reading “Adding the Contents of a Source File to the Beginning of a Target File”

Use awk to Print from nth element to the End of the Line

If you want to extract from the nth token to the end of the line, following is how you can do that with awk:

Given a source file with the following:

line1 -- 01   0011 1
line2 -- 01   0011 2
line3 -- 01   0011 3
line4 -- 01   0011 4
line5 -- 01   0011 5
line6 -- 01   0011 6
line7 -- 01   0011 7
line8 -- 01   0011 8
line9 -- 01   0011 9
line10 -- 01   0011 
Continue reading “Use awk to Print from nth element to the End of the Line”

Generate a Random String of a Specified Size with a Shell Script

The following is a one-liner for generating a random string of a fixed size in bash, where the possible characters to use in the string are any digit, letter, and a newline.

By adding the newline, you are fairly sure to prevent getting one long line of text.

< /dev/urandom tr -dc "[:digit:][:alpha:][\n]" | head -c1000 file.out
Continue reading “Generate a Random String of a Specified Size with a Shell Script”