Passing an Array as an Argument to a Bash Function

If you want to pass an array of items to a bash function, the simple answer is that you need to pass the expanded values.  That means that you can pass the data as a quoted value, assuming that the elements are whitespace delimited, or you can pass it as a string and then split it using an updated IFS (Internal Field Separator) inside the function.

Following is an example of taking the output of a Hive query (a single → Continue reading “Passing an Array as an Argument to a Bash Function”

Removing the Last Token From a String in Bash with awk

Let’s say that you have some number of files for which you want to create a containing directory that is named with all but the last token of the file name, and you want to remove just the last token to create the name of the directory.

Much easier to explain with an example.  Given this list of files:

ls -1
foo_10_10_sometrash
foo_1_sometrash
foo_2_sometrash
foo_3_sometrash
foo_4_sometrash
foo_5_5_sometrash
foo_5_sometrash
foo_6_6_sometrash
foo_7_7_sometrash
foo_8_8_sometrash
foo_9_9_sometrash

You want to create a directory for each → Continue reading “Removing the Last Token From a String in Bash with awk”

Checking that Input or a Variable is an Integer in BASH

Here is a quick snippet for checking whether or not a variable is a valid integer in BASH.  It is also a howto for regular expressions in a shell script.

# Make sure that FOO is an integer
if [[ ! "$FOO" =~ ^[0-9]+$ ]]; then
        echo "The FOO was NOT an integer"
fi
Continue reading “Checking that Input or a Variable is an Integer in BASH”

Creating an Array in Bash from a File With Each Element on a Separate Line

Let’s say that you have a file and you would like to convert each line in the file to an element in an array.

The key to this is knowing about and how to manipulate the IFS (Internal Field Separator).  The default IFS is whitespace (a space, tab, or newline) and if you create an array passing it a whitespace delimited list of strings, each token will be set to an element in the array.

ARRAY=(a b d 
Continue reading “Creating an Array in Bash from a File With Each Element on a Separate Line”

Removing The Last N Character From a String in Bash Script with sed

Here is a quick one-liner for trimming a specific number of characters from the end of a string under bash:

# Remove the last 5 characters
$ echo "somestringwith12345" | sed "s/.....$//g"
$ somestringwith

# Remove the last 3 characters
$ echo "somestringwith12345" | sed "s/...$//g"
$ somestringwith12
Continue reading “Removing The Last N Character From a String in Bash Script with sed”

Splitting a String in Bash on the FIRST Occurrence of a Character

About a year ago I posted an article about how to split into an array of values based on a given delimiter in bash.

The following is how to take that same string and split it on the first occurrence of the same user defined delimiter.

Both use the ‘read’ command, but in a slightly different way.

Instead of passing read the -a [aname] parameter which tells it that “The words are assigned to sequential indices of the array → Continue reading “Splitting a String in Bash on the FIRST Occurrence of a Character”

BASH Script With Default Arguments Defined in The Script

Often times you will want to write a BASH script where you don’t want to have to keep track of all of the positional command line arguments and/or you might want to configure it with a set of environmental variables while having a default value for each in the script.

Following is the syntax for declaring them in the shell script, and then an example on how to invoke it.

#!/bin/bash

: ${ARG1:="somedefault_arg1"}
: ${ARG2:="10"}

echo "ARG1 = $ARG1"
echo 
Continue reading “BASH Script With Default Arguments Defined in The Script”

Running Dynamically Generated Hive Queries From a Shell Script

If you want to write a HQL hive query and run it mulitple times from a shell script, each time passing it different data for the query, here is a quick example that should get you started.

The first thing to know is that by specifying n number of -hivevar key value pairs when invoking hive on the command line will allow you to pass that data into the hive process.

For example, if you do the following

$ hive 
Continue reading “Running Dynamically Generated Hive Queries From a Shell Script”