Passing an Array as an Argument to a Bash Function

If you want to pass an array of items to a bash function, the simple answer is that you need to pass the expanded values.  That means that you can pass the data as a quoted value, assuming that the elements are whitespace delimited, or you can pass it as a string and then split it using an updated IFS (Internal Field Separator) inside the function.

Following is an example of taking the output of a Hive query (a single column that is separated by new lines), wrapping it in quotes and passing it as a single value to the function.

#!/bin/bash

#
# This function will accept the expanded elements of the array
#
function foo() {
# Loop through elements in the first argument passed.
   # In this case, each is separated by whitespace so we do
   # not need to change the IFS
   for i in $1
   do
      echo "i = $i"
   done
}

# Dynamically build our hive query
HIVE_QRY="use somedb; select some_column from some_table;"

# Dynamically build the hive command to execute
CMD="hive -e '$HIVE_QRY'"

# Execute the hive query in a subshell and store the result in
# the 'QRY_RETVAL' variable
QRY_RETVAL=$(eval $CMD)

# Call the foo method and pass it the output of the query, /QUOTED/
# so that it will be passed as a single argument and not a series
# of arguments for each row returned by the query
foo "${QRY_RETVAL}"

Restarting Individual Services or the Entire HDP Stack in the Hortornworks Virtual Sandbox

I’m using the Hortonworks Virtual Sandbox for development and testing and wanted to restart the HDP stack without (of course) having to restart the VM.

It took me a little while to figure out how to go about it as Internet searches on the topic revealed very little.

It turns out that Hortonworks have set up their own service on the box, startup_script.

If you take a look at /etc/init.d/startup_script you will see that it calls a number of other shell scripts in /usr/lib/hue/tools/start_scripts/

To restart the whole stack simply issue the following command:

service startup_script restart

Vim Search and Replacing with Backreferences

It is often helpful to write search and replace commands that save segments of the matched text to use in the replacement string.

Source text:

This is some text (we want to change) with a phone number (301) 555-1234.
We want to remove the parenthesis, but only from the phone number string.

In this example we have a text file that has a phone number in it and we want to remove the parenthesis that surround ONLY the area code in the phone number, plus the trailing space and replace it with the aread code plus a trailing “-“.

Following is the search and replace command. The forward slashes are separators in the search command.  They will be omitted in the full explanation below:

:1,%s/(\([0-9]\+\))\s/\1-/g

Here is it explained:

:1,%s

Tells, vi to execute the search and replace command from line 1 to the end of the document

(\([0-9]\+\))\s

The regex including the backreference that will find the area code including the parenthesis and a trailing space.  The \(  and \) parenthesis wrap the numerical part of the string that we are trying to save and mark it as a backreference.  The \+ indicates that we one one or more of the numerical characters (the plus char must be escaped).  The \s indicates a whitespace character the we also want to match.

\1-

The characters that we want to use to replace what we have found.  The \1 indicates we want to use the first backreference that was matched followed by a ‘-‘ character. 

g

Tells, vi to execute the search and replace on all occurances found in a line.