Welcome to my website. I am always posting links to photo albums, art, technology and other creations. Everything that you will see on my numerous personal sites is powered by the formVistaTM Website Management Engine.

icon.linkedin.jpgicon.twitter.jpg

  • Subscribe to this RSS Feed
  • BASH Script With Default Arguments Defined in The Script
    06/02/2014 12:41PM

    Often times you will want to write a BASH script where you don't want to have to keep track of all of the positional command line arguments and/or you might want to configure it with a set of environmental variables while having a default value for each in the script.

    Following is the syntax for declaring them in the shell script, and then an example on how to invoke it.

     #!/bin/bash

    : ${ARG1:="somedefault_arg1"}
    : ${ARG2:="10"}

    echo "ARG1 = $ARG1"
    echo "ARG2 = $ARG2"



    $ ./default-bash-vars.sh
    ARG1 = somedefault_arg1
    ARG2 = 10
    $ ARG1="someOtherArg1" ARG2="20" ./default-bash-vars.sh
    ARG1 = someOtherArg1
    ARG2 = 20

    In the example script, we have two variables, ARG1 and ARG2.  When running the script without providing any additional configuration the default values will be used.  When invoking it and defining the variables on the command line prior to executing the script those values will be used instead.

    This prevents the situation where you potentially have many command-line arguments and then have to jugle the positional $1, $2, .... vars in the script.

  • Unable to Set the Path To java.exe When Running Oracle SQL Developer Under Windows 7
    05/22/2014 1:08PM

    I was trying to run Oracle SQL Developer for the first time on a new machine.  When firing it up, it presented me with a dialog box asking me to "Enter the full pathname for the java.exe file".

    OK, no problem.  So I find the path to the java.exe binary that was just installed with the SDK.  Hit submit . . . and nothing happens.  It blanks out the text field and the dialog box stares back at me.

    I tried pointing it to the java.exe that was in the jre dir.  No joy.

    After a lot of futzing around and doing some searches it turns out that there are a few things that you have to do to get it to run for the first time.

    First, right-click on the Oracle SQL Developer short-cut and select "Run as Administrator".

    Then, in the dialog box, click Browse and navigate to the JDK that comes with the Oracle install.  For me it was in C:\Oracle11G\11.2.0.3\jdk\bin\java.exe

    Once I did that it fired right up.  I then quit, and ran it as my user and it seemed to start up just fine.

  • Pin a "Show Desktop" Link to the Task Bar in Windows 7
    05/19/2014 12:16PM

    To create a short cut and pin it to the task bar in Windows 7 do the following:

    1. Right-click on the Desktop and select New -> Shortcut.
    2. Enter the following in the field labeled "Type the location of the item:"  %windir%\explorer.exe shell:::{3080F90D-D7AD-11D9-BD98-0000947B0257}
    3. Click "Next" and name the shortcut and then click "Finish".
    4. Update the icon so that you will recognize it when you pin it to the task bar.  To do so, right-click on the shortcut and select "Change Icon"
    5. Paste the following in the "Look for icons in this file"  %SystemRoot%\system32\imageres.dll
    6. Select an appropriate icon and click "Apply" and then "OK".
    7. Right-click on the icon and select "Pin to Taskbar".
  • Count The Number of Characters In a Field With awk
    05/12/2014 2:09PM

    A quick one-liner for counting the number of characters in a given field for each line in an ASCII file.

    for i in `cat source.txt`; do echo $i | awk  {'print length($1)}' >> counts.txt

  • Running Dynamically Generated Hive Queries From a Shell Script
    03/26/2014 10:35AM

    If you want to write a HQL hive query and run it mulitple times from a shell script, each time passing it different data for the query, here is a quick example that should get you started.

    The first thing to know is that by specifying n number of -hivevar key value pairs when invoking hive on the command line will allow you to pass that data into the hive process.

    For example, if you do the following

    $ hive -e 'SELECT * FROM some_table' -hivevar FOO=blah

    You will have passed in a key of FOO with the value of 'blah' to the hive process.

    A more practical example would be wanting to run the same hive query over multiple data partitions.

    In this example, I've got a hive database that has a 'packets' table partitioned by hours which looks like 2014032601.

    The hive query file (dest_ip_hive.sql) would look like:

    SELECT packets.sourceip FROM packets
    WHERE packets.destip = "${hivevar:DEST_IP}"
    AND packets.hour = ${hivevar:HOUR}
    GROUP BY packets.sourceip

    And a shell script that would dynamically set those values for each invocation of hive would look like:

    #!/bin/bash

    #
    # Destination IP that we are using to determine which
    # packets we will examine.
    #
    DEST_IP="10.0.1.10"

    for HOUR in 2014032209 2014032210 2014032211 2014032212
    do

       echo "Running hive query for HOUR $HOUR"

       # Run a hive query from the command line setting variables that will be
       # expaned in the .sql file.
       hive -hivevar HOUR=$HOUR -hivevar DEST_IP=$DEST_IP \
       -f dest_ip_hive.sql > ${DEST_IP}-{$HOUR}.out

    done

    For each hour defined in the for loop, we will execute a hive command telling it to run the query contained in the file dest_ip_hive.sql.  The DEST_IP and HOUR variables that will be expanded in the query are passed to hive via the

    -hivevar HOUR=$HOUR -hivevar DEST_IP=$DEST_IP

    part of the hive command.  And the output for each query will be written to a different file for each query.



Advanced Search

Categories

Archives