A good read for any Java programmer.→ Continue reading “An Excellent Article on finalize in Java”
Let’s say you have some program that is generating output to STDOUT and you want to write a script to read that output from STDIN and use it as a variable in your script.
To do so:
echo “SOME_VAR = $SOME_VAR” → Continue reading “Writing a BASH Script to Read from STDIN to a Variable”
I have just finished releasing my first project to Maven Central Repository and wanted to capture my notes for the setup of the project and all of the steps required.
Publishing Artifacts to Maven Central:
– Create account on OSSRH: You will need an account to the sonatype JIRA for OSSRH. From there, you can request the creation of a new project.
. See http://central.sonatype.org/pages/ossrh-guide.html for details
– Setup → Continue reading “How To Distributed Artifacts in the Maven Central Repository”
Following is how to set-up your environment to be able to set breakpoints, step-through, and debug your MapReduce code in Eclipse.
All of the this was done on a machine running Linux, but should work just fine for any *nix machine, and perhaps Windows running Cygwin (assuming that you can get Hadoop and its naitive libraries compiled under Windows).
This also assumes that you are building your project with maven.
Install a pseudo-distributed hadooop cluster on your development box. (Yes, → Continue reading “Debugging MapReduce MRv2 Code in Eclipse”
When writing unit tests to cover your entire program you will undoubtedly come accross the need to test private methods. This can be acheived using reflection in Java JUnit tests.
What is a little tricky, and was not completely obvious, was how to use reflection to test a private static method that accepted an array of primitives. Following is a simple example, with explainations in the comments.
Note, this code will not run as it, you would need to transpose → Continue reading “Unit Testing Private Static Methods With Primitive Array Arguments”
If you have text files created under DOS/Windows and need to convert the CRLF (carriage return and line feed) characters to LF (line feed) character, here is a quick one-liner.
$ cat file.txt | perl -ne ‘s/\x0D\x0A/\x0A/g; print’ > file.txt.mod
You can also use dos2unix, however, especially under Cygwin I have seen dos2unix fail without giving any meaningful information about why it was unable to complete the task. In that case, you can just do it by hand. → Continue reading “One-Liner for Converting CRLF to LF in Text Files”
Following are two basic settings (I believe that there are other language specific, C++ for instance, settings as well).
Click on ‘New’ to create a new profile and select the profile that you want to copy
Then click ‘Edit’ and select ‘Spaces Only’ from the ‘Tab Policy’ dropdown.
You can further set the indentation and tab size.
For default text editor:
Window->Preferences->General->Editors->Text Editors->Insert spaces for tabs→ Continue reading “Configuring Eclipse to Replace Tabs with Spaces for Indentation”
When writing utility scripts in Bash it is tempting to simply pass positional arguments, use $1, $2, etc. and be done with it. However if you want to either share this utility with other members of your team and/or incorporate it into your system, it makes sense to implement your command line argument parsing in a more flexible and maintainable manner.
Using getopt you can very easily pass a variety of command line options and arguments.
Following is a link → Continue reading “Parsing Command Line Arguments with getopt in Bash”
If you want to pass an array of items to a bash function, the simple answer is that you need to pass the expanded values. That means that you can pass the data as a quoted value, assuming that the elements are whitespace delimited, or you can pass it as a string and then split it using an updated IFS (Internal Field Separator) inside the function.
Following is an example of taking the output of a Hive query (a single → Continue reading “Passing an Array as an Argument to a Bash Function”
I’m using the Hortonworks Virtual Sandbox for development and testing and wanted to restart the HDP stack without (of course) having to restart the VM.
It took me a little while to figure out how to go about it as Internet searches on the topic revealed very little.
It turns out that Hortonworks have set up their own service on the box, startup_script.
If you take a look at /etc/init.d/startup_script you will see that it calls a number of other → Continue reading “Restarting Individual Services or the Entire HDP Stack in the Hortornworks Virtual Sandbox”