The title of this post is a bit of mouthful and requires a bit more explanation.
I am running a pure open-source version of Kafka (currently running 2.7) and am using SASL/GSSAPI connections between all of the brokers and ZooKeeper. Currently, the whole system, including ZooKeeper, is running Java 8 and it is long-overdue to be upgraded to Java 11.
When maintaining a Hadoop cluster, you will need to restart various service from time-to-time when/if you update Hadoop configurations.
I ran into a problem today with Ambari where I wanted to do a rollling restart of all of my DataNodes, but when I clicked on the “Restart DataNodes” entry in the “Restart” drop down the dialog indicated “There are no DataNodes to do rolling restarts”.
I was having some issues with the ambari-metrics family of services on a ‘pseudo-distributed’ cluster that I have installed on my workstation.
The symptoms were:
1. Ambari indicated the following CRITICAL errors in the Ambari Dashboard under the Ambari Metrics section
Connection failed: [Errno 111] Connection refused to rchapin-wrkstn:6188
2. After attempting to restart the ambari-metrics-collector via either the Ambari Dashboard or through the commandline (# ambari-metrics-collector [stop|start]) you see the following (similar) messages in the ambari-metrics-collector.log
I am working on a project and have decided to use Avro for the data serialization format.
I encountered the following error when trying to set up the unit test to test the mapper implementation through Eclipse:
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Following is how to set-up your environment to be able to set breakpoints, step-through, and debug your MapReduce code in Eclipse.
All of the this was done on a machine running Linux, but should work just fine for any *nix machine, and perhaps Windows running Cygwin (assuming that you can get Hadoop and its naitive libraries compiled under Windows).
This also assumes that you are building your project with maven.
If you want to pass an array of items to a bash function, the simple answer is that you need to pass the expanded values. That means that you can pass the data as a quoted value, assuming that the elements are whitespace delimited, or you can pass it as a string and then split it using an updated IFS (Internal Field Separator) inside the function.