Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can’t attach to the process [SOLVED]

If you are attempting to use jmap or another Java memory analysis tool to connect to a running JVM to generate a heap dump, even when running jmap as the same user as that of the running process, and encounter the following error:

Attaching to process ID 2712, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process

Following is the (likely) solution to your problem.

It is likely that the ptrace_scope setting for your system is set to a restrictive mode which will not allow another process access to the memory space of the other process.  See this link for more details.

The fix is to update the setting in /proc/sys/kernel/yama/ptrace_scope.

First, verify the current setting:

 cat /proc/sys/kernel/yama/ptrace_scope

1, indicates a restricted setting.  To update it so that another process with the same UID can attach to that processes memory space do the following (as root)

echo 0 | tee /proc/sys/kernel/yama/ptrace_scope

Now, checking that value should return 0

cat /proc/sys/kernel/yama/ptrace_scope

At this point you should now be able to do a heapdump with jmap on your process with a command similar to the following:

jmap -J-d64 -heap 2712

Firewall for Ubuntu 14.04 LTS

For whatever reason, Ubuntu 14.04 does not seem to come with a firewall.

There are however two packages which provide, both a firewall and a handy GUI front-end for it.  UncomplicatedFireWall is the main package (ufw) and the GUI is gufw.

To install:

apt-get install gufw

This will install the front-end and the dependent packages

To turn it on:

ufw enable

The default is to block all incoming traffic.

To update and add your own firewall rules and allow incoming connections

gufw

The GUI is quite intuitive and allows advanced users the ability to create their own custom rules.

Possible Remediation for Stagefright Android Vulnerability

By now, everyone is aware of the Stagefright vulnerability in Android.  It isn’t as much the name of the vulnerability but a media player library used in Android.

It seems that one of the ways to remediate the vulnerability is to configure your text messaging application to NOT auto-retrieve Multimedia messages (MMS).  On my phone that is under the Advanced Settings, “Auto-retrieve Automatically retrieve messages”.

The other thing that I did was disable Google Hangouts.  I don’t use it on my phone anyway so I probably should have turned it off to begin with.

If anyone has any other ideas, or corrections to this post, please contact me and I will update it.  (I am in the process of opening up the blog to comments).

Generate a Random String of a Specified Size with a Shell Script

The following is a one-liner for generating a random string of a fixed size in bash, where the possible characters to use in the string are any digit, letter, and a newline.

By adding the newline, you are fairly sure to prevent getting one long line of text.

< /dev/urandom tr -dc "[:digit:][:alpha:][\n]" | head -c1000 > file.out

Connecting To a Test Kitchen Instance Via SFTP, SSH, or SCP

If you are using Chef and Test Kitchen to test your cookbooks you may have need to connect to the Test Kitchen VM in some other fashion other than $ kitchen login instance-name.

To do so:

Do a $ kitchen list to see the running vms

kitchen list
Instance                      Driver   Provisioner  Verifier  Transport  Last Action
default-centos-66             Vagrant  ChefSolo     Busser    Ssh        Converged

Then look in the .kitchen directory from where you ran your $ kitchen command and look for the corresponding .yml config file for the vm

ls .kitchen
default-centos-66.yml  kitchen-vagrant  logs

Inside the .yml file for the vm is the path to the ssh key

more .kitchen/default-centos-66.yml
---
hostname: 127.0.0.1
port: '2222'
username: vagrant
ssh_key: "/home/rchapin/.vagrant.d/insecure_private_key"
last_action: converge

With that, just use the -i argument to point to the identity file and sftp to the box

sftp -P 2222 -i /home/rchapin/.vagrant.d/insecure_private_key vagrant@127.0.0.1

If the VM prompts you for a user name when logging in, you can always login via kitchen, change the password for that user and then retry

To login via kitchen issue the following command:

kitchen login default-centos-66

Setting the Compiler Version for Maven from the Command Line

By default maven sets the compiler version for you.  Of course, you can always set it in the pom, but there are cases where you cannot modify the pom, and/or you might want to test compilation and tests with different versions of java.

Following are the specific arguments to pass the compiler version to maven from the command line:

mvn clean install -Dmaven.compiler.source=1.7 -Dmaven.compiler.target=1.7

How To Publish Artifacts to the Maven Central Repository

I have just finished releasing my first project to Maven Central Repository and wanted to capture my notes for the setup of the project and all of the steps required.

Resources

Create account on OSSRH

You will need an account to the sonatype JIRA for OSSRH.  From there, you can request the creation of a new project. See http://central.sonatype.org/pages/ossrh-guide.html for details.

Setup of the Project/pom and Pre-requisites:

PGP keys

http://central.sonatype.org/pages/working-with-pgp-signatures.html

Create a set of PGP keys

gpg2 --gen-key

List the keys

gpg2 --list-keys
/usr/local2/home/rchapin/.gnupg/pubring.gpg
-------------------------------------------
pub   4096R/E5170CE8 2015-03-26 [expires: 2016-03-25]
uid                  Ryan Chapin <rchapin@nbinteractive.com>
sub   4096R/8DAF9AD6 2015-03-26 [expires: 2016-03-25]

The id for the key created is ‘E5170CE8’

Distribute the public key

gpg2 --keyserver hkp://pool.sks-keyservers.net --send-keys E5170CE8

Additions to be added to the pom

Add the <license> tag to your pom. Following are some links regarding available licenses and an example of it’s use in the pom.

I typically use the BSD 3 License, and added the following in the pom:

<project>
  ...
  <licenses>
    <license>
      <name>The BSD 3-Clause License</name>
      <url>http://opensource.org/licenses/BSD-3-Clause</url>
      <distribution>repo</distribution>
    </license>
  </licenses>
  ...
</project>

...

Add the <developers> tag to your pom and add a <developer> entry for yourself:

<project>
  ...
  <developers>
    <developer>
      <id>rchapin</id>
      <name>Ryan Chapin</name>
      <email>rchapin@nbinteractive.com</email>
      <url>http://www.ryanchapin.com</url>
      <roles>
        <role>architect</role>
        <role>developer</role>
      </roles>
      <timezone>America/New_York</timezone>
      <properties>
        <picUrl>http://www.gravatar.com/516f2158d74d134faa9649e9180ef782</picUrl>
      </properties>
    </developer>
  </developers>
  ...
</project>

 Add the <scm> tag to your pom with the details for your repository:

<project>
  ...
  <scm>
    <connection>scm:git:git@github.com:rchapin/hash-generator.git</connection>
    <developerConnection>scm:git:git@github.com:rchapin/hash-generator.git</developerConnection>
    <url>git@github.com:rchapin/hash-generator.git</url>
    <tag>HEAD</tag>
  </scm>
  ...
</project>

Distribution Management and Authentication:  Configure the pom to enable maven to deploy to OSSRH Nexus server with the Nexus Staging Maven plugin.  See below for the release profile configuration which will contain the configs for the nexus-staging-maven-plugin.

<project>
  ...
  <distributionManagement>
    <snapshotRepository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
  </distributionManagement>
  ...
</project>

Create a profile to encapsulate the creation of the javadoc and source jar as well as the pgp signing of the artifacts.  In my case, I set the profile to <activeByDefault>true</activeByDefault> to ease release builds and simply invoke a build as such during development of the project to turn off the release profile

mvn package -P\!release

Also, configure the nexus-staging-maven-plugin setting <autoReleaseAfterClose>false</autoReleaseAfterClose> to enable manual inspection of the staging repository BEFORE it is released to central.  To deploy to OSSRH and release to the Central Repository in one step, set autoReleaseAfterClose to true.

<profiles>
    <profile>
      <id>release</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <build>
        <plugins>

          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>${maven.javadoc.plugin.version}</version>
            <executions>
              <execution>
                <id>attach-javadocs</id>
                <goals>
                  <goal>jar</goal>
                </goals>
              </execution>
            </executions>
          </plugin>

          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-source-plugin</artifactId>
            <version>${maven.source.plugin.version}</version>
            <executions>
              <execution>
                <id>attach-sources</id>
                <goals>
                  <goal>jar</goal>
                </goals>
              </execution>
            </executions>
          </plugin>

          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-gpg-plugin</artifactId>
            <version>1.6</version>
            <executions>
              <execution>
              <id>sign-artifacts</id>
              <phase>verify</phase>
              <goals>
                <goal>sign</goal>
              </goals>
              </execution>
            </executions>
          </plugin>

          <plugin>
            <groupId>org.sonatype.plugins</groupId>
            <artifactId>nexus-staging-maven-plugin</artifactId>
            <version>1.6.5</version>
            <extensions>true</extensions>
            <configuration>
              <serverId>ossrh</serverId>
              <nexusUrl>https://oss.sonatype.org/</nexusUrl>
              <autoReleaseAfterClose>false</autoReleaseAfterClose>
            </configuration>
          </plugin>

        </plugins>
      </build>
    </profile>

</profiles>

Add the maven-release-plugin to the <build> section.  This should include disabling the ‘release’ profile that we added/describe above, and then specify the deploy goal together with the activation of the ‘release’ profile for the deploy goal.

<project>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-release-plugin</artifactId>
        <version>2.5.1</version>
        <configuration>
          <autoVersionSubmodules>true</autoVersionSubmodules>
          <useReleaseProfile>false</useReleaseProfile>
          <releaseProfiles>release</releaseProfiles>
          <goals>deploy</goals>
        </configuration>
      </plugin>
      ...
    </plugins>
    ...
  </build>
  ...
</project>

Adding credentials for both the distributionManagement and the pgp signing need to be added to the ~/.m2/settings.xml. One entry for your OSSRH login, and another entry for your PGP signing passphrase

OSSRH Login

<setting>
  ...
  <servers>
    <server>
      <id>ossrh</id>
      <username>your-uid</username>
      <password>your-passwd</password>
    </server>
  </servers>
  ...
</settings>

PGP Signing Passphrase

<setting>
  ...
  <profiles>
    <profile>
      <id>ossrh</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <gpg.executable>gpg2</gpg.executable>
        <gpg.passphrase>your-passphrase</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
  ...
</settings>

Performing a SNAPSHOT Deployment

As long as your current pom version still ends in ‘SNAPSHOT’ you can deploy a snapshot version to the OSSRH

mvn clean deploy

To use the SNAPSHOT version in a project, users will need to add the snapshot repo to their Nexus, settings.xml or pom.xml. 

<project>
  ...
  <repositories>
    <repository>
      <id>ossrh-snapshots</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    </repository>
  </repositories>
  ...
</project>

Release and Deployment

First make sure that all of the code is pushed to the remote repo and that everything is merged into your main branch.  Then make sure that you are ON the main branch on the local machine.

Prepare Release

To do a dry run and insure that pom will be bumped as expected and to check that everything is in order. This will NOT check-in or tag anyting in the scm repository.

mvn release:prepare -DdryRun=true

Check the output and then do the following before running the actual prepare command

mvn release:clean

Execute the release:prepare

mvn release:prepare

 If you encounter any errors

mvn release:prepare -Dresume=false

Alternatively, you can use

mvn release:clean release:prepare

See http://maven.apache.org/maven-release/maven-release-plugin/examples/prepare-release.html for details.

Once the release has been performed the scm repository should be updated with the new tag and the pom should be bumped to the next version for the next iteration of development.

Perform Release

Once the release has been prepared and a tag created and the scm repository has been update, you can deploy to OSSRH. The following will deploy to a staging repository.

mvn release:perform

Inspecting Staging Repo and Releasing to Central

Login to OSSRH via https://oss.sonatype.org/.  Uid and passwd are the same for issues.sonatype.org. See http://central.sonatype.org/pages/releasing-the-deployment.html for details

Releasing to Central

Once you have inspected the staging repo and are ready to release the deployment to Central do the following

cd target/checkout
mvn nexus-staging:release

If this is your first release to central for this project, don’t forget to go back to your project creation request ticket and add a comment that you have promoted your first release so that your promotion can be verified and your artifacts synced with Central.  This step only needs to be done the first time you promote to central with a new OSSRH project.

Debugging MapReduce MRv2 Code in Eclipse

Following is how to set-up your environment to be able to set breakpoints, step-through, and debug your MapReduce code in Eclipse.

All of the this was done on a machine running Linux, but should work just fine for any *nix machine, and perhaps Windows running Cygwin (assuming that you can get Hadoop and its naitive libraries compiled under Windows).

This also assumes that you are building your project with maven.

Install a pseudo-distributed hadooop cluster on your development box.  (Yes, this calls for another article on exactly how to do that which I will do shortly and link to from here).

Add the following environment variables to .bash_profile to ensure that they will be applied to any login shells (make sure to check the location of the directories for your installed hadoop distribution):

export LD_LIBRARY_PATH=/usr/lib/hadoop/lib/native
export HADOOP_HOME=/usr/lib/hadoop

Make sure to include the following dependencies in your pom:

  • hadoop-mapreduce-client-core
  • hadoop-common
  • hadoop-hdfs
  • hadoop-client

After you import your maven project into Eclipse update the Build Path to include the correct path to the Native library shared objects:

  1. Right-click on your project and select ‘Build Path -> Configure Build Path:
  2. Click on ‘Libraries’ tab:
  3. Click the drop-down arrow for the ‘Maven Dependencies’
  4. Click on the drop-down arrow on the ‘hadoop-common’.jar
  5. Select the ‘Native library location’ entry, and click ‘Edit’
  6. Browse to the path of the native directory, in my case it was /usr/lib/hadoop/lib/native.
  7. Click ‘OK’
  8. Click ‘OK’ to close the build path dialogue

Create a run configuration for the Main class in your project:

Make sure that you do not add the /etc/hadoop/conf* dir to the class path.

Add any commandline arguments for input and output directories to the ‘Program arguments’ section of the run configuration, that points to your LOCAL file system and not HDFS.

Afterwhich, you should be able to run your M/R code and debug it through Eclipse.