Category: Programming

Quick-Start and Minimal Cheat Sheet for Chef and Test Kitchen

Posted on by rchapin

Following is a quick-start for getting a Chef and Test Kitchen environment set up to start writing and testing Chef cookbooks.  This is an work in progress, so please if you find something that is incorrect or does not work as expected, send me an e-mail.

Development Environment Set-up:

Unless you are a Ruby developer and already have a complete environmenet set-up the easiest thing to do is to install the Chef Development Kit, https://downloads.chef.io/chef-dk/.

After downloading and installing the package add a CHEF_HOME and RUBY_HOME var pointed to the embedded version of Ruby that comes with the Chef SDK.

  export CHEF_HOME=/opt/chefdk
  export RUBY_HOME=/opt/chefdk/embedded
  PATH=”$CHEF_HOME/bin:$RUBY_HOME/bin:$PATH”

Building Cookbooks:

Generate the cookbook skeleton:

$ chef generate cookbook <name-of-cookbook>

Then cd into that directory and initialize the cookbook:

$ kitchen init –create-gemfile

You will also want to run bundle install, which will require your user to have sudo permissions.

$ bundle install

Run kitchen converge to verify before starting dev.

$ kitchen converge <name-of-vm>

Create a recipe

$ chef generate recipe <name-of-recipe>

[Optional] To use attributes, first create the attributes directory structure

$ chef generate attribute default

Some helpful Tips:

1.  Answer the following questions for each new cookbook you are going to create.  It will help you focus the purpose of the cookbook:

. Name
. Purpose
. Success criteria
. App/Service
. Required steps

2.  Creating template files:

$ chef generate template <name-of-file>

    . Sample variable to be added to a template:
      <%= node[‘hostname’] %>

3.  Available host variables.  You can see all of the variables available to be used in templates by logging in to a chef managed VM and issuing the following command:

$ kitchen login <name-of-vm>

$ ohia | less

4.  Add a IP to a kitchen VM so that additional kitchen VMs can talk to each other.  Add the following to your .kitchen.yml file.

platforms:
  – name: centos-7.1
    driver:
      network:
      – [“private_network”, {type: “dhcp”}]

or

platforms:
  – name: centos-7.1
    driver:
      network:
      – [“private_network”, {ip: “192.168.33.10”}]

When specifying an IP, check the vboxnet0 interface on the host machine to see what network the VirtualBox host is configured and pick an IP in that range.

In my case, I had to make sure that only one machine was configured under ‘platform’, or that each had a network configuration.

Also, I had to have ‘require_chef_omnibus: true’ under the vagrant driver as follows:

driver:
  name: vagrant
  require_chef_omnibus: true

5.  Debugging Attributes:  Using the ‘pp’ module you can format the output of the node.debug(‘<attribute-name>’).

Add the following to a recipe, and run it with kitchen converge:

require ‘pp’

node.default[‘ipaddress’] = ‘1.1.1.1’
pp node.debug_value(‘ipaddress’)

Perl One-Liner for Replacing Multiple Lines of a Text file With Multiple Lines of Text

Posted on by rchapin

When executing ‘search-and-replace’ commands on ASCII under Linux, Unix (or *nix) operating systems, sed works or most cases and makes for reasonably straightforward reading of the script.

If  you want to replace multiple lines of text with multiple lines of text, following is a perl one-liner that does the trick and is much easier to wrangle than trying to do it in sed.

perl -i -pe "BEGIN{undef $/;} s:${EXISTING_LINES}:${REPLACEMENT_TEXT}:smg" file.txt

Updating all of the pom.xml Version Numbers in a Multi-Module Maven Project

Posted on by rchapin

To update the versions of all of the poms in a multiple module project use the versions-maven plugin.

To update

mvn versions:set -DnewVersion=1.4.0-SNAPSHOT

Will modify all of the versions of each of the poms to the version specified.  It will create a pom.xml.versionsBackup for each pom file that it modified.  You can then examine each to make sure that it is as you intended.

If you want, you can revert your change with

mvn versions:revert

If you are satisfied with the change, you can commit the change with

mvn versions:commit

Use awk to Print from nth element to the End of the Line

Posted on by rchapin

If you want to extract from the nth token to the end of the line, following is how you can do that with awk:

Given a source file with the following:

line1 -- 01   0011 1
line2 -- 01   0011 2
line3 -- 01   0011 3
line4 -- 01   0011 4
line5 -- 01   0011 5
line6 -- 01   0011 6
line7 -- 01   0011 7
line8 -- 01   0011 8
line9 -- 01   0011 9
line10 -- 01   0011 10

If you want remove the 1st, 2nd, and 3rd items from the list, you can use awk to set those fields to an empty value as follows

awk '{$1=$2=$3=""; print $0}' test.out

Which will result in:

   0011 1
   0011 2
   0011 3
   0011 4
   0011 5
   0011 6
   0011 7
   0011 8
   0011 9
   0011 10

JVM Option for Increasing the Default Number of Lines in the StackTrace

Posted on by rchapin

By default (Java 1.6 or greater), the JVM will output, at most, 1024 lines of the stack trace.

In the situation where you have some recursion problem or some infinite loop that results in a stack overflow error you will need to increase this value with a JVM option to see the origin of your crash.

To do so, add the following option to the java command

$ java -XX:MaxJavaStackTraceDepth=-1 -jar some.jar some.package.Class  etc, etc,

-1 indicates no limit.  Any positive integer indicates the limit to the number of lines in the stack trace.  0 means exactly what it means and will output 0 lines.

A great resource for java options.

Debugging Maven Tests by Connecting an IDE to the Maven JVM

Posted on by rchapin

In some instances you cannot reproduce a failure or condition running a test in an IDE that manifests itself when you run it on your build server or via maven on the command line.

In that case, it is very helpful to be able to remotely attach your IDE to the running maven process and then step through the code.

To do so you will need to:

Execute maven on  the command line as follows (adding any additional -D args as required by your project):

mvn -Dmaven.surefire.debug test -pl module-in-question

This will run the maven automatically pausing the JVM awaiting for a remote debugger to connect to port 5005.  If you want to have it listen on a different port you can pass it in as follows:

mvn -Dmaven.surefire.debug="-Xdebug  -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8081 -Xnoagent -Djava.compiler=NONE" test -pl module-in-question

Create a debugging run profile in either Eclipse or IntelliJ or your favorite IDE configured to connect to a JVM listening on the specified port.

Then once you have run maven on the command line, simply execute the run configuration in your IDE and debug your application as usual.

If need be, you can run the maven JVM on the cli such that maven does not fork the tests as follows:

mvnDebug -DforkCount=0 test

Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can’t attach to the process [SOLVED]

Posted on by rchapin

If you are attempting to use jmap or another Java memory analysis tool to connect to a running JVM to generate a heap dump, even when running jmap as the same user as that of the running process, and encounter the following error:

Attaching to process ID 2712, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process

Following is the (likely) solution to your problem.

It is likely that the ptrace_scope setting for your system is set to a restrictive mode which will not allow another process access to the memory space of the other process.  See this link for more details.

The fix is to update the setting in /proc/sys/kernel/yama/ptrace_scope.

First, verify the current setting:

 cat /proc/sys/kernel/yama/ptrace_scope

1, indicates a restricted setting.  To update it so that another process with the same UID can attach to that processes memory space do the following (as root)

echo 0 | tee /proc/sys/kernel/yama/ptrace_scope

Now, checking that value should return 0

cat /proc/sys/kernel/yama/ptrace_scope

At this point you should now be able to do a heapdump with jmap on your process with a command similar to the following:

jmap -J-d64 -heap 2712

Generate a Random String of a Specified Size with a Shell Script

Posted on by rchapin

The following is a one-liner for generating a random string of a fixed size in bash, where the possible characters to use in the string are any digit, letter, and a newline.

By adding the newline, you are fairly sure to prevent getting one long line of text.

< /dev/urandom tr -dc "[:digit:][:alpha:][\n]" | head -c1000 > file.out

Connecting To a Test Kitchen Instance Via SFTP, SSH, or SCP

Posted on by rchapin

If you are using Chef and Test Kitchen to test your cookbooks you may have need to connect to the Test Kitchen VM in some other fashion other than $ kitchen login instance-name.

To do so:

Do a $ kitchen list to see the running vms

kitchen list
Instance                      Driver   Provisioner  Verifier  Transport  Last Action
default-centos-66             Vagrant  ChefSolo     Busser    Ssh        Converged

Then look in the .kitchen directory from where you ran your $ kitchen command and look for the corresponding .yml config file for the vm

ls .kitchen
default-centos-66.yml  kitchen-vagrant  logs

Inside the .yml file for the vm is the path to the ssh key

more .kitchen/default-centos-66.yml
---
hostname: 127.0.0.1
port: '2222'
username: vagrant
ssh_key: "/home/rchapin/.vagrant.d/insecure_private_key"
last_action: converge

With that, just use the -i argument to point to the identity file and sftp to the box

sftp -P 2222 -i /home/rchapin/.vagrant.d/insecure_private_key vagrant@127.0.0.1

If the VM prompts you for a user name when logging in, you can always login via kitchen, change the password for that user and then retry

To login via kitchen issue the following command:

kitchen login default-centos-66
© 2005 Ryan Chapin