Category: Golang

Go template that properly renders delimiters for a slice of structs

Posted on by rchapin

It is common to write go text/templates that range over a slice of objects to write out a list or array of items that are separated by some delimiter defined in the template.

In the case of a JSON array of objects, the cleanest output would be a , separating each object without a leading or trailing comma.

Because go template if statements are falsey in that a 0 value will evaluate to false you can write a template as follows and it will only render a comma between items as it ranges over the slice

{
  "items":[
  {{- range $index, $item := .Items -}}
    {{- if $index -}},{{ end }}
    {
      "id": {{ $item.Id }}, "name": {{ $item.Name }}
    }
    {{- end }}
  ]
}

Click here for a working example.

Convert a Slice of Any Type to a CSV in Go

Posted on by rchapin

There are times, mostly for logging and debugging, when you have a slice of a given type that you would like to print to a log file or just convert to a CSV of values.

A quick and easy way to convert a slice to a CSV in Golang is to use the json module to Marshal it a JSON encoded array.

For example

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	ints := []int64{1, 2, 3, 4}
	intsCSV, _ := json.Marshal(ints)
	fmt.Printf("ints = %s\n", intsCSV)

	floats := []float64{3.14, 6.47}
	floatsCSV, _ := json.Marshal(floats)
	fmt.Printf("floats = %s\n", floatsCSV)
}

Specifying a commit in go.mod instead of a local replace for development

Posted on by rchapin

Sometimes you are making changes to a dependency in another of your go projects and instead of adding a replace command in the go.mod file you want to update that entry in go.mod to point to a specific commit in the repo.

To do so, all that you need to do is:

  1. Get the git commit that you want included in your build
  2. Change directories to the same directory that your project’s go.mod file resides in which you want to refer to the library with the specific commit
  3. Run the following command which will automatically update your go.mod file with the correct “version” for your dependency go get github.com/rchapin/rlog@<commit-hash>

Implementing a Stack in Go

Posted on by rchapin

One of the key features in go is simplicity. As a result there are a number of utilities and data structures that are common in other high-level languages that do not come with the go stdlib.

One of them is a stack. Following is a very simple implementation of a stack that uses a slice to store the data.

The following is an implementation of a simple stack that takes any kind of pointer.

import "fmt"

type Stack[T any] struct {
	stack []*T
}

func (s *Stack[T]) IsEmpty() bool {
	return len(s.stack) == 0
}

func (s *Stack[T]) Push(q *T) {
	// Add the new value to the end of the slice, which acts as the "top" of the Stack
	s.stack = append(s.stack, q)
}

func (s *Stack[T]) Pop() *T {
	if s.IsEmpty() {
		return nil
	} else {
		// Get the index of the "top" of the Stack
		idx := len(s.stack) - 1
		// Retrieve the element from the stack that we are going to "pop" and return
		retval := s.stack[idx]
		// Remove it from the underlying slice by re-slicing the slice.
		s.stack = s.stack[:idx]
		return retval
	}
}

func main() {
	i := 1
	j := 2
	s1 := &Stack[int]{}
	s1.Push(&i)
	s1.Push(&j)

	fmt.Printf("Pop! = %d\n", *s1.Pop())
	fmt.Printf("Pop! = %d\n", *s1.Pop())
}

Code Coverage for Golang Integration Tests with Build Tags

Posted on by rchapin

I’ve been writing a lot of Golang code lately. For the most part, I always write unit and integration tests for whatever program that I am building. I’ve been using VSCode as my IDE and really like it. The one thing that I was having trouble with was getting the code coverage to show in the IDE for my integration tests.

I have a different build tag in my unit test code and my integration test code so that I can run the two separately in my CI/CD pipeline. The following build tag is in all of the integration code

// +build integration

I have the following build tag in all of my unit tests

// +build unit

VSCode has the hooks to enable configuration to run and show the coverage of the unit test code in the UI, but not for the integration tests.

I was able to get a visual representation of my code coverage based on my integration tests by running the native go tools. As of Golang 1.2 you can now display test coverage results. It does require a separate cover program that can be installed. Run the following in your project

 go get golang.org/x/tools/cmd/cover

Once that is installed you can run the following to generate a coverage profile file

go test -cover -coverpkg github.com/rchapin/mypackage -tags=integration ./... -coverprofile=/var/tmp/c.out

-cover enables the coverage analysis. -coverpkg will apply coverage analysis to in the tests to the packages that match the CSV of patterns provided. -coverprofile will write out a coverage profile to the path given.

After that has finished, run the following to generate an HTML representation of the coverage profile

go tool cover -html=/var/tmp/c.out

Configuring a Bitbucket Pipeline for a Golang Project with Privately Hosted Dependencies

Posted on by rchapin

Go’s dependency management is based solely on git repos. Most of the libraries have publicly available repos from which you can clone the source and then build your project.

In my case, most of the Go that I’m writing ends up in private repos in Bitbucket. Currently, I’ve got a program that I am writing that is dependent on a library that I have written that is hosted as a private repo in Bitbucket. I wanted to setup a Bitbucket Pipeline to build and then run the unit and integration tests when a PR was submitted.

Initially, I had a bit of a hard time figuring out how to set up the pipeline to work correctly and posted this Stack Overflow question. Not being one to give up, I kept at it and stumbled upon some Atlassian documentation that gave me what I needed to get it sorted out.

The key (no pun intended) ended up being the distribution of SSH keys between the repos and some additional go specific environment variables in the pipeline code.

bitbucket-pipelines.yml

image: golang:1.16

pipelines:
  pull-requests:
    '**':
      - step:
          name: Build and test
          script:
            - /bin/bash build/bitbucket-pipelines.sh

The pipeline yaml is straightforward. I’m using a golang builder image, defining a pull-request trigger for any branch, and specifying a shell script that contains the details for buiding and running my tests. The script is in the build directory which is in the base of my repository.

bitbucket-pipelines.sh

# Export the following go specific env vars to configure the tool chain so that I can clone
# repos from my private Bitbucket project.
export GONOPROXY="bitbucket.org/my-project"
export GONOSUMDB="bitbucket.org/my-project"
export GOPRIVATE="bitbucket.org/my-project"

# Configure git to force using ssh instead of https to clone the repos hosted in
# my Bitbucket project
git config --global url."git@bitbucket.org:my-project".insteadof "https://bitbucket.org/my-project"

CGO_ENABLED=0 go build
CGO_ENABLED=0 go test -count=1 -tags=unit ./...
CGO_ENABLED=0 go test -count=1 -tags=integration ./integration_tests/

SSH Key Configuration

With this basic pipeline code you then need to create a distribute ssh keys between the projects

Create the Pipeline SSH key pair in the Repo where you are running the pipeline

  1. In the repo where you are running the build, and composing the library, go to Repository settings. Then under PIPELINES, click on SSH keys
  2. Click Generate keys to create a new key pair.
  3. Copy the Public key to your clipboard. This public key will be added to all of dependent repos to enable us to clone and build with their source code.

For each of the dependency repos do the following:

  1. Go to Repository Settings and under GENERAL click on Access keys
  2. Click Add key and enter a Label and then paste the public key in the Key field.
  3. Then click Add SSH key

Once the public keys have been distributed to the dependent repositories your pipeline will have the permissions required to clone during the build.

© 2005 Ryan Chapin