Category: Programming

Go template that properly renders delimiters for a slice of structs

Posted on by rchapin

It is common to write go text/templates that range over a slice of objects to write out a list or array of items that are separated by some delimiter defined in the template.

In the case of a JSON array of objects, the cleanest output would be a , separating each object without a leading or trailing comma.

Because go template if statements are falsey in that a 0 value will evaluate to false you can write a template as follows and it will only render a comma between items as it ranges over the slice

{
  "items":[
  {{- range $index, $item := .Items -}}
    {{- if $index -}},{{ end }}
    {
      "id": {{ $item.Id }}, "name": {{ $item.Name }}
    }
    {{- end }}
  ]
}

Click here for a working example.

Convert a Slice of Any Type to a CSV in Go

Posted on by rchapin

There are times, mostly for logging and debugging, when you have a slice of a given type that you would like to print to a log file or just convert to a CSV of values.

A quick and easy way to convert a slice to a CSV in Golang is to use the json module to Marshal it a JSON encoded array.

For example

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	ints := []int64{1, 2, 3, 4}
	intsCSV, _ := json.Marshal(ints)
	fmt.Printf("ints = %s\n", intsCSV)

	floats := []float64{3.14, 6.47}
	floatsCSV, _ := json.Marshal(floats)
	fmt.Printf("floats = %s\n", floatsCSV)
}

Use printf to join an array in Bash

Posted on by rchapin

If you would like to join an array of elements with a defined delimiter in Bash there is an easy way to go about it by using printf. Following is an example

#!/bin/bash

declare -a arr=()

for i in `seq 1 5`
do
  arr=("${arr[@]}" $i)
done

# Generate a single string joined by a comma.  The printf string can contain
# any arbitrary delimiter.
printf -v joined '%s,' "${arr[@]}"

# Print out the string minus the trailing comma
echo "${joined%,}"

Specifying a commit in go.mod instead of a local replace for development

Posted on by rchapin

Sometimes you are making changes to a dependency in another of your go projects and instead of adding a replace command in the go.mod file you want to update that entry in go.mod to point to a specific commit in the repo.

To do so, all that you need to do is:

  1. Get the git commit that you want included in your build
  2. Change directories to the same directory that your project’s go.mod file resides in which you want to refer to the library with the specific commit
  3. Run the following command which will automatically update your go.mod file with the correct “version” for your dependency go get github.com/rchapin/rlog@<commit-hash>

Implementing a Stack in Go

Posted on by rchapin

One of the key features in go is simplicity. As a result there are a number of utilities and data structures that are common in other high-level languages that do not come with the go stdlib.

One of them is a stack. Following is a very simple implementation of a stack that uses a slice to store the data.

The following is an implementation of a simple stack that takes any kind of pointer.

import "fmt"

type Stack[T any] struct {
	stack []*T
}

func (s *Stack[T]) IsEmpty() bool {
	return len(s.stack) == 0
}

func (s *Stack[T]) Push(q *T) {
	// Add the new value to the end of the slice, which acts as the "top" of the Stack
	s.stack = append(s.stack, q)
}

func (s *Stack[T]) Pop() *T {
	if s.IsEmpty() {
		return nil
	} else {
		// Get the index of the "top" of the Stack
		idx := len(s.stack) - 1
		// Retrieve the element from the stack that we are going to "pop" and return
		retval := s.stack[idx]
		// Remove it from the underlying slice by re-slicing the slice.
		s.stack = s.stack[:idx]
		return retval
	}
}

func main() {
	i := 1
	j := 2
	s1 := &Stack[int]{}
	s1.Push(&i)
	s1.Push(&j)

	fmt.Printf("Pop! = %d\n", *s1.Pop())
	fmt.Printf("Pop! = %d\n", *s1.Pop())
}

Diffing the output of two commands

Posted on by rchapin

The GNU diff command on most Linux and UNIX systems will diff the contents of two files. With Bash, you can, using process substitution, take the output of any arbitrary command and process its input, or output, as a file descriptor. In this way, you can then use diff against the output of two commands as follows

diff <(cmd1) <(cmd2)

Both cmd1 and cmd2 will appear as a file name/file descriptor. The < character indicates that the file descriptor should be read to obtain the output.

Declaring, Exporting, and Reading Dynamic Variables in Bash

Posted on by rchapin

If you want to dynamically define and export variable names in Bash here is the TLDR;

# Define the name of the variable
key="my-dynamic-var-name"

# Declare it and export it
declare -gx "$key"="some-value"

To then access that value via a dynamically generated variable name

# Create a variable that contains the variable name
var_to_access="my-dynamic-var-name"

# Read the value
my_var_value=${!var_to_access}

Read the man page for declare for more details and read this article for a really good explanation and further examples.

To read the declare man page

help declare

How to check if a file is sourced in Bash

Posted on by rchapin

Sometimes you will want to ensure that a file is sourced instead of executed. This ensures, among other things, that any environment variables that the script defines remain in your current shell after the script completes.

To do so, use the following to check whether the file was sourced or run in a sub-shell

(return 0 2>/dev/null) && sourced=1 || sourced=0
echo "sourced=$sourced"

Bash allows return statements only from functions and in a scripts top level scope IF it is sourced. If you try to use a return statement outside of a function in a non-sourced context it returns an error.

Pruning directories from find

Posted on by rchapin

I have no idea why, but for some reason I always have a hard time remembering the exact syntax for find when I want to prune some list of directories from a search.

Let’s say that you want to execute a find in a directory where there are a lot of .git directories and you don’t want to search through the guts of the repo directories. With the following command we specify the prune predicate ahead of the search for any file that has ‘*.json’ in the file name.

find ./ -type f -iwholename '*.git' -prune -o -name '*.json' -print

Another way to do it is to exclude specific directories from a search. With the following command we first specify a set of directories to exclude from the search, by specific path and name, and then execute a search for the specific files.

find ./ -type d \( -path ./grpc-java -o -path ./go-in-mem-datastore \) -prune -o -name '*.json' -print

Using cut with a delimiter of any amount of whitespace

Posted on by rchapin

The TLDR; is to first use tr to replace all occurrences of any horizontal whitespace character with a single space, and then squeeze down any number of spaces to a single space and then define the delimiter for cut as a single space. The following example assumes that you want to see from the 5th column to the end of the line.

<do-something-to-generate-input> | tr '[:blank:]' ' ' | tr -s ' ' | cut -d ' ' -f5-

The previous command will, after using -s to squeeze repeated spaces into one and then cut from the 5th field to the end of the line.

© 2005 Ryan Chapin