Convert a Slice of Any Type to a CSV in Go

Posted on by rchapin

There are times, mostly for logging and debugging, when you have a slice of a given type that you would like to print to a log file or just convert to a CSV of values.

A quick and easy way to convert a slice to a CSV in Golang is to use the json module to Marshal it a JSON encoded array.

For example

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	ints := []int64{1, 2, 3, 4}
	intsCSV, _ := json.Marshal(ints)
	fmt.Printf("ints = %s\n", intsCSV)

	floats := []float64{3.14, 6.47}
	floatsCSV, _ := json.Marshal(floats)
	fmt.Printf("floats = %s\n", floatsCSV)
}

PostgreSQL Query to Find Tables With Name LIKE

Posted on by rchapin

Sometimes you are working with a PostgreSQL database with A LOT of tables and looking for tables that contain a sub-string in their name. Following is a query that you can run that will return all of the tables that have the string in their name:

SELECT
  table_schema,
  table_name
FROM
  information_schema.tables
WHERE
      table_name LIKE '%<string>%'
  AND table_schema not in ('information_schema', 'pg_catalog')
  AND table_type = 'BASE TABLE'
ORDER BY
  table_name, table_schema
;

Earthly Cheat Sheet

Posted on by rchapin

I have just recently started using Earthly and the following is a list of commands that I want to keep track of

Caching

Completely clearing the cache: Earthly stores artifacts in docker volumes. If you want to completely flush that data and start fresh run the following

docker stop earthly-buildkitd && \
docker rm earthly-buildkitd && \
docker volume rm earthly-cache

git Cheat Sheet

Posted on by rchapin

A handful of handy git commands that I don’t use all that often but want to keep track of:

Stashing

Stash a single file

git stash push -m 'message here' -- path/to/file

Drop a specific stash

First figure out the id of the stash you want to drop with git stash list, then issue the following command

git stash drop stash@{n}

Add all untracked files across the entire repository

git add -A

Editing Commits

Changing author of already pushed commit

If you need to change the author of a commit that has already pushed, interactive rebase is your friend

First, determine a commit on top of which you want to rebase the commit, or a series of commits. I usually just use the commit that the series of commits is based off of (the root of the branch).

In that case you can just rebase it against that branch (but the hash of a commit before the commit you want to edit works just as well)

git rebase -i main

Your editor will open and you will be able to mark the commit that you want to edit. Change the pick token for that commit to edit and save:quit from your editor. Git will then start rebasing each commit onto the branch/commit you specified and will stop when it reaches the commit you flagged for editing.

At this point, issue the following command (updating the specific email address and author name)

git commit --amend --author="Ryan Chapin <rchapin@nbinteractive.com>" --no-edit

And then issue the commit to continue rebasing the remainder of the commits.

Undo last commit and keep changes

Sometimes you mistakenly commit things that you are not ready to commit. Run the following

git reset --soft HEAD~1

This command will

  1. git reset: move the current branch’s pointer (HEAD) to a specific commit
  2. --soft: ensure that the changes from the undone commit are preserved in your staging area and working directory as uncommitted local modifications.
  3. HEAD~1: is the reference to the commit just before the current one – the parent of HEAD

gitk

gitk is a GUI git client that has a lot of nice features and is easy to use

See orphaned branches and commits

Run gitk with the following command. Running git log with the same arguments will enable you to see the same commits, but it is much easier to browse and see with gitk.

gitk --all --reflog &

Diffing

Viewing more or less context with git diff

The default number of lines displayed above and below a change is three. To increase or decrease the number of lines displayed in the diff use the -U flag as follows, which will display 10 lines before and after the changes.

git diff -U10

Finding changes to a specific file

git log --follow --patch -- <file-path>

Reverting

When reverting your un-staged changes to your current branch you are checking out the revision from the base branch.

Reverting Changes in an entire subdirectory

git checkout <base-branch> -- path/to/directory

Changing URLs

Changing the remote URL

git remote set-url origin ssh://git@<host>:<path>/REPOSITORY.git

Use printf to join an array in Bash

Posted on by rchapin

If you would like to join an array of elements with a defined delimiter in Bash there is an easy way to go about it by using printf. Following is an example

#!/bin/bash

declare -a arr=()

for i in `seq 1 5`
do
  arr=("${arr[@]}" $i)
done

# Generate a single string joined by a comma.  The printf string can contain
# any arbitrary delimiter.
printf -v joined '%s,' "${arr[@]}"

# Print out the string minus the trailing comma
echo "${joined%,}"

Specifying a commit in go.mod instead of a local replace for development

Posted on by rchapin

Sometimes you are making changes to a dependency in another of your go projects and instead of adding a replace command in the go.mod file you want to update that entry in go.mod to point to a specific commit in the repo.

To do so, all that you need to do is:

  1. Get the git commit that you want included in your build
  2. Change directories to the same directory that your project’s go.mod file resides in which you want to refer to the library with the specific commit
  3. Run the following command which will automatically update your go.mod file with the correct “version” for your dependency go get github.com/rchapin/rlog@<commit-hash>

Using Microsoft PowerRename to Rename Batches of Files

Posted on by rchapin

If you have to rename a large number of files under Windows it is very tedious to do it one-by-one via the gui. Instead of writing a batch file, Microsoft has a suite of tools called PowerToys. PowerToys installs a utility called PowerRename that will do the job.

I did this under Windows 10, but I imagine that it is the same in Windows 11 based on the documentation on the PowerToys installation page.

Installation

  1. Start a PowerShell as an admin user
    1. Click on Start and search for powershell
    2. Right click on Windows PowerShell (not PowerShell ISE) and select Run as administrator
  2. Enter the following command to install PowerToys: winget install Microsoft.PowerToys --source winget
  3. It will download the installed, check the hash of the binary, and execute the install. When I did it one of those “Do you want this app to make changes to this computer” prompts showed up in the task bar that I did not see right away and I had to click on the shield to maximize it and click OK to continue the installation.
  4. After the installation finished there was a “Welcome to PowerToys” window that provided some information about the package.
  5. I am not 100% sure if this is required, but I scrolled down until I found the PowerRename entry in the left-hand side bar and clicked on it.

Usage

Once installed navigate to the directory that contains the directory of files that you want to modify.

It is a good idea to make a backup of the folder before you make changes to it in-case something goes wrong and you want to start over. Simply right-click on the directory and select Send to -> Compressed (zipped) folder to create a zip file.

  1. Right-click on the folder that contains the files and select PowerRename
  2. In the dialog box that appears enter the string that you want to change in the Search for field, and enter the string to which you want it changed in the Replace with field
  3. There are additional settings that enable you to apply the change to just the filename and/or the extension
  4. The files that will be mutated are listed in the right-hand side of the dialog box
  5. Click apply and PowerRename will execute the configured changes on the files

Docker Cheat Sheet

Posted on by rchapin

Following are a number of my commonly used docker commands for my own reference

Building

Run the following in the same directory in which your Dockerfile resides

docker build -t <image-name>:<version> .

Or you can specify the path to the Dockerfile

docker build . -t <image-name>:<version> -f /path/to/Dockerfile

Running

Run a container interactively

Especially useful when debugging commands that you will encapsulate in a Docker file, this will enable you to run a base image and then execute commands interactively without having to continuously run docker build . to test things out. Note the -u root to start the interactive session as the root user. If you just want to get on the container and look around you can omit this argument.

docker run -it -u root <image:tag> <shell-in-image>

Copy files to and from a running container

Assuming that you already have the container running, first run docker ps to get the id of the container

# Copy file from local file system into the container
docker cp <file-path> <container-id>:<target-path-in-container>

# Copy file from container to local file system
docker cp <container-id>:<target-path-in-container> <file-path-on-local-filesystem>

Purging images and volumes

Remove all unused or dangling images, containers, and networks. The -a will remove any stopped containers and unused images

docker system prune [-a]

To remove dangling volumes you must run a docker volume rm <volume-name> for each volume you want to remove. Running docker volume prune does not remove unused volumes.

docker volume ls | tail -n +2 | awk '{print $2}' | xargs docker volume rm

Saving an image to a tarball

docker save -o my_image.tar my_image:latest

Git Merge Conflict Resolution Cheat Sheet

Posted on by rchapin

Some of git’s nomenclature can be confusing, especially since it is context dependent. Following are some TLDR;s for dealing with resolving merge conflicts in different scenarios.

–ours vs –theirs

The meaning of --ours vs --theirs can depend on whether you are doing a rebase or a merge.

Assuming that the feature branch is checked out

git merge developgit rebase develop
To keep changes from develop--theirs--ours
To keep changes from feature--ours--theirs

If, during a rebase there is a conflict and you know you want to take ALL of the changes from the branch you are rebasing onto, or ALL of the changes from the feature branch you can do the following (note that --ours vs --theirs follows the same semantics as described in the table above):

# Take all changes from the branch on which you are rebasing
git checkout --ours <path-to-file>

# Or, if you want to take all of the changes from the feature branch
git checkout --theirs <path-to-file>

# Then you can
git add <path-to-file>
git rebase --continue

Current Change vs. Incoming Change When Rebasing

Sometimes you may also see “Incoming” vs. “Current” changes when rebasing a feature branch. Assuming that the feature branch is checked out

git rebase develop
To keep changes from developAccept Current Change
To keep changes from featureAccept Incoming Change

If,

kubectl/k8s Cheat Sheet

Posted on by rchapin
  • Namespaces
    • List all namespaces: kubectl get namespace
    • Set a namespace: kubens <namespace-name>
    • See currently set namespace: kubens -c
  • Pods
    • List all pods: kubectl get pods
    • List all pods in specific namespace: kubectl get pods -n <namespace>
    • Kill a pod: kubectl delete pod <pod-name>
    • Describe/get details of pod: kubectl describe pods <pod-name>
    • InitContainers
      • Get logs: First describe the pod and look for the name of the init container. Then run kubectl logs <pod-name> -c <init-container-name>
  • Deployments
    • Get the deployment name from a pod: kubectl get pod -n -o jsonpath='{.metadata.ownerReferences[0].name}'
    • Get the manifest for a deployment: kubectl get deploy <deployment-name> -o yaml
    • Scaling a deployment: kubectl scale --replicas=<n> deployment/<deployment-name>
    • Gen env vars defined on a pod: kubectl exec <pod> -- env
    • Restart a deployment: kubectl -n <namespace> rollout restart deploy/<deployment-name>
  • ConfigMaps
    • View data in a ConfigMap: kubectl describe configmaps <name>
  • Port-Forwarding
    • kubectl port-forward <resource-type>/<name> <local-port>:<pod-port>
  • Secrets
    • Getting an unsealed secret: kubectl -n <namespace> get secret <secret-name> -o json | jq -r '.data | map_values(@base64d)'
  • List all resouces: kubectl api-resources

kubectl JSONPath — the must-know minimal set

Enough to read any jsonpath you’ll encounter and write the few that kubectl requires. For everything else, use -o json | jq.

ConstructMeansExample
{ }Expression delimiter; text outside braces is literal{.metadata.name}
.foo.barWalk into fields.status.phase
[*]All elements of an array.items[*]
[0]Index.items[0].metadata.name
{range …}{end}Loop over an array{range .items[*]}{.metadata.name}{"\n"}{end}
?(@.x=="y")Filter — pick array elements where a condition holds; @ = current element.status.conditions[?(@.type=="Ready")].status

The four common variants

# 1. pull one field
kubectl get pod mypod -o jsonpath='{.status.podIP}'

# 2. loop with literal text + newlines
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.phase}{"\n"}{end}'

# 3. the FILTER idiom — read a specific condition/container by name
kubectl get kustomization ionic-query-engine \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

# 4. kubectl wait (jsonpath is mandatory here)
kubectl wait pod/mypod --for=jsonpath='{.status.phase}'=Running --timeout=60s

What to memorize

The single most important construct is the filter [?(@.type=="...")] — it’s the non-obvious one, and it shows up constantly for reading conditions, containers, ports, etc. The rest you can infer on sight.

Bonus cousin (not jsonpath, but same niche and often cleaner for tables):

kubectl get pods -o custom-columns=NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status

© 2005 Ryan Chapin