Category: Computers

All things computing.

git Cheat Sheet

Posted on by rchapin

A handful of handy git commands that I don’t use all that often but want to keep track of:

Stashing

Stash a single file

git stash push -m 'message here' -- path/to/file

Drop a specific stash

First figure out the id of the stash you want to drop with git stash list, then issue the following command

git stash drop stash@{n}

Add all untracked files across the entire repository

git add -A

Editing Commits

Changing author of already pushed commit

If you need to change the author of a commit that has already pushed, interactive rebase is your friend

First, determine a commit on top of which you want to rebase the commit, or a series of commits. I usually just use the commit that the series of commits is based off of (the root of the branch).

In that case you can just rebase it against that branch (but the hash of a commit before the commit you want to edit works just as well)

git rebase -i main

Your editor will open and you will be able to mark the commit that you want to edit. Change the pick token for that commit to edit and save:quit from your editor. Git will then start rebasing each commit onto the branch/commit you specified and will stop when it reaches the commit you flagged for editing.

At this point, issue the following command (updating the specific email address and author name)

git commit --amend --author="Ryan Chapin <rchapin@nbinteractive.com>" --no-edit

And then issue the commit to continue rebasing the remainder of the commits.

Undo last commit and keep changes

Sometimes you mistakenly commit things that you are not ready to commit. Run the following

git reset --soft HEAD~1

This command will

  1. git reset: move the current branch’s pointer (HEAD) to a specific commit
  2. --soft: ensure that the changes from the undone commit are preserved in your staging area and working directory as uncommitted local modifications.
  3. HEAD~1: is the reference to the commit just before the current one – the parent of HEAD

gitk

gitk is a GUI git client that has a lot of nice features and is easy to use

See orphaned branches and commits

Run gitk with the following command. Running git log with the same arguments will enable you to see the same commits, but it is much easier to browse and see with gitk.

gitk --all --reflog &

Diffing

Viewing more or less context with git diff

The default number of lines displayed above and below a change is three. To increase or decrease the number of lines displayed in the diff use the -U flag as follows, which will display 10 lines before and after the changes.

git diff -U10

Finding changes to a specific file

git log --follow --patch -- <file-path>

Reverting

When reverting your un-staged changes to your current branch you are checking out the revision from the base branch.

Reverting Changes in an entire subdirectory

git checkout <base-branch> -- path/to/directory

Changing URLs

Changing the remote URL

git remote set-url origin ssh://git@<host>:<path>/REPOSITORY.git

Use printf to join an array in Bash

Posted on by rchapin

If you would like to join an array of elements with a defined delimiter in Bash there is an easy way to go about it by using printf. Following is an example

#!/bin/bash

declare -a arr=()

for i in `seq 1 5`
do
  arr=("${arr[@]}" $i)
done

# Generate a single string joined by a comma.  The printf string can contain
# any arbitrary delimiter.
printf -v joined '%s,' "${arr[@]}"

# Print out the string minus the trailing comma
echo "${joined%,}"

Specifying a commit in go.mod instead of a local replace for development

Posted on by rchapin

Sometimes you are making changes to a dependency in another of your go projects and instead of adding a replace command in the go.mod file you want to update that entry in go.mod to point to a specific commit in the repo.

To do so, all that you need to do is:

  1. Get the git commit that you want included in your build
  2. Change directories to the same directory that your project’s go.mod file resides in which you want to refer to the library with the specific commit
  3. Run the following command which will automatically update your go.mod file with the correct “version” for your dependency go get github.com/rchapin/rlog@<commit-hash>

Using Microsoft PowerRename to Rename Batches of Files

Posted on by rchapin

If you have to rename a large number of files under Windows it is very tedious to do it one-by-one via the gui. Instead of writing a batch file, Microsoft has a suite of tools called PowerToys. PowerToys installs a utility called PowerRename that will do the job.

I did this under Windows 10, but I imagine that it is the same in Windows 11 based on the documentation on the PowerToys installation page.

Installation

  1. Start a PowerShell as an admin user
    1. Click on Start and search for powershell
    2. Right click on Windows PowerShell (not PowerShell ISE) and select Run as administrator
  2. Enter the following command to install PowerToys: winget install Microsoft.PowerToys --source winget
  3. It will download the installed, check the hash of the binary, and execute the install. When I did it one of those “Do you want this app to make changes to this computer” prompts showed up in the task bar that I did not see right away and I had to click on the shield to maximize it and click OK to continue the installation.
  4. After the installation finished there was a “Welcome to PowerToys” window that provided some information about the package.
  5. I am not 100% sure if this is required, but I scrolled down until I found the PowerRename entry in the left-hand side bar and clicked on it.

Usage

Once installed navigate to the directory that contains the directory of files that you want to modify.

It is a good idea to make a backup of the folder before you make changes to it in-case something goes wrong and you want to start over. Simply right-click on the directory and select Send to -> Compressed (zipped) folder to create a zip file.

  1. Right-click on the folder that contains the files and select PowerRename
  2. In the dialog box that appears enter the string that you want to change in the Search for field, and enter the string to which you want it changed in the Replace with field
  3. There are additional settings that enable you to apply the change to just the filename and/or the extension
  4. The files that will be mutated are listed in the right-hand side of the dialog box
  5. Click apply and PowerRename will execute the configured changes on the files

Docker Cheat Sheet

Posted on by rchapin

Following are a number of my commonly used docker commands for my own reference

Building

Run the following in the same directory in which your Dockerfile resides

docker build -t <image-name>:<version> .

Or you can specify the path to the Dockerfile

docker build . -t <image-name>:<version> -f /path/to/Dockerfile

Running

Run a container interactively

Especially useful when debugging commands that you will encapsulate in a Docker file, this will enable you to run a base image and then execute commands interactively without having to continuously run docker build . to test things out. Note the -u root to start the interactive session as the root user. If you just want to get on the container and look around you can omit this argument.

docker run -it -u root <image:tag> <shell-in-image>

Copy files to and from a running container

Assuming that you already have the container running, first run docker ps to get the id of the container

# Copy file from local file system into the container
docker cp <file-path> <container-id>:<target-path-in-container>

# Copy file from container to local file system
docker cp <container-id>:<target-path-in-container> <file-path-on-local-filesystem>

Purging images and volumes

Remove all unused or dangling images, containers, and networks. The -a will remove any stopped containers and unused images

docker system prune [-a]

To remove dangling volumes you must run a docker volume rm <volume-name> for each volume you want to remove. Running docker volume prune does not remove unused volumes.

docker volume ls | tail -n +2 | awk '{print $2}' | xargs docker volume rm

Saving an image to a tarball

docker save -o my_image.tar my_image:latest

Git Merge Conflict Resolution Cheat Sheet

Posted on by rchapin

Some of git’s nomenclature can be confusing, especially since it is context dependent. Following are some TLDR;s for dealing with resolving merge conflicts in different scenarios.

–ours vs –theirs

The meaning of --ours vs --theirs can depend on whether you are doing a rebase or a merge.

Assuming that the feature branch is checked out

git merge developgit rebase develop
To keep changes from develop--theirs--ours
To keep changes from feature--ours--theirs

If, during a rebase there is a conflict and you know you want to take ALL of the changes from the branch you are rebasing onto, or ALL of the changes from the feature branch you can do the following (note that --ours vs --theirs follows the same semantics as described in the table above):

# Take all changes from the branch on which you are rebasing
git checkout --ours <path-to-file>

# Or, if you want to take all of the changes from the feature branch
git checkout --theirs <path-to-file>

# Then you can
git add <path-to-file>
git rebase --continue

Current Change vs. Incoming Change When Rebasing

Sometimes you may also see “Incoming” vs. “Current” changes when rebasing a feature branch. Assuming that the feature branch is checked out

git rebase develop
To keep changes from developAccept Current Change
To keep changes from featureAccept Incoming Change

If,

kubectl/k8s Cheat Sheet

Posted on by rchapin
  • Namespaces
    • List all namespaces: kubectl get namespace
    • Set a namespace: kubens <namespace-name>
    • See currently set namespace: kubens -c
  • Pods
    • List all pods: kubectl get pods
    • List all pods in specific namespace: kubectl get pods -n <namespace>
    • Kill a pod: kubectl delete pod <pod-name>
    • Describe/get details of pod: kubectl describe pods <pod-name>
    • InitContainers
      • Get logs: First describe the pod and look for the name of the init container. Then run kubectl logs <pod-name> -c <init-container-name>
  • Deployments
    • Get the deployment name from a pod: kubectl get pod -n -o jsonpath='{.metadata.ownerReferences[0].name}'
    • Get the manifest for a deployment: kubectl get deploy <deployment-name> -o yaml
    • Scaling a deployment: kubectl scale --replicas=<n> deployment/<deployment-name>
    • Gen env vars defined on a pod: kubectl exec <pod> -- env
    • Restart a deployment: kubectl -n <namespace> rollout restart deploy/<deployment-name>
  • ConfigMaps
    • View data in a ConfigMap: kubectl describe configmaps <name>
  • Port-Forwarding
    • kubectl port-forward <resource-type>/<name> <local-port>:<pod-port>
  • Secrets
    • Getting an unsealed secret: kubectl -n <namespace> get secret <secret-name> -o json | jq -r '.data | map_values(@base64d)'
  • List all resouces: kubectl api-resources

kubectl JSONPath — the must-know minimal set

Enough to read any jsonpath you’ll encounter and write the few that kubectl requires. For everything else, use -o json | jq.

ConstructMeansExample
{ }Expression delimiter; text outside braces is literal{.metadata.name}
.foo.barWalk into fields.status.phase
[*]All elements of an array.items[*]
[0]Index.items[0].metadata.name
{range …}{end}Loop over an array{range .items[*]}{.metadata.name}{"\n"}{end}
?(@.x=="y")Filter — pick array elements where a condition holds; @ = current element.status.conditions[?(@.type=="Ready")].status

The four common variants

# 1. pull one field
kubectl get pod mypod -o jsonpath='{.status.podIP}'

# 2. loop with literal text + newlines
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.phase}{"\n"}{end}'

# 3. the FILTER idiom — read a specific condition/container by name
kubectl get kustomization ionic-query-engine \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

# 4. kubectl wait (jsonpath is mandatory here)
kubectl wait pod/mypod --for=jsonpath='{.status.phase}'=Running --timeout=60s

What to memorize

The single most important construct is the filter [?(@.type=="...")] — it’s the non-obvious one, and it shows up constantly for reading conditions, containers, ports, etc. The rest you can infer on sight.

Bonus cousin (not jsonpath, but same niche and often cleaner for tables):

kubectl get pods -o custom-columns=NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status

helm Cheat Sheet

Posted on by rchapin

Development Tips and Tricks

Test Template Rendering

Run the following. Instead of it installing the chart it will render the template and display the output

helm install --debug --dry-run <release-name> <path-to-chart-dir>

To test with an overriding value

helm install <release-name> <path-to-chart-dir> --dry-run --debug --set k=v

Deployments

  • List releases: helm list
  • Get the manifest for a release: helm get manifest <release-name> [flags]

Implementing a Stack in Go

Posted on by rchapin

One of the key features in go is simplicity. As a result there are a number of utilities and data structures that are common in other high-level languages that do not come with the go stdlib.

One of them is a stack. Following is a very simple implementation of a stack that uses a slice to store the data.

The following is an implementation of a simple stack that takes any kind of pointer.

import "fmt"

type Stack[T any] struct {
	stack []*T
}

func (s *Stack[T]) IsEmpty() bool {
	return len(s.stack) == 0
}

func (s *Stack[T]) Push(q *T) {
	// Add the new value to the end of the slice, which acts as the "top" of the Stack
	s.stack = append(s.stack, q)
}

func (s *Stack[T]) Pop() *T {
	if s.IsEmpty() {
		return nil
	} else {
		// Get the index of the "top" of the Stack
		idx := len(s.stack) - 1
		// Retrieve the element from the stack that we are going to "pop" and return
		retval := s.stack[idx]
		// Remove it from the underlying slice by re-slicing the slice.
		s.stack = s.stack[:idx]
		return retval
	}
}

func main() {
	i := 1
	j := 2
	s1 := &Stack[int]{}
	s1.Push(&i)
	s1.Push(&j)

	fmt.Printf("Pop! = %d\n", *s1.Pop())
	fmt.Printf("Pop! = %d\n", *s1.Pop())
}

curl HTTPS Over an SSH Tunnel

Posted on by rchapin

If you want to execute curl commands on your local machine and connect to an HTTPS server that is only reachable from a bastion or other host through which you can only get to via SSH, the following is how you set up the SSH tunnel and execute the curl command.

The following will not work

# Create ssh tunnel
#
ssh -L localhost:8443:example.com:443 user@bastion.example.com

# Attempt to hit the endpoint otherwise accessible from bastion.example.com
# with curl -X GET https://example.com/v1/endpoint
#
curl -X GET https://localhost:8443/v1/endpoint

The reason that this does not work is that with the port forwarded ssh tunnel curl is unable to resolve the IP of the example.com HTTPS server on the other side of the connection on bastion.example.com and the connection fails.

If we execute curl -v we can see the details. Notice that curl is connecting to localhost:8443 and not resolving example.com

$ curl -v https://localhost:8443/subjects
*   Trying ::1:8443...
* Connected to localhost (::1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.example.com
*  start date: Aug  3 00:00:00 2022 GMT
*  expire date: Sep  1 23:59:59 2023 GMT
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x560827a1b2c0)
> GET /subjects HTTP/2
> Host: localhost:8443
> user-agent: curl/7.74.0
> accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 404 
< server: awselb/2.0
< date: Wed, 08 Feb 2023 15:20:04 GMT
< content-type: text/plain; charset=utf-8
< content-length: 0

Instead of just forwarding TCP packets we need to tell ssh client to setup a SOCKS5 proxy through which we will tunnel traffic.

ssh -D 8443 -f -C -q -N bastion.example.com

The -D option creates a SOCKS5 proxy server listening on port 8443 which tunnels the traffic over ssh to bastion.example.com from which hostnames for the destination webserver can be resolved. This creates a proxy server that enables you to connect to “dynamic” destinations on the other side of the tunnel.

The other options

-D 8443- start a SOCKS server listening on port 8443 on the localhost
-f - fork the process, running it in the background
-C - compress data
-q - quite mode
-N - indicate to the ssh client that there are no commands to be sent over the tunnel

Once you create the proxy and tunnel you can then execute curl commands as follows on the localhost telling curl to use the SOCKS5 proxy listening on localhost:8443

curl -v -x socks5h://0:8443 https://example.com/v1/endpoint

The verbose output shows that the SOCKS5 proxy is the connecting to example.com:443 over the tunnel and remotely resolving the IP to the correct HTTPS server on the other side of the tunnel.

$ curl -v -x socks5h://0:8443 https://example.com/v1/endpoint
*   Trying 0.0.0.0:8443...
* SOCKS5 connect to example.com:443 (remotely resolved)
* SOCKS5 request granted.
* Connected to 0 (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.example.com
*  start date: Aug  3 00:00:00 2022 GMT
*  expire date: Sep  1 23:59:59 2023 GMT
*  subjectAltName: host "example.com" matched cert's "*.example.com"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562ada1d52c0)
> GET /subjects HTTP/2
> Host: example.com
> user-agent: curl/7.74.0
> accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< date: Wed, 08 Feb 2023 15:22:08 GMT
< content-type: application/vnd.schemaregistry.v1+json
< content-length: 1937
< vary: Accept-Encoding, User-Agent
© 2005 Ryan Chapin