Category: Programming

VS Code “Test result not found for:” When Running Tests for a Python Project [SOLVED]

Posted on by rchapin

I finally was able to get Visual Studio Code set-up correctly to run and debug unit and integration tests for a Python 3.8 project that I am working on (I’ll add a link to that post here once it is up).

After making some changes to the code and adding a test I got the following error when trying to debug the test:

Test result not found for: ./mylibs/integration_tests/myclient_integration_test.py::MyClientIntegrationTest::test_happy_path

? An odd error message, to be sure.

After a little while I figured out that when this happens it is ultimately the result of some syntax, interpretation error that occurs at runtime that the IDE may not flag as a problem for you.

Check the Output panel and click on the drop-down and select Python Test Log to see the stack trace of the error to see where you have a typo.

Compiling Python Under Linux

Posted on by rchapin

The following should work with just about any version of Python. I am using it to compile, currently 3.10.x, on distros where those packages are not readily available for installation. The following is a quick how to on getting it compiled under both RedHat/CentOS/Almalinux and Debian based systems.

Download the Tarball for the Version You Want To Install

Download the tar.gz archive for the version that you want to install from here. Verify the download and then save the path to this file for later.

Install Dependencies

This assumes that you already have the “build-essentials” and kernel headers installed on the box, which is an exercise for the reader.

RedHat/CentOS/Almalinux

yum install -y bzip2-devel expat-devel gdbm-devel ncurses-devel openssl-devel readline-devel wget sqlite-devel tk-devel xz-devel zlib-devel libffi-devel gmp-devel libmpc-devel mpfr-devel openssl-devel liblzma-devel

Debian

apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev liblzma-dev

Compile Python

The following enables a non-root user to unpack, compile, and install it into their home directory. Copy this file to /var/tmp/compile-python.sh and then run as follows

/var/tmp/compile-python.sh <path-to-tarball>
#!/bin/bash

set -u
set -e

# The path to the downloaded tarball
py_tarball=$1

export PY_DIR=$(echo $py_tarball | awk -F/ '{ print $NF }' | sed 's/.tgz//')
export PY_PREFIX=$(echo ~/usr/local/$PY_DIR | tr [:upper:] [:lower:])

mkdir -p ~/usr/local/src ~/usr/local/bin ~/usr/local/include $PY_PREFIX
rm -rf $PY_PREFIX
tar -xzf $py_tarball -C ~/usr/local/src/
cd ~/usr/local/src/$PY_DIR
./configure --prefix=$PY_PREFIX --exec-prefix=$PY_PREFIX
make && make install

Add the following to your PATH in ~/.bash_profile

PYTHON_HOME=~/usr/local/python-<version>

export PATH=$PATH:$PYTHON_HOME/bin

Using fc to Edit and Re-execute Bash Commands

Posted on by rchapin

I recently learned about the Bash built-in fc. It is a great tool that enables you to edit and re-execute commands from your bash history.

Oftentimes there is a command in your history that instead of just grepping through the history and then re-executing as-is you’ll want to make a modification or two. With fc you can first edit it in your favorite editor and then when closing the editor fc will execute the command.

For me, vim is my editor of choice. Add the following to your .bashrc and fc will automatically open vim for you.

export FCEDIT=vim

Then, simply run fc passing it the id of the command in your history that you want to edit and then execute.

fc 1234

Creating a Launch Config in VSCode to Debug a Python Invoke Script

Posted on by rchapin

I regularly use Python Invoke and Fabric for the automation of various tasks; from deploying code to developing my own set of tools for various projects. Following is an example on how to write a launch.json launch configuration for vscode so that you can step through the tasks.py code and debug it.

Assuming that you have created a virtual environment and pip installed invoke into it. And, assuming that you have defined a task in your tasks.py file as follows:

from invoke import task

@task()
def do_something(ctx, some_path, some_other_path):
    # Do something with data in these dirs . . .

The following is a template you can use for a launch configuration that you can use to debug your task. 

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "invoke",
      "type": "python",
      "request": "launch",
      // The complete path to the invoke python script in your virtual environment
      "program": "/my/virtualenv/path/bin/invoke",
      "justMyCode": false,
      // The args that you would otherwise enter on the command line
      // when invoking your task
      "args": [
        "do-something",
        "--some-path",
        "/var/tmp/a/",
        "--some-other-path",
        "/var/tmp/b/"
      ],
      "cwd": "/the/path/to/the/dir/that/contains/your/tasks/script",
    }
  ]
}

Flush Commands to BASH History Immediately

Posted on by rchapin

I cannot take credit for figuring this one out. Original post is here.

TLDR; is to add the following to your ~/.bashrc

export PROMPT_COMMAND='history -a'

Following are the history configs that I use

######################################################################
shopt -s histappend
HISTSIZE=-1
HISTFILESIZE=-1
HISTCONTROL=ignoreboth
HISTTIMEFORMAT="[%F %T] "
export PROMPT_COMMAND='history -a'
######################################################################

Setting Per File Type Tab Configurations in VSCode

Posted on by rchapin

If you would like to have different tab configurations (tabs or spaces) along with the number of tab chars for different file types you can update your user settings.

The first thing you need to do is figure out what the file type code thinks the file that you want to change is. Open the file in vscode and then look at the bottom right of your window. In my case, I’m looking at an avro schema (.avsc) file:

In this case, code sees this file type as “JSON with Comments” and is configured to use spaces with 2 spaces per “tab”.

In order to change that, press CTRL+Shift+P and select “Preferences: Configure Language Specific Settings”. You will then be presented with a host of file types. Scroll down to the entry you want (JSON with comments) and take note of of the string within the parenthesis. That is the key that you will need to make an entry into settings.json. If you click on that entry it will open up your user settings.json. I was baffled by what to do next until I saw this stackoverflow post.

Once I read that, I knew what to add to the settings.json. Simply add a new key, defined at the root of the JSON object that is the string displayed in the parenthesis that you saw for the file type in the dropdown that was displayed after selecting “Preferences: Configure Language Specific Settings”.

The JSON in settings.json will look like the following

   "[jsonc]": {
        "editor.tabSize": 2,
        "editor.detectIndentation": false,
        "editor.insertSpaces": true,
        "editor.quickSuggestions": {
            "strings": true
        },
        "editor.suggest.insertMode": "replace",
        "gitlens.codeLens.scopes": [
            "document"
        ]
    }

Check the language specific settings docs for the configuration options.

VSCode Keyboard Shortcut to Toggle Visibility of the Explorer Side Panel

Posted on by rchapin

I usually have at least two panes in my IDE so that I can see two files, or different parts of the same file, at the same time. VSCode pegs the debug variables in the Explorer side bar so I also end up having to make that panel large at times to see the variables while debugging.

Following are the keybindings that you can add to enable you to toggle the visibility of the left-hand side panel.

Add the following to the keybindings.json file by typing CTRL+Shift+P and then enter Open Keyboard Shortcuts (JSON)

Under Linux this file should be in ~/.config/Code/User/keybindings.json

    {
        "key": "cmd+b",
        "command": "workbench.view.explorer"
    },
    {
        "key": "cmd+b",
        "command": "-workbench.view.explorer"
    },
    {
        "key": "cmd+b",
        "command": "workbench.action.toggleSidebarVisibility",
        "when": "explorerViewletVisible"
    },
    {
        "key": "cmd+b",
        "command": "-workbench.action.toggleSidebarVisibility",
        "when": "explorerViewletVisible"
    }

Code Coverage for Golang Integration Tests with Build Tags

Posted on by rchapin

I’ve been writing a lot of Golang code lately. For the most part, I always write unit and integration tests for whatever program that I am building. I’ve been using VSCode as my IDE and really like it. The one thing that I was having trouble with was getting the code coverage to show in the IDE for my integration tests.

I have a different build tag in my unit test code and my integration test code so that I can run the two separately in my CI/CD pipeline. The following build tag is in all of the integration code

// +build integration

I have the following build tag in all of my unit tests

// +build unit

VSCode has the hooks to enable configuration to run and show the coverage of the unit test code in the UI, but not for the integration tests.

I was able to get a visual representation of my code coverage based on my integration tests by running the native go tools. As of Golang 1.2 you can now display test coverage results. It does require a separate cover program that can be installed. Run the following in your project

 go get golang.org/x/tools/cmd/cover

Once that is installed you can run the following to generate a coverage profile file

go test -cover -coverpkg github.com/rchapin/mypackage -tags=integration ./... -coverprofile=/var/tmp/c.out

-cover enables the coverage analysis. -coverpkg will apply coverage analysis to in the tests to the packages that match the CSV of patterns provided. -coverprofile will write out a coverage profile to the path given.

After that has finished, run the following to generate an HTML representation of the coverage profile

go tool cover -html=/var/tmp/c.out

Running Multiple VSCode Windows from the Same Workspace

Posted on by rchapin

In order to have two separate VSCode windows open that point to the same workspace, press CTRL+Shift+P and then search for Duplicate As Workspace in New Window. This will open a new window that is associated with the current workspace.

The thing to keep in mind is that it just DUPLICATED the existing workspace in a new window. Notice that at the top of the EXPLORER it will say UNTITLED (WORKSPACE). You can now work with two windows editing your code, but when you quit out of VSCode close the duplicate workspace first. It will ask you to save the workspace. You can tell it to close without saving.

Configuring a Bitbucket Pipeline for a Golang Project with Privately Hosted Dependencies

Posted on by rchapin

Go’s dependency management is based solely on git repos. Most of the libraries have publicly available repos from which you can clone the source and then build your project.

In my case, most of the Go that I’m writing ends up in private repos in Bitbucket. Currently, I’ve got a program that I am writing that is dependent on a library that I have written that is hosted as a private repo in Bitbucket. I wanted to setup a Bitbucket Pipeline to build and then run the unit and integration tests when a PR was submitted.

Initially, I had a bit of a hard time figuring out how to set up the pipeline to work correctly and posted this Stack Overflow question. Not being one to give up, I kept at it and stumbled upon some Atlassian documentation that gave me what I needed to get it sorted out.

The key (no pun intended) ended up being the distribution of SSH keys between the repos and some additional go specific environment variables in the pipeline code.

bitbucket-pipelines.yml

image: golang:1.16

pipelines:
  pull-requests:
    '**':
      - step:
          name: Build and test
          script:
            - /bin/bash build/bitbucket-pipelines.sh

The pipeline yaml is straightforward. I’m using a golang builder image, defining a pull-request trigger for any branch, and specifying a shell script that contains the details for buiding and running my tests. The script is in the build directory which is in the base of my repository.

bitbucket-pipelines.sh

# Export the following go specific env vars to configure the tool chain so that I can clone
# repos from my private Bitbucket project.
export GONOPROXY="bitbucket.org/my-project"
export GONOSUMDB="bitbucket.org/my-project"
export GOPRIVATE="bitbucket.org/my-project"

# Configure git to force using ssh instead of https to clone the repos hosted in
# my Bitbucket project
git config --global url."git@bitbucket.org:my-project".insteadof "https://bitbucket.org/my-project"

CGO_ENABLED=0 go build
CGO_ENABLED=0 go test -count=1 -tags=unit ./...
CGO_ENABLED=0 go test -count=1 -tags=integration ./integration_tests/

SSH Key Configuration

With this basic pipeline code you then need to create a distribute ssh keys between the projects

Create the Pipeline SSH key pair in the Repo where you are running the pipeline

  1. In the repo where you are running the build, and composing the library, go to Repository settings. Then under PIPELINES, click on SSH keys
  2. Click Generate keys to create a new key pair.
  3. Copy the Public key to your clipboard. This public key will be added to all of dependent repos to enable us to clone and build with their source code.

For each of the dependency repos do the following:

  1. Go to Repository Settings and under GENERAL click on Access keys
  2. Click Add key and enter a Label and then paste the public key in the Key field.
  3. Then click Add SSH key

Once the public keys have been distributed to the dependent repositories your pipeline will have the permissions required to clone during the build.

© 2005 Ryan Chapin