List the Roles for a User or Service Account in a Specific GCP Project

If you do not have web console permissions to do so, but have the ability to activate a service account that has the viewer permissions or IAM permissons to list IAM roles in a given project, the following is how you can list the roles for a given user or service account.

gcloud projects get-iam-policy <gcp-project\
--filter="bindings.members:<email-address" \
--flatten="bindings[].members" --format="table(bindings.role)"
Continue reading “List the Roles for a User or Service Account in a Specific GCP Project”

Using bq load Command to Load logicalType Partitioned Data into a BigQuery Table

Following is the syntax and bq load command that you need to issue if you want to load data in avro file into a partitioned BigQuery table based on avro field defined as a logicalType.

Given the following schema

{
  "type" : "record",
  "name" : "logicalType",
  "namespace" : "com.ryanchapin.tests",
  "fields" : [ {
    "name" : "id",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "value",
    "type" : [ "null", "long" ],
    "default" : null
  }, 
Continue reading “Using bq load Command to Load logicalType Partitioned Data into a BigQuery Table”

[SOLVED] Unable to Sign-In to Gmail with Thunderbird with OAuth2, Keeps Asking for Email or Phone Over and Over

If you are setting up Thunderbird to use your Gmail account you may find that when Thunderbird opens a new window to a Google web portal into which you are to provide your email address and password that it will keep asking you over and over again for your email and never enable to you to enter the password.

This occurs when Thunderbird’s privacy settings do not allow it to store cookies.

First, ensure that your gmail account has Allow Continue reading “[SOLVED] Unable to Sign-In to Gmail with Thunderbird with OAuth2, Keeps Asking for Email or Phone Over and Over”

Mocking an HTTPS RESTful endpoint with Netcat

Netcat is generally known as a TCP/IP Swiss Army Knife and is incredibly helpful for both debugging and mocking up network services

Following is an example on how to setup a mock RESTful service that communicates over HTTPS.

On the “server” side, run the following command.  The -l command instructs Netcat to listen.

while true; do { echo -e “HTTP/1.1 200 OK\r\n$(date)\r\n\r\n<h1>hello world from $(hostname) on $(date)</h1>” |  nc -vl –ssl 8080; } done

On the “client” side, run the → Continue reading “Mocking an HTTPS RESTful endpoint with Netcat”

Using netcat to Mock a RESTful Webservice that Returns JSON

Let’s say that you are working on a part of a project that needs to consume some JSON data from a forthcoming HTTP service.  That sub-system is being developed by one of your colleagues and is not yet ready for you to stand-up to develop against.

You can use netcat to mock the webservice and return some static JSON data for which you can develop and test against with a simple one-liner.

First, put together your JSON in a file, → Continue reading “Using netcat to Mock a RESTful Webservice that Returns JSON”

How to Use Credentials That Contain Special Characters with curl

In order to execute curl commands to endpoints with passwords that contain special characters, the cleanest way that I have found to do so is to Base64 encode the authentication string for curl and then pass an Authorization request header along with the request.

In this example the credentials are uid ‘rchapin’ and passwd ‘abc123!@#’.  Normally we would pass this to curl as follows:

$ curl -u rchapin:abc123!@# -X GET https://some-endpoint:443

However, this will not work and the password will → Continue reading “How to Use Credentials That Contain Special Characters with curl”

[SOLVED] Configuring chrooted bind and rndc-confgen Hangs Not Generating a Key

I am putting together a chrooted installation of named and ran into a problem whereby attempting to generate an rndc.key with rndc-confgen just hangs, never returning and not generating a key.

After doing some searching I discovered that I needed to run the command as follows:

rndc-confgen -a -r /dev/urandom  -t /var/named/chroot

Which outputs the following, generating the key file that I expected.

wrote key file "/etc/rndc.key"
wrote key file "/var/named/chroot/etc/rndc.key"

Continue reading “[SOLVED] Configuring chrooted bind and rndc-confgen Hangs Not Generating a Key”

Installing Chrome Extensions Without Signing in With a Google Account

Google requires that you login with a Google account before you can install any Chrome extensions.

The following is how to install an extension without logging in (under Windows.  The same should work under Linux and Mac):

  1. Find the ID for the extension.  When you browse the extension in the store you will see a URL similar to the following:  https://chrome.google.com/webstore/detail/cookies/iphcomljdfghbkdcfndaijbokpgddeno?hl=en.   The hash string after the ‘cookies’ string (the name of the extension) up to the ? is the
Continue reading “Installing Chrome Extensions Without Signing in With a Google Account”

s3cmd ‘ERROR: Test failed: 403 (AccessDenied): Access Denied’ and ‘ERROR: Config: verbosity level ’20’ is not valid’ [SOLVED]

I’m working on a project that includes sending data via Amazon Simple Storage Service (S3) and was having some problems configuring and using the s3cmd client.

The first thing I discovered about s3cmd is not to trust what it tells you when invoking s3cmd –configure to get things set up to use the bucket.

$ s3cmd -v --configure s3://some-bucket/some-prefix/

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access 
Continue reading “s3cmd ‘ERROR: Test failed: 403 (AccessDenied): Access Denied’ and ‘ERROR: Config: verbosity level ’20’ is not valid’ [SOLVED]”

Do Not Use Symlinks in Jetty’s webapps Directory

I set up a development instance of Jetty on my local machine and have been happily coding, compiling and deploying via a shell script.  The script copies the war from my user’s target directory to the jetty users’s home dir changes the permissions and then moves it to the /webapp dir creating a symlink to the name of the .war that is referenced in a number of config files.

This was working just fine until I did a merge with → Continue reading “Do Not Use Symlinks in Jetty’s webapps Directory”