Welcome to my website. I am always posting links to photo albums, art, technology and other creations. Everything that you will see on my numerous personal sites is powered by the formVistaTM Website Management Engine.


  • Mocking an HTTPS RESTful endpoint with Netcat
    06/12/2019 2:20PM

    Netcat is generally known as a TCP/IP Swiss Army Knife and is incredibly helpful for both debugging and mocking up network services

    Following is an example on how to setup a mock RESTful service that communicates over HTTPS.

    On the "server" side, run the following command.  The -l command instructs Netcat to listen.

    while true; do { echo -e "HTTP/1.1 200 OK\r\n$(date)\r\n\r\n<h1>hello world from $(hostname) on $(date)</h1>" |  nc -vl --ssl 8080; } done

    On the "client" side, run the following to PUT a sample json document.

    curl https://localhost:8080/foo/blah -k -XPUT -d @sample.json

    Alternatively, you can also generate a key cert pair to use if you have to test importing of certs

    To do so, first generate a self-signed cert and an ssl key without a passphrase for your nc "server".  Place the server.key and server.cert file in /var/tmp/server-cert

    openssl req -nodes -new -x509 -keyout server.key -out server.cert

    Then run nc as follows:

    while true; do { echo -e "HTTP/1.1 200 OK\r\n$(date)\r\n\r\n<h1>hello world from $(hostname) on $(date)</h1>" |  nc -vl --ssl 8080 --ssl-key /var/tmp/server-cert/server.key --ssl-cert /var/tmp/server-cert/server.cert; } done

  • Using netcat to Mock a RESTful Webservice that Returns JSON
    05/10/2016 10:26AM

    Let's say that you are working on a part of a project that needs to consume some JSON data from a forthcoming HTTP service.  That sub-system is being developed by one of your colleagues and is not yet ready for you to stand-up to develop against.

    You can use netcat to mock the webservice and return some static JSON data for which you can develop and test against with a simple one-liner.

    First, put together your JSON in a file, 'my.json' and then run the following command in a terminal:

    while true; do echo -e "HTTP/1.1 200 OK\r\n\r\n$(cat my.json)" | nc -l 9998; done

    An hitting http://localhost:9998 will return your test JSON data.

  • How to Use Credentials That Contain Special Characters with curl
    05/02/2016 12:00PM
    In order to execute curl commands to endpoints with passwords that contain special characters, the cleanest way that I have found to do so is to Base64 encode the authentication string for curl and then pass an Authorization request header along with the request.

    In this example the credentials are uid 'rchapin' and passwd 'abc123!@#'.  Normally we would pass this to curl as follows:

    $ curl -u rchapin:abc123!@# -X GET https://some-endpoint:443

    However, this will not work and the password will need to be sent in some other fashion other than ASCII.

    Following are the steps to pass the credentials as Base64:

    1. Using your favorite Base64 encoder, generate and ASCII string of the entire 'username:password' strin
    g that you would normally pass with the -u option for curl

    rchapin:abc123!@# converted = cmNoYXBpbjphYmMxMjMhQCM=

    2. Now modify the curl command as follows:

    $ curl -H "Authorization: Basic cmNoYXBpbjphYmMxMjMhQCM=" -X GET https://some-endpoint:443
  • Internet Search for Code: SymbolHound
    09/26/2014 11:38AM


    Allows you to enter search strings that include the typical set of characters included in various programming languages.

  • s3cmd 'ERROR: Test failed: 403 (AccessDenied): Access Denied' and 'ERROR: Config: verbosity level '20' is not valid' [SOLVED]
    03/19/2014 10:17AM

    I'm working on a project that includes sending data via Amazon Simple Storage Service (S3) and was having some problems configuring and using the s3cmd client.

    The first thing I discovered about s3cmd is not to trust what it tells you when invoking s3cmd --configure to get things set up to use the bucket.

    $ s3cmd -v --configure s3://some-bucket/some-prefix/

    Enter new values or accept defaults in brackets with Enter.
    Refer to user manual for detailed description of all options.

    Access key and Secret key are your identifiers for Amazon S3
    Access Key: thisisanaccesskey
    Secret Key: thisisasecretkey

    Encryption password is used to protect your files from reading
    by unauthorized persons while in transfer to S3
    Encryption password:
    Path to GPG program:

    When using secure HTTPS protocol all communication with Amazon S3
    servers is protected from 3rd party eavesdropping. This method is
    slower than plain HTTP and can't be used if you're behind a proxy
    Use HTTPS protocol [No]: Y

    New settings:
      Access Key:
      Secret Key:
      Encryption password:
      Path to GPG program: None
      Use HTTPS protocol: True
      HTTP Proxy server name:
      HTTP Proxy server port: 0

    Test access with supplied credentials? [Y/n] y
    Please wait, attempting to list bucket:
    ERROR: Test failed: 403 (AccessDenied): Access Denied

    Retry configuration? [Y/n] n

    Save settings? [y/N] y
    Configuration saved to '/home/rchapin/.s3cfg'

    As you can see, when I ran configure and opted to test the configs, I got a 403 error.  At that point, I assumed that I didn't have acces to the bucket and went back to the client to try and figure out if I had the right key, if they set up the bucket with the right permissions, blah, blah, blah.

    It turns out, that s3cmd simply gave me incorrect information, or the command that it was using to test it wasn't valid, or it was trying to do something with the bucket that I didn't have permission to do.

    After running the config above, I tried:

    $ s3cmd put test.txt s3://some-bucket/some-prefix/
    ERROR: Config: verbosity level '20' is not valid
    test.txt -> s3://some-bucket/some-prefix/test.txt  [1 of 1]
     15 of 15   100% in    0s    67.58 B/s  done

    Turns out that I have access after all.

    $ s3cmd ls s3://some-bucket/some-prefix/
    ERROR: Config: verbosity level '20' is not valid
    2014-03-01 00:25         0   s3://some-bucket/some-prefix/
    2014-03-19 14:06        15   s3://some-bucket/some-prefix/test.txt

    It also turns out that appending the '-v' arg when configuring s3cmd causes it to throw the 'ERROR: Config: verbosity level '20' is not valid' error.

    If you delete the .s3cfg file in your home dir, and re-run s3cmd --configure without the -v command it should work as expected.

    Just don't trust the s3cmd --configure test . . . test it yourself and you might find that you have access already.

  • 1 2 3 4 5 >>
Advanced Search