If you want to execute curl commands on your local machine and connect to an HTTPS server that is only reachable from a bastion or other host through which you can only get to via SSH, the following is how you set up the SSH tunnel and execute the curl command.
The following will not work
# Create ssh tunnel
#
ssh -L localhost:8443:example.com:443 user@bastion.example.com
# Attempt to hit the endpoint otherwise accessible from bastion.example.com
# with curl -X GET https://example.com/v1/endpoint
#
curl -X GET https://localhost:8443/v1/endpoint
The reason that this does not work is that with the port forwarded ssh tunnel curl is unable to resolve the IP of the example.com
HTTPS server on the other side of the connection on bastion.example.com
and the connection fails.
If we execute curl -v
we can see the details. Notice that curl is connecting to localhost:8443
and not resolving example.com
$ curl -v https://localhost:8443/subjects
* Trying ::1:8443...
* Connected to localhost (::1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.example.com
* start date: Aug 3 00:00:00 2022 GMT
* expire date: Sep 1 23:59:59 2023 GMT
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x560827a1b2c0)
> GET /subjects HTTP/2
> Host: localhost:8443
> user-agent: curl/7.74.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 404
< server: awselb/2.0
< date: Wed, 08 Feb 2023 15:20:04 GMT
< content-type: text/plain; charset=utf-8
< content-length: 0
Instead of just forwarding TCP packets we need to tell ssh client to setup a SOCKS5 proxy through which we will tunnel traffic.
ssh -D 8443 -f -C -q -N bastion.example.com
The -D
option creates a SOCKS5 proxy server listening on port 8443 which tunnels the traffic over ssh to bastion.example.com
from which hostnames for the destination webserver can be resolved. This creates a proxy server that enables you to connect to “dynamic” destinations on the other side of the tunnel.
The other options
-D 8443- start a SOCKS server listening on port 8443 on the localhost
-f - fork the process, running it in the background
-C - compress data
-q - quite mode
-N - indicate to the ssh client that there are no commands to be sent over the tunnel
Once you create the proxy and tunnel you can then execute curl commands as follows on the localhost telling curl to use the SOCKS5 proxy listening on localhost:8443
curl -v -x socks5h://0:8443 https://example.com/v1/endpoint
The verbose output shows that the SOCKS5 proxy is the connecting to example.com:443
over the tunnel and remotely resolving the IP to the correct HTTPS server on the other side of the tunnel.
$ curl -v -x socks5h://0:8443 https://example.com/v1/endpoint
* Trying 0.0.0.0:8443...
* SOCKS5 connect to example.com:443 (remotely resolved)
* SOCKS5 request granted.
* Connected to 0 (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.example.com
* start date: Aug 3 00:00:00 2022 GMT
* expire date: Sep 1 23:59:59 2023 GMT
* subjectAltName: host "example.com" matched cert's "*.example.com"
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562ada1d52c0)
> GET /subjects HTTP/2
> Host: example.com
> user-agent: curl/7.74.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< date: Wed, 08 Feb 2023 15:22:08 GMT
< content-type: application/vnd.schemaregistry.v1+json
< content-length: 1937
< vary: Accept-Encoding, User-Agent