Transport Adapters

The toolbelt comes with several different transport adapters for you to use with requests. The transport adapters are all kept in requests_toolbelt.adapters and include

FingerprintAdapter

New in version 0.4.0.

By default, requests will validate a server’s certificate to ensure a connection is secure. In addition to this, the user can provide a fingerprint of the certificate they’re expecting to receive. Unfortunately, the requests API does not support this fairly rare use-case. When a user needs this extra validation, they should use the FingerprintAdapter class to perform the validation.

class requests_toolbelt.adapters.fingerprint.FingerprintAdapter(fingerprint, **kwargs)

A HTTPS Adapter for Python Requests that verifies certificate fingerprints, instead of certificate hostnames.

Example usage:

import requests
import ssl
from requests_toolbelt.adapters.fingerprint import FingerprintAdapter

twitter_fingerprint = '...'
s = requests.Session()
s.mount(
    'https://twitter.com',
    FingerprintAdapter(twitter_fingerprint)
)

The fingerprint should be provided as a hexadecimal string, optionally containing colons.

SSLAdapter

The SSLAdapter is the canonical implementation of the adapter proposed on Cory Benfield’s blog, here. This adapter allows the user to choose one of the SSL/TLS protocols made available in Python’s ssl module for outgoing HTTPS connections.

In principle, this shouldn’t be necessary: compliant SSL servers should be able to negotiate the required SSL version. In practice there have been bugs in some versions of OpenSSL that mean that this negotiation doesn’t go as planned. It can be useful to be able to simply plug in a Transport Adapter that can paste over the problem.

For example, suppose you’re having difficulty with the server that provides TLS for GitHub. You can work around it by using the following code:

from requests_toolbelt.adapters.ssl import SSLAdapter

import requests
import ssl

s = requests.Session()
s.mount('https://github.com/', SSLAdapter(ssl.PROTOCOL_TLSv1))

Any future requests to GitHub made through that adapter will automatically attempt to negotiate TLSv1, and hopefully will succeed.

class requests_toolbelt.adapters.ssl.SSLAdapter(ssl_version=None, **kwargs)

A HTTPS Adapter for Python Requests that allows the choice of the SSL/TLS version negotiated by Requests. This can be used either to enforce the choice of high-security TLS versions (where supported), or to work around misbehaving servers that fail to correctly negotiate the default TLS version being offered.

Example usage:

>>> import requests
>>> import ssl
>>> from requests_toolbelt import SSLAdapter
>>> s = requests.Session()
>>> s.mount('https://', SSLAdapter(ssl.PROTOCOL_TLSv1))

You can replace the chosen protocol with any that are available in the default Python SSL module. All subsequent requests that match the adapter prefix will use the chosen SSL version instead of the default.

This adapter will also attempt to change the SSL/TLS version negotiated by Requests when using a proxy. However, this may not always be possible: prior to Requests v2.4.0 the adapter did not have access to the proxy setup code. In earlier versions of Requests, this adapter will not function properly when used with proxies.

HostHeaderSSLAdapter

New in version 0.7.0.

Requests supports SSL Verification by default. However, it relies on the user making a request with the URL that has the hostname in it. If, however, the user needs to make a request with the IP address, they cannot actually verify a certificate against the hostname they want to request. This adapter sets the Server Name Indication to, and verifies the certificate against, the hostname in the Host header.

To accomodate this very rare need, we’ve added HostHeaderSSLAdapter. Example usage:

import requests
from requests_toolbelt.adapters import host_header_ssl
s = requests.Session()
s.mount('https://', host_header_ssl.HostHeaderSSLAdapter())
s.get("https://93.184.216.34", headers={"Host": "example.org"})
class requests_toolbelt.adapters.host_header_ssl.HostHeaderSSLAdapter(pool_connections=10, pool_maxsize=10, max_retries=0, pool_block=False)

A HTTPS Adapter for Python Requests that sets the hostname for certificate verification based on the Host header.

This allows requesting the IP address directly via HTTPS without getting a “hostname doesn’t match” exception.

Example usage:

>>> s.mount('https://', HostHeaderSSLAdapter())
>>> s.get("https://93.184.216.34", headers={"Host": "example.org"})

SourceAddressAdapter

New in version 0.3.0.

The SourceAddressAdapter allows a user to specify a source address for their connnection.

class requests_toolbelt.adapters.source.SourceAddressAdapter(source_address, **kwargs)

A Source Address Adapter for Python Requests that enables you to choose the local address to bind to. This allows you to send your HTTP requests from a specific interface and IP address.

Two address formats are accepted. The first is a string: this will set the local IP address to the address given in the string, and will also choose a semi-random high port for the local port number.

The second is a two-tuple of the form (ip address, port): for example, ('10.10.10.10', 8999). This will set the local IP address to the first element, and the local port to the second element. If 0 is used as the port number, a semi-random high port will be selected.

Warning

Setting an explicit local port can have negative interactions with connection-pooling in Requests: in particular, it risks the possibility of getting “Address in use” errors. The string-only argument is generally preferred to the tuple-form.

Example usage:

import requests
from requests_toolbelt.adapters.source import SourceAddressAdapter

s = requests.Session()
s.mount('http://', SourceAddressAdapter('10.10.10.10'))
s.mount('https://', SourceAddressAdapter(('10.10.10.10', 8999)))

SocketOptionsAdapter

New in version 0.4.0.

Note

This adapter will only work with requests 2.4.0 or newer. The ability to set arbitrary socket options does not exist prior to requests 2.4.0.

The SocketOptionsAdapter allows a user to pass specific options to be set on created sockets when constructing the Adapter without subclassing. The adapter takes advantage of urllib3’s support for setting arbitrary socket options for each urllib3.connection.HTTPConnection (and HTTPSConnection).

To pass socket options, you need to send a list of three-item tuples. For example, requests and urllib3 disable Nagle’s Algorithm by default. If you need to re-enable it, you would do the following:

import socket
import requests
from requests_toolbelt.adapters.socket_options import SocketOptionsAdapter

nagles = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 0)]
session = requests.Session()
for scheme in session.adapters.keys():
    session.mount(scheme, SocketOptionsAdapter(socket_options=nagles))

This would re-enable Nagle’s Algorithm for all http:// and https:// connections made with that session.

class requests_toolbelt.adapters.socket_options.SocketOptionsAdapter(**kwargs)

An adapter for requests that allows users to specify socket options.

Since version 2.4.0 of requests, it is possible to specify a custom list of socket options that need to be set before establishing the connection.

Example usage:

>>> import socket
>>> import requests
>>> from requests_toolbelt.adapters import socket_options
>>> s = requests.Session()
>>> opts = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 0)]
>>> adapter = socket_options.SocketOptionsAdapter(socket_options=opts)
>>> s.mount('http://', adapter)

You can also take advantage of the list of default options on this class to keep using the original options in addition to your custom options. In that case, opts might look like:

>>> opts = socket_options.SocketOptionsAdapter.default_options + opts

TCPKeepAliveAdapter

New in version 0.4.0.

Note

This adapter will only work with requests 2.4.0 or newer. The ability to set arbitrary socket options does not exist prior to requests 2.4.0.

The TCPKeepAliveAdapter allows a user to pass specific keep-alive related options as keyword parameters as well as arbitrary socket options.

Note

Different keep-alive related socket options may not be available for your platform. Check the socket module for the availability of the following constants:

  • socket.TCP_KEEPIDLE
  • socket.TCP_KEEPCNT
  • socket.TCP_KEEPINTVL

The adapter will silently ignore any option passed for a non-existent option.

An example usage of the adapter:

import requests
from requests_toolbelt.adapters.socket_options import TCPKeepAliveAdapter

session = requests.Session()
keep_alive = TCPKeepAliveAdapter(idle=120, count=20, interval=30)
session.mount('https://region-a.geo-1.compute.hpcloudsvc.com', keep_alive)
session.post('https://region-a.geo-1.compute.hpcloudsvc.com/v2/1234abcdef/servers',
             # ...
             )

In this case we know that creating a server on HP Public Cloud can cause requests to hang without using TCP Keep-Alive. So we mount the adapter specifically for that domain, instead of adding it to every https:// and http:// request.

class requests_toolbelt.adapters.socket_options.TCPKeepAliveAdapter(**kwargs)

An adapter for requests that turns on TCP Keep-Alive by default.

The adapter sets 4 socket options:

  • SOL_SOCKET SO_KEEPALIVE - This turns on TCP Keep-Alive
  • IPPROTO_TCP TCP_KEEPINTVL 20 - Sets the keep alive interval
  • IPPROTO_TCP TCP_KEEPCNT 5 - Sets the number of keep alive probes
  • IPPROTO_TCP TCP_KEEPIDLE 60 - Sets the keep alive time if the socket library has the TCP_KEEPIDLE constant

The latter three can be overridden by keyword arguments (respectively):

  • interval
  • count
  • idle

You can use this adapter like so:

>>> from requests_toolbelt.adapters import socket_options
>>> tcp = socket_options.TCPKeepAliveAdapter(idle=120, interval=10)
>>> s = requests.Session()
>>> s.mount('http://', tcp)

X509Adapter

Requests supports SSL Verification using a certificate in .pem format by default. In some cases it is necessary to pass a full cert chain as part of a request or it is deemed too great a risk to decrypt the certificate into a .pem file.

For such use cases we have created X509Adapter. Example usage:

import requests
from requests_toolbelt.adapters.x509 import X509Adapter
s = requests.Session()
a = X509Adapter(max_retries=3,
                cert_bytes=b'...', pk_bytes=b'...', encoding='...')
s.mount('https://', a)