Transport Adapters¶
The toolbelt comes with several different transport adapters for you to use
with requests. The transport adapters are all kept in
requests_toolbelt.adapters
and include
requests_toolbelt.adapters.fingerprint.FingerprintAdapter
requests_toolbelt.adapters.socket_options.SocketOptionsAdapter
requests_toolbelt.adapters.socket_options.TCPKeepAliveAdapter
requests_toolbelt.adapters.source.SourceAddressAdapter
requests_toolbelt.adapters.ssl.SSLAdapter
requests_toolbelt.adapters.host_header_ssl.HostHeaderSSLAdapter
requests_toolbelt.adapters.x509.X509Adapter
FingerprintAdapter¶
New in version 0.4.0.
By default, requests will validate a server’s certificate to ensure a
connection is secure. In addition to this, the user can provide a fingerprint
of the certificate they’re expecting to receive. Unfortunately, the requests
API does not support this fairly rare use-case. When a user needs this extra
validation, they should use the
FingerprintAdapter
class to
perform the validation.
-
class
requests_toolbelt.adapters.fingerprint.
FingerprintAdapter
(fingerprint, **kwargs)¶ A HTTPS Adapter for Python Requests that verifies certificate fingerprints, instead of certificate hostnames.
Example usage:
import requests import ssl from requests_toolbelt.adapters.fingerprint import FingerprintAdapter twitter_fingerprint = '...' s = requests.Session() s.mount( 'https://twitter.com', FingerprintAdapter(twitter_fingerprint) )
The fingerprint should be provided as a hexadecimal string, optionally containing colons.
SSLAdapter¶
The SSLAdapter
is the canonical implementation of the adapter proposed on
Cory Benfield’s blog, here. This adapter allows the user to choose one of
the SSL/TLS protocols made available in Python’s ssl
module for outgoing
HTTPS connections.
In principle, this shouldn’t be necessary: compliant SSL servers should be able to negotiate the required SSL version. In practice there have been bugs in some versions of OpenSSL that mean that this negotiation doesn’t go as planned. It can be useful to be able to simply plug in a Transport Adapter that can paste over the problem.
For example, suppose you’re having difficulty with the server that provides TLS for GitHub. You can work around it by using the following code:
from requests_toolbelt.adapters.ssl import SSLAdapter
import requests
import ssl
s = requests.Session()
s.mount('https://github.com/', SSLAdapter(ssl.PROTOCOL_TLSv1))
Any future requests to GitHub made through that adapter will automatically attempt to negotiate TLSv1, and hopefully will succeed.
-
class
requests_toolbelt.adapters.ssl.
SSLAdapter
(ssl_version=None, **kwargs)¶ A HTTPS Adapter for Python Requests that allows the choice of the SSL/TLS version negotiated by Requests. This can be used either to enforce the choice of high-security TLS versions (where supported), or to work around misbehaving servers that fail to correctly negotiate the default TLS version being offered.
Example usage:
>>> import requests >>> import ssl >>> from requests_toolbelt import SSLAdapter >>> s = requests.Session() >>> s.mount('https://', SSLAdapter(ssl.PROTOCOL_TLSv1))
You can replace the chosen protocol with any that are available in the default Python SSL module. All subsequent requests that match the adapter prefix will use the chosen SSL version instead of the default.
This adapter will also attempt to change the SSL/TLS version negotiated by Requests when using a proxy. However, this may not always be possible: prior to Requests v2.4.0 the adapter did not have access to the proxy setup code. In earlier versions of Requests, this adapter will not function properly when used with proxies.
HostHeaderSSLAdapter¶
New in version 0.7.0.
Requests supports SSL Verification by default. However, it relies on the user making a request with the URL that has the hostname in it. If, however, the user needs to make a request with the IP address, they cannot actually verify a certificate against the hostname they want to request. This adapter sets the Server Name Indication to, and verifies the certificate against, the hostname in the Host header.
To accomodate this very rare need, we’ve added
HostHeaderSSLAdapter
.
Example usage:
import requests
from requests_toolbelt.adapters import host_header_ssl
s = requests.Session()
s.mount('https://', host_header_ssl.HostHeaderSSLAdapter())
s.get("https://93.184.216.34", headers={"Host": "example.org"})
-
class
requests_toolbelt.adapters.host_header_ssl.
HostHeaderSSLAdapter
(pool_connections=10, pool_maxsize=10, max_retries=0, pool_block=False)¶ A HTTPS Adapter for Python Requests that sets the hostname for certificate verification based on the Host header.
This allows requesting the IP address directly via HTTPS without getting a “hostname doesn’t match” exception.
Example usage:
>>> s.mount('https://', HostHeaderSSLAdapter()) >>> s.get("https://93.184.216.34", headers={"Host": "example.org"})
SourceAddressAdapter¶
New in version 0.3.0.
The SourceAddressAdapter
allows a
user to specify a source address for their connnection.
-
class
requests_toolbelt.adapters.source.
SourceAddressAdapter
(source_address, **kwargs)¶ A Source Address Adapter for Python Requests that enables you to choose the local address to bind to. This allows you to send your HTTP requests from a specific interface and IP address.
Two address formats are accepted. The first is a string: this will set the local IP address to the address given in the string, and will also choose a semi-random high port for the local port number.
The second is a two-tuple of the form (ip address, port): for example,
('10.10.10.10', 8999)
. This will set the local IP address to the first element, and the local port to the second element. If0
is used as the port number, a semi-random high port will be selected.Warning
Setting an explicit local port can have negative interactions with connection-pooling in Requests: in particular, it risks the possibility of getting “Address in use” errors. The string-only argument is generally preferred to the tuple-form.
Example usage:
import requests from requests_toolbelt.adapters.source import SourceAddressAdapter s = requests.Session() s.mount('http://', SourceAddressAdapter('10.10.10.10')) s.mount('https://', SourceAddressAdapter(('10.10.10.10', 8999)))
SocketOptionsAdapter¶
New in version 0.4.0.
Note
This adapter will only work with requests 2.4.0 or newer. The ability to set arbitrary socket options does not exist prior to requests 2.4.0.
The SocketOptionsAdapter
allows a user to pass specific options to be set
on created sockets when constructing the Adapter without subclassing. The
adapter takes advantage of urllib3
’s support for setting arbitrary
socket options for each urllib3.connection.HTTPConnection
(and
HTTPSConnection
).
To pass socket options, you need to send a list of three-item tuples. For
example, requests
and urllib3
disable Nagle’s Algorithm by default.
If you need to re-enable it, you would do the following:
import socket
import requests
from requests_toolbelt.adapters.socket_options import SocketOptionsAdapter
nagles = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 0)]
session = requests.Session()
for scheme in session.adapters.keys():
session.mount(scheme, SocketOptionsAdapter(socket_options=nagles))
This would re-enable Nagle’s Algorithm for all http://
and https://
connections made with that session.
-
class
requests_toolbelt.adapters.socket_options.
SocketOptionsAdapter
(**kwargs)¶ An adapter for requests that allows users to specify socket options.
Since version 2.4.0 of requests, it is possible to specify a custom list of socket options that need to be set before establishing the connection.
Example usage:
>>> import socket >>> import requests >>> from requests_toolbelt.adapters import socket_options >>> s = requests.Session() >>> opts = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 0)] >>> adapter = socket_options.SocketOptionsAdapter(socket_options=opts) >>> s.mount('http://', adapter)
You can also take advantage of the list of default options on this class to keep using the original options in addition to your custom options. In that case,
opts
might look like:>>> opts = socket_options.SocketOptionsAdapter.default_options + opts
TCPKeepAliveAdapter¶
New in version 0.4.0.
Note
This adapter will only work with requests 2.4.0 or newer. The ability to set arbitrary socket options does not exist prior to requests 2.4.0.
The TCPKeepAliveAdapter
allows a user to pass specific keep-alive related
options as keyword parameters as well as arbitrary socket options.
Note
Different keep-alive related socket options may not be available for your platform. Check the socket module for the availability of the following constants:
socket.TCP_KEEPIDLE
socket.TCP_KEEPCNT
socket.TCP_KEEPINTVL
The adapter will silently ignore any option passed for a non-existent option.
An example usage of the adapter:
import requests
from requests_toolbelt.adapters.socket_options import TCPKeepAliveAdapter
session = requests.Session()
keep_alive = TCPKeepAliveAdapter(idle=120, count=20, interval=30)
session.mount('https://region-a.geo-1.compute.hpcloudsvc.com', keep_alive)
session.post('https://region-a.geo-1.compute.hpcloudsvc.com/v2/1234abcdef/servers',
# ...
)
In this case we know that creating a server on HP Public Cloud can cause
requests to hang without using TCP Keep-Alive. So we mount the adapter
specifically for that domain, instead of adding it to every https://
and
http://
request.
-
class
requests_toolbelt.adapters.socket_options.
TCPKeepAliveAdapter
(**kwargs)¶ An adapter for requests that turns on TCP Keep-Alive by default.
The adapter sets 4 socket options:
SOL_SOCKET
SO_KEEPALIVE
- This turns on TCP Keep-AliveIPPROTO_TCP
TCP_KEEPINTVL
20 - Sets the keep alive intervalIPPROTO_TCP
TCP_KEEPCNT
5 - Sets the number of keep alive probesIPPROTO_TCP
TCP_KEEPIDLE
60 - Sets the keep alive time if the socket library has theTCP_KEEPIDLE
constant
The latter three can be overridden by keyword arguments (respectively):
interval
count
idle
You can use this adapter like so:
>>> from requests_toolbelt.adapters import socket_options >>> tcp = socket_options.TCPKeepAliveAdapter(idle=120, interval=10) >>> s = requests.Session() >>> s.mount('http://', tcp)
X509Adapter¶
Requests supports SSL Verification using a certificate in .pem format by default. In some cases it is necessary to pass a full cert chain as part of a request or it is deemed too great a risk to decrypt the certificate into a .pem file.
For such use cases we have created
X509Adapter
.
Example usage:
import requests
from requests_toolbelt.adapters.x509 import X509Adapter
s = requests.Session()
a = X509Adapter(max_retries=3,
cert_bytes=b'...', pk_bytes=b'...', encoding='...')
s.mount('https://', a)