Python Http Header

Python – HTTP Headers – Tutorialspoint

The request and response between client and server involves header and body in the message.
Headers contain protocol specific information that appear at the beginning of the raw message that is sent over TCP connection. The body of the message is separated from headers using a blank line.
Example of Headers
The headers in the response can be categorized into following types. Below is a description of the header and an example.
The Cache-Control general-header field is used to specify directives that MUST be obeyed by all the caching system. The syntax is as follows:
Cache-Control: cache-request-directive|cache-response-directive
An HTTP client or server can use the Cache-control general header to specify parameters for the cache or to request certain kinds of documents from the cache. The caching directives are specified in a comma-separated list. For example:
Cache-control: no-cache
The Connection general-header field allows the sender to specify options that are desired for that particular connection and must not be communicated by proxies over further connections. Following is the simple syntax for using connection header:
Connection: “Connection”
HTTP/1. 1 defines the “close” connection option for the sender to signal that the connection will be closed after completion of the response. For example:
Connection: close
By default, HTTP 1. 1 uses persistent connections, where the connection does not automatically close after a transaction. HTTP 1. 0, on the other hand, does not have persistent connections by default. If a 1. 0 client wishes to use persistent connections, it uses the keep-alive parameter as follows:
Connection: keep-alive
All HTTP date/time stamps MUST be represented in Greenwich Mean Time (GMT), without exception. HTTP applications are allowed to use any of the following three representations of date/time stamps:
Sun, 06 Nov 1994 08:49:37 GMT; RFC 822, updated by RFC 1123
Sunday, 06-Nov-94 08:49:37 GMT; RFC 850, obsoleted by RFC 1036
Sun Nov 6 08:49:37 1994; ANSI C’s asctime() format
The Transfer-Encoding general-header field indicates what type of transformation has been applied to the message body in order to safely transfer it between the sender and the recipient. This is not the same as content-encoding because transfer-encodings are a property of the message, not of the entity-body. The syntax of Transfer-Encoding header field is as follows:
Transfer-Encoding: chunked
All transfer-coding values are case-insensitive.
The Upgrade general-header allows the client to specify what additional communication protocols it supports and would like to use if the server finds it appropriate to switch protocols. For example:
Upgrade: HTTP/2. 0, SHTTP/1. 3, IRC/6. 9, RTA/x11
The Upgrade header field is intended to provide a simple mechanism for transition from HTTP/1. 1 to some other, incompatible protocol.
The Via general-header must be used by gateways and proxies to indicate the intermediate protocols and recipients. For example, a request message could be sent from an HTTP/1. 0 user agent to an internal proxy code-named “fred”, which uses HTTP/1. 1 to forward the request to a public proxy at, which completes the request by forwarding it to the origin server at. The request received by would then have the following Via header field:
Via: 1. 0 fred, 1. 1 (Apache/1. 1)
The Warning general-header is used to carry additional information about the status or transformation of a message which might not be reflected in the message. A response may carry more than one Warning header.
Warning: warn-code SP warn-agent SP warn-text SP warn-date
In the below example we use the urllib2 module to get a response using urlopen. Next we apply the info() method to get the header information for that response.
import urllib2
response = urllib2. urlopen(”)
html = ()
print html
When we run the above program, we get the following output −
Access-Control-Allow-Headers: X-Requested-With
Access-Control-Allow-Origin: *
Cache-Control: max-age=2592000
Content-Type: text/html; charset=UTF-8
Date: Mon, 02 Jul 2018 11:06:07 GMT
Expires: Wed, 01 Aug 2018 11:06:07 GMT
Last-Modified: Sun, 01 Jul 2018 21:05:38 GMT
Server: ECS (tir/CDD1)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 22063
Connection: close
http.client — HTTP protocol client — Python 3.10.0 ...

http.client — HTTP protocol client — Python 3.10.0 …

Source code: Lib//
This module defines classes which implement the client side of the HTTP and
HTTPS protocols. It is normally not used directly — the module
quest uses it to handle URLs that use HTTP and HTTPS.
See also
The Requests package
is recommended for a higher-level HTTP client interface.
HTTPS support is only available if Python was compiled with SSL support
(through the ssl module).
The module provides the following classes:
class (host, port=None, [timeout, ]source_address=None, blocksize=8192)¶
An HTTPConnection instance represents one transaction with an HTTP
server. It should be instantiated passing it a host and optional port
number. If no port number is passed, the port is extracted from the host
string if it has the form host:port, else the default HTTP port (80) is
used. If the optional timeout parameter is given, blocking
operations (like connection attempts) will timeout after that many seconds
(if it is not given, the global default timeout setting is used).
The optional source_address parameter may be a tuple of a (host, port)
to use as the source address the HTTP connection is made from.
The optional blocksize parameter sets the buffer size in bytes for
sending a file-like message body.
For example, the following calls all create instances that connect to the server
at the same host and port:
>>> h1 = (”)
>>> h2 = (”)
>>> h3 = (”, 80)
>>> h4 = (”, 80, timeout=10)
Changed in version 3. 2: source_address was added.
Changed in version 3. 4: The strict parameter was removed. HTTP 0. 9-style “Simple Responses” are
not longer supported.
Changed in version 3. 7: blocksize parameter was added.
class (host, port=None, key_file=None, cert_file=None, [timeout, ]source_address=None, *, context=None, check_hostname=None, blocksize=8192)¶
A subclass of HTTPConnection that uses SSL for communication with
secure servers. Default port is 443. If context is specified, it
must be a LContext instance describing the various SSL
Please read Security considerations for more information on best practices.
Changed in version 3. 2: source_address, context and check_hostname were added.
Changed in version 3. 2: This class now supports HTTPS virtual hosts if possible (that is,
if ssl. HAS_SNI is true).
no longer supported.
Changed in version 3. 4. 3: This class now performs all the necessary certificate and hostname checks
by default. To revert to the previous, unverified, behavior
ssl. _create_unverified_context() can be passed to the context
Changed in version 3. 8: This class now enables TLS 1. 3
_handshake_auth for the default context or
when cert_file is passed with a custom context.
class (sock, debuglevel=0, method=None, url=None)¶
Class whose instances are returned upon successful connection. Not
instantiated directly by user.
Changed in version 3. 9 style “Simple Responses” are
This module provides the following function:
Parse the headers from a file pointer fp representing a HTTP
request/response. The file has to be a BufferedIOBase reader
(i. e. not text) and must provide a valid RFC 2822 style header.
This function returns an instance of
that holds the header fields, but no payload
(the same as
After returning, the file pointer fp is ready to read the HTTP body.
parse_headers() does not parse the start-line of a HTTP message;
it only parses the Name: value lines. The file has to be ready to
read these field lines, so the first line should already be consumed
before calling the function.
The following exceptions are raised as appropriate:
exception ¶
The base class of the other exceptions in this module. It is a subclass of
A subclass of HTTPException.
A subclass of HTTPException, raised if a port is given and is either
non-numeric or empty.
A subclass of ImproperConnectionState.
A subclass of HTTPException. Raised if a server responds with a HTTP
status code that we don’t understand.
A subclass of HTTPException. Raised if an excessively long line
is received in the HTTP protocol from the server.
A subclass of ConnectionResetError and BadStatusLine. Raised
by tresponse() when the attempt to read the response
results in no data read from the connection, indicating that the remote end
has closed the connection.
New in version 3. 5: Previously, BadStatusLine(”) was raised.
The constants defined in this module are:

The default port for the HTTP protocol (always 80).
The default port for the HTTPS protocol (always 443).
This dictionary maps the HTTP 1. 1 status codes to the W3C names.
Example: [] is ‘Not Found’.
See HTTP status codes for a list of HTTP status codes that are
available in this module as constants.
HTTPConnection Objects¶
HTTPConnection instances have the following methods:
quest(method, url, body=None, headers={}, *, encode_chunked=False)¶
This will send a request to the server using the HTTP request
method method and the selector url.
If body is specified, the specified data is sent after the headers are
finished. It may be a str, a bytes-like object, an
open file object, or an iterable of bytes. If body
is a string, it is encoded as ISO-8859-1, the default for HTTP. If it
is a bytes-like object, the bytes are sent as is. If it is a file
object, the contents of the file is sent; this file object should
support at least the read() method. If the file object is an
instance of io. TextIOBase, the data returned by the read()
method will be encoded as ISO-8859-1, otherwise the data returned by
read() is sent as is. If body is an iterable, the elements of the
iterable are sent as is until the iterable is exhausted.
The headers argument should be a mapping of extra HTTP headers to send
with the request.
If headers contains neither Content-Length nor Transfer-Encoding,
but there is a request body, one of those
header fields will be added automatically. If
body is None, the Content-Length header is set to 0 for
methods that expect a body (PUT, POST, and PATCH). If
body is a string or a bytes-like object that is not also a
file, the Content-Length header is
set to its length. Any other type of body (files
and iterables in general) will be chunk-encoded, and the
Transfer-Encoding header will automatically be set instead of
The encode_chunked argument is only relevant if Transfer-Encoding is
specified in headers. If encode_chunked is False, the
HTTPConnection object assumes that all encoding is handled by the
calling code. If it is True, the body will be chunk-encoded.
Chunked transfer encoding has been added to the HTTP protocol
version 1. 1. Unless the HTTP server is known to handle HTTP 1. 1,
the caller must either specify the Content-Length, or must pass a
str or bytes-like object that is not also a file as the
body representation.
New in version 3. 2: body can now be an iterable.
Changed in version 3. 6: If neither Content-Length nor Transfer-Encoding are set in
headers, file and iterable body objects are now chunk-encoded.
The encode_chunked argument was added.
No attempt is made to determine the Content-Length for file
Should be called after a request is sent to get the response from the server.
Returns an HTTPResponse instance.
Note that you must have read the whole response before you can send a new
request to the server.
Changed in version 3. 5: If a ConnectionError or subclass is raised, the
HTTPConnection object will be ready to reconnect when
a new request is sent.
Set the debugging level. The default debug level is 0, meaning no
debugging output is printed. Any value greater than 0 will cause all
currently defined debug output to be printed to stdout. The debuglevel
is passed to any new HTTPResponse objects that are created.
New in version 3. 1.
t_tunnel(host, port=None, headers=None)¶
Set the host and the port for HTTP Connect Tunnelling. This allows running
the connection through a proxy server.
The host and port arguments specify the endpoint of the tunneled connection
(i. the address included in the CONNECT request, not the address of the
proxy server).
The headers argument should be a mapping of extra HTTP headers to send with
the CONNECT request.
For example, to tunnel through a HTTPS proxy server running locally on port
8080, we would pass the address of the proxy to the HTTPSConnection
constructor, and the address of the host that we eventually want to reach to
the set_tunnel() method:
>>> import
>>> conn = (“localhost”, 8080)
>>> t_tunnel(“)
>>> quest(“HEAD”, “/”)
New in version 3. 2.
Connect to the server specified when the object was created. By default,
this is called automatically when making a request if the client does not
already have a connection.
Close the connection to the server.
HTTPConnection. blocksize¶
Buffer size in bytes for sending a file-like message body.
New in version 3. 7.
As an alternative to using the request() method described above, you can
also send your request step by step, by using the four functions below.
HTTPConnection. putrequest(method, url, skip_host=False, skip_accept_encoding=False)¶
This should be the first call after the connection to the server has been
made. It sends a line to the server consisting of the method string,
the url string, and the HTTP version (HTTP/1. 1). To disable automatic
sending of Host: or Accept-Encoding: headers (for example to accept
additional content encodings), specify skip_host or skip_accept_encoding
with non-False values.
Send an RFC 822-style header to the server. It sends a line to the server
consisting of the header, a colon and a space, and the first argument. If more
arguments are given, continuation lines are sent, each consisting of a tab and
an argument.
Send a blank line to the server, signalling the end of the headers. The
optional message_body argument can be used to pass a message body
associated with the request.
If encode_chunked is True, the result of each iteration of
message_body will be chunk-encoded as specified in RFC 7230,
Section 3. 3. How the data is encoded is dependent on the type of
message_body. If message_body implements the buffer interface the encoding will result in a single chunk.
If message_body is a, each iteration
of message_body will result in a chunk. If message_body is a
file object, each call to () will result in a chunk.
The method automatically signals the end of the chunk-encoded data
immediately after message_body.
Due to the chunked encoding specification, empty chunks
yielded by an iterator body will be ignored by the chunk-encoder.
This is to avoid premature termination of the read of the request by
the target server due to malformed encoding.
New in version 3. 6: Chunked encoding support. The encode_chunked parameter was
Send data to the server. This should be used directly only after the
endheaders() method has been called and before getresponse() is
HTTPResponse Objects¶
An HTTPResponse instance wraps the HTTP response from the
server. It provides access to the request headers and the entity
body. The response is an iterable object and can be used in a with
Changed in version 3. 5: The io. BufferedIOBase interface is now implemented and
all of its reader operations are supported.
Reads and returns the response body, or up to the next amt bytes.
Reads up to the next len(b) bytes of the response body into the buffer b.
Returns the number of bytes read.
New in version 3. 3.
Return the value of the header name, or default if there is no header
matching name. If there is more than one header with the name name,
return all of the values joined by ‘, ‘. If ‘default’ is any iterable other
than a single string, its elements are similarly returned joined by commas.
Return a list of (header, value) tuples.
Return the fileno of the underlying socket.
A instance containing the response
headers. is a subclass of
HTTP protocol version used by server. 10 for HTTP/1. 0, 11 for HTTP/1. 1.
URL of the resource retrieved, commonly used to determine if a redirect was followed.
Headers of the response in the form of an Message instance.
Status code returned by server.
Reason phrase returned by server.
A debugging hook. If debuglevel is greater than zero, messages
will be printed to stdout as the response is read and parsed.
Is True if the stream is closed.
Deprecated since version 3. 9: Deprecated in favor of url.
Deprecated since version 3. 9: Deprecated in favor of headers.
Deprecated since version 3. 9: Deprecated in favor of status.
Here is an example session that uses the GET method:
>>> conn = (“)
>>> quest(“GET”, “/”)
>>> r1 = tresponse()
>>> print(, )
200 OK
>>> data1 = () # This will return entire content.
>>> # The following example demonstrates reading data in chunks.
>>> while chunk:= (200):… print(repr(chunk))
b’\n>> # Example of an invalid request
>>> conn = (“”)
>>> r2 = tresponse()
404 Not Found
>>> data2 = ()
>>> ()
Here is an example session that uses the HEAD method. Note that the
HEAD method never returns any data.
>>> res = tresponse()
>>> data = ()
>>> print(len(data))
>>> data == b”
Here is an example session that shows how to POST requests:
>>> import,
>>> params = ({‘@number’: 12524, ‘@type’: ‘issue’, ‘@action’: ‘show’})
>>> headers = {“Content-type”: “application/x-www-form-urlencoded”,… “Accept”: “text/plain”}
>>> quest(“POST”, “”, params, headers)
>>> response = tresponse()
302 Found
>>> data
b’Redirecting to

Python’s Requests Library (Guide)

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Making HTTP Requests With Python
The requests library is the de facto standard for making HTTP requests in Python. It abstracts the complexities of making requests behind a beautiful, simple API so that you can focus on interacting with services and consuming data in your application.
Throughout this article, you’ll see some of the most useful features that requests has to offer as well as how to customize and optimize those features for different situations you may come across. You’ll also learn how to use requests in an efficient way as well as how to prevent requests to external services from slowing down your application.
In this tutorial, you’ll learn how to:
Make requests using the most common HTTP methods
Customize your requests’ headers and data, using the query string and message body
Inspect data from your requests and responses
Make authenticated requests
Configure your requests to help prevent your application from backing up or slowing down
Though I’ve tried to include as much information as you need to understand the features and examples included in this article, I do assume a very basic general knowledge of HTTP. That said, you still may be able to follow along fine anyway.
Now that that is out of the way, let’s dive in and see how you can use requests in your application!
Getting Started With requests
Let’s begin by installing the requests library. To do so, run the following command:
If you prefer to use Pipenv for managing Python packages, you can run the following:
$ pipenv install requests
Once requests is installed, you can use it in your application. Importing requests looks like this:
Now that you’re all set up, it’s time to begin your journey through requests. Your first goal will be learning how to make a GET request.
The GET Request
HTTP methods such as GET and POST, determine which action you’re trying to perform when making an HTTP request. Besides GET and POST, there are several other common methods that you’ll use later in this tutorial.
One of the most common HTTP methods is GET. The GET method indicates that you’re trying to get or retrieve data from a specified resource. To make a GET request, invoke ().
To test this out, you can make a GET request to GitHub’s Root REST API by calling get() with the following URL:
>>>>>> (”)

Congratulations! You’ve made your first request. Let’s dive a little deeper into the response of that request.
The Response
A Response is a powerful object for inspecting the results of the request. Let’s make that same request again, but this time store the return value in a variable so that you can get a closer look at its attributes and behaviors:
>>>>>> response = (”)
In this example, you’ve captured the return value of get(), which is an instance of Response, and stored it in a variable called response. You can now use response to see a lot of information about the results of your GET request.
Status Codes
The first bit of information that you can gather from Response is the status code. A status code informs you of the status of the request.
For example, a 200 OK status means that your request was successful, whereas a 404 NOT FOUND status means that the resource you were looking for was not found. There are many other possible status codes as well to give you specific insights into what happened with your request.
By accessing. status_code, you can see the status code that the server returned:
>>>>>> atus_code
200. status_code returned a 200, which means your request was successful and the server responded with the data you were requesting.
Sometimes, you might want to use this information to make decisions in your code:
if atus_code == 200:
print(‘Success! ‘)
elif atus_code == 404:
print(‘Not Found. ‘)
With this logic, if the server returns a 200 status code, your program will print Success!. If the result is a 404, your program will print Not Found.
requests goes one step further in simplifying this process for you. If you use a Response instance in a conditional expression, it will evaluate to True if the status code was between 200 and 400, and False otherwise.
Therefore, you can simplify the last example by rewriting the if statement:
if response:
print(‘An error has occurred. ‘)
Keep in mind that this method is not verifying that the status code is equal to 200. The reason for this is that other status codes within the 200 to 400 range, such as 204 NO CONTENT and 304 NOT MODIFIED, are also considered successful in the sense that they provide some workable response.
For example, the 204 tells you that the response was successful, but there’s no content to return in the message body.
So, make sure you use this convenient shorthand only if you want to know if the request was generally successful and then, if necessary, handle the response appropriately based on the status code.
Let’s say you don’t want to check the response’s status code in an if statement. Instead, you want to raise an exception if the request was unsuccessful. You can do this using. raise_for_status():
import requests
from requests. exceptions import HTTPError
for url in [”, ”]:
response = (url)
# If the response was successful, no Exception will be raised
response. raise_for_status()
except HTTPError as _err:
print(f’HTTP error occurred: {_err}’) # Python 3. 6
except Exception as err:
print(f’Other error occurred: {err}’) # Python 3. 6
If you invoke. raise_for_status(), an HTTPError will be raised for certain status codes. If the status code indicates a successful request, the program will proceed without that exception being raised.
Now, you know a lot about how to deal with the status code of the response you got back from the server. However, when you make a GET request, you rarely only care about the status code of the response. Usually, you want to see more. Next, you’ll see how to view the actual data that the server sent back in the body of the response.
The response of a GET request often has some valuable information, known as a payload, in the message body. Using the attributes and methods of Response, you can view the payload in a variety of different formats.
To see the response’s content in bytes, you use. content:
>>> ntent
b'{“current_user_url”:”, “current_user_authorizations_html_url”:”/client_id}”, “authorizations_url”:”, “code_search_url”:”query}{&page, per_page, sort, order}”, “commit_search_url”:”query}{&page, per_page, sort, order}”, “emails_url”:”, “emojis_url”:”, “events_url”:”, “feeds_url”:”, “followers_url”:”, “following_url”:”/target}”, “gists_url”:”/gist_id}”, “hub_url”:”, “issue_search_url”:”query}{&page, per_page, sort, order}”, “issues_url”:”, “keys_url”:”, “notifications_url”:”, “organization_repositories_url”:”org}/repos{? type, page, per_page, sort}”, “organization_url”:”org}”, “public_gists_url”:”, “rate_limit_url”:”, “repository_url”:”owner}/{repo}”, “repository_search_url”:”query}{&page, per_page, sort, order}”, “current_user_repositories_url”:”? type, page, per_page, sort}”, “starred_url”:”/owner}{/repo}”, “starred_gists_url”:”, “team_url”:”, “user_url”:”user}”, “user_organizations_url”:”, “user_repositories_url”:”user}/repos{? type, page, per_page, sort}”, “user_search_url”:”query}{&page, per_page, sort, order}”}’
While. content gives you access to the raw bytes of the response payload, you will often want to convert them into a string using a character encoding such as UTF-8. response will do that for you when you access
‘{“current_user_url”:”, “current_user_authorizations_html_url”:”/client_id}”, “authorizations_url”:”, “code_search_url”:”query}{&page, per_page, sort, order}”, “commit_search_url”:”query}{&page, per_page, sort, order}”, “emails_url”:”, “emojis_url”:”, “events_url”:”, “feeds_url”:”, “followers_url”:”, “following_url”:”/target}”, “gists_url”:”/gist_id}”, “hub_url”:”, “issue_search_url”:”query}{&page, per_page, sort, order}”, “issues_url”:”, “keys_url”:”, “notifications_url”:”, “organization_repositories_url”:”org}/repos{? type, page, per_page, sort}”, “organization_url”:”org}”, “public_gists_url”:”, “rate_limit_url”:”, “repository_url”:”owner}/{repo}”, “repository_search_url”:”query}{&page, per_page, sort, order}”, “current_user_repositories_url”:”? type, page, per_page, sort}”, “starred_url”:”/owner}{/repo}”, “starred_gists_url”:”, “team_url”:”, “user_url”:”user}”, “user_organizations_url”:”, “user_repositories_url”:”user}/repos{? type, page, per_page, sort}”, “user_search_url”:”query}{&page, per_page, sort, order}”}’
Because the decoding of bytes to a str requires an encoding scheme, requests will try to guess the encoding based on the response’s headers if you do not specify one. You can provide an explicit encoding by setting. encoding before accessing
>>>>>> response. encoding = ‘utf-8’ # Optional: requests infers this internally
If you take a look at the response, you’ll see that it is actually serialized JSON content. To get a dictionary, you could take the str you retrieved from and deserialize it using (). However, a simpler way to accomplish this task is to use ():
>>>>>> ()
{‘current_user_url’: ”, ‘current_user_authorizations_html_url’: ‘/client_id}’, ‘authorizations_url’: ”, ‘code_search_url’: ‘query}{&page, per_page, sort, order}’, ‘commit_search_url’: ‘query}{&page, per_page, sort, order}’, ’emails_url’: ”, ’emojis_url’: ”, ‘events_url’: ”, ‘feeds_url’: ”, ‘followers_url’: ”, ‘following_url’: ‘/target}’, ‘gists_url’: ‘/gist_id}’, ‘hub_url’: ”, ‘issue_search_url’: ‘query}{&page, per_page, sort, order}’, ‘issues_url’: ”, ‘keys_url’: ”, ‘notifications_url’: ”, ‘organization_repositories_url’: ‘org}/repos{? type, page, per_page, sort}’, ‘organization_url’: ‘org}’, ‘public_gists_url’: ”, ‘rate_limit_url’: ”, ‘repository_url’: ‘owner}/{repo}’, ‘repository_search_url’: ‘query}{&page, per_page, sort, order}’, ‘current_user_repositories_url’: ‘? type, page, per_page, sort}’, ‘starred_url’: ‘/owner}{/repo}’, ‘starred_gists_url’: ”, ‘team_url’: ”, ‘user_url’: ‘user}’, ‘user_organizations_url’: ”, ‘user_repositories_url’: ‘user}/repos{? type, page, per_page, sort}’, ‘user_search_url’: ‘query}{&page, per_page, sort, order}’}
The type of the return value of () is a dictionary, so you can access values in the object by key.
You can do a lot with status codes and message bodies. But, if you need more information, like metadata about the response itself, you’ll need to look at the response’s headers.
Query String Parameters
One common way to customize a GET request is to pass values through query string parameters in the URL. To do this using get(), you pass data to params. For example, you can use GitHub’s Search API to look for the requests library:
# Search GitHub’s repositories for requests
response = (
params={‘q’: ‘requests+language:python’}, )
# Inspect some attributes of the `requests` repository
json_response = ()
repository = json_response[‘items’][0]
print(f’Repository name: {repository[“name”]}’) # Python 3. 6+
print(f’Repository description: {repository[“description”]}’) # Python 3. 6+
By passing the dictionary {‘q’: ‘requests+language:python’} to the params parameter of (), you are able to modify the results that come back from the Search API.
You can pass params to get() in the form of a dictionary, as you have just done, or as a list of tuples:
>>>>>> (… ”,… params=[(‘q’, ‘requests+language:python’)],… )
You can even pass the values as bytes:
>>>>>> (… params=b’q=requests+language:python’,… )
Query strings are useful for parameterizing GET requests. You can also customize your requests by adding or modifying the headers you send.
Other HTTP Methods
Aside from GET, other popular HTTP methods include POST, PUT, DELETE, HEAD, PATCH, and OPTIONS. requests provides a method, with a similar signature to get(), for each of these HTTP methods:
>>>>>> (”, data={‘key’:’value’})
>>> (”, data={‘key’:’value’})
>>> (”)
>>> requests. options(”)
Each function call makes a request to the bin service using the corresponding HTTP method. For each method, you can inspect their responses in the same way you did before:
>>> response. headers[‘Content-Type’]
>>> response = (”)
>>> json_response = ()
>>> json_response[‘args’]
Headers, response bodies, status codes, and more are returned in the Response for each method. Next you’ll take a closer look at the POST, PUT, and PATCH methods and learn how they differ from the other request types.
The Message Body
According to the HTTP specification, POST, PUT, and the less common PATCH requests pass their data through the message body rather than through parameters in the query string. Using requests, you’ll pass the payload to the corresponding function’s data parameter.
data takes a dictionary, a list of tuples, bytes, or a file-like object. You’ll want to adapt the data you send in the body of your request to the specific needs of the service you’re interacting with.
For example, if your request’s content type is application/x-www-form-urlencoded, you can send the form data as a dictionary:
You can also send that same data as a list of tuples:
>>>>>> (”, data=[(‘key’, ‘value’)])
If, however, you need to send JSON data, you can use the json parameter. When you pass JSON data via json, requests will serialize your data and add the correct Content-Type header for you.
is a great resource created by the author of requests, Kenneth Reitz. It’s a service that accepts test requests and responds with data about the requests. For instance, you can use it to inspect a basic POST request:
>>>>>> response = (”, json={‘key’:’value’})
>>> json_response[‘data’]
‘{“key”: “value”}’
>>> json_response[‘headers’][‘Content-Type’]
You can see from the response that the server received your request data and headers as you sent them. requests also provides this information to you in the form of a PreparedRequest.
Inspecting Your Request
When you make a request, the requests library prepares the request before actually sending it to the destination server. Request preparation includes things like validating headers and serializing JSON content.
You can view the PreparedRequest by accessing. request:
>>> quest. headers[‘Content-Type’]

b'{“key”: “value”}’
Inspecting the PreparedRequest gives you access to all kinds of information about the request being made such as payload, URL, headers, authentication, and more.
So far, you’ve made a lot of different kinds of requests, but they’ve all had one thing in common: they’re unauthenticated requests to public APIs. Many services you may come across will want you to authenticate in some way.
Authentication helps a service understand who you are. Typically, you provide your credentials to a server by passing data through the Authorization header or a custom header defined by the service. All the request functions you’ve seen to this point provide a parameter called auth, which allows you to pass your credentials.
One example of an API that requires authentication is GitHub’s Authenticated User API. This endpoint provides information about the authenticated user’s profile. To make a request to the Authenticated User API, you can pass your GitHub username and password in a tuple to get():
>>>>>> from getpass import getpass
>>> (”, auth=(‘username’, getpass()))
The request succeeded if the credentials you passed in the tuple to auth are valid. If you try to make this request with no credentials, you’ll see that the status code is 401 Unauthorized:

When you pass your username and password in a tuple to the auth parameter, requests is applying the credentials using HTTP’s Basic access authentication scheme under the hood.
Therefore, you could make the same request by passing explicit Basic authentication credentials using HTTPBasicAuth:
>>>>>> from import HTTPBasicAuth
>>> from getpass import getpass
>>> (… auth=HTTPBasicAuth(‘username’, getpass())… )
Though you don’t need to be explicit for Basic authentication, you may want to authenticate using another method. requests provides other methods of authentication out of the box such as HTTPDigestAuth and HTTPProxyAuth.
You can even supply your own authentication mechanism. To do so, you must first create a subclass of AuthBase. Then, you implement __call__():
from import AuthBase
class TokenAuth(AuthBase):
“””Implements a custom authentication scheme. “””
def __init__(self, token):
= token
def __call__(self, r):
“””Attach an API token to a custom auth header. “””
r. headers[‘X-TokenAuth’] = f'{}’ # Python 3. 6+
return r
(”, auth=TokenAuth(‘12345abcde-token’))
Here, your custom TokenAuth mechanism receives a token, then includes that token in the X-TokenAuth header of your request.
Bad authentication mechanisms can lead to security vulnerabilities, so unless a service requires a custom authentication mechanism for some reason, you’ll always want to use a tried-and-true auth scheme like Basic or OAuth.
While you’re thinking about security, let’s consider dealing with SSL Certificates using requests.
SSL Certificate Verification
Any time the data you are trying to send or receive is sensitive, security is important. The way that you communicate with secure sites over HTTP is by establishing an encrypted connection using SSL, which means that verifying the target server’s SSL Certificate is critical.
The good news is that requests does this for you by default. However, there are some cases where you might want to change this behavior.
If you want to disable SSL Certificate verification, you pass False to the verify parameter of the request function:
>>>>>> (”, verify=False)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: InsecureRequestWarning)
requests even warns you when you’re making an insecure request to help you keep your data safe!
When using requests, especially in a production application environment, it’s important to consider performance implications. Features like timeout control, sessions, and retry limits can help you keep your application running smoothly.
When you make an inline request to an external service, your system will need to wait upon the response before moving on. If your application waits too long for that response, requests to your service could back up, your user experience could suffer, or your background jobs could hang.
By default, requests will wait indefinitely on the response, so you should almost always specify a timeout duration to prevent these things from happening. To set the request’s timeout, use the timeout parameter. timeout can be an integer or float representing the number of seconds to wait on a response before timing out:
>>>>>> (”, timeout=1)
>>> (”, timeout=3. 05)
In the first request, the request will timeout after 1 second. In the second request, the request will timeout after 3. 05 seconds.
You can also pass a tuple to timeout with the first element being a connect timeout (the time it allows for the client to establish a connection to the server), and the second being a read timeout (the time it will wait on a response once your client has established a connection):
>>>>>> (”, timeout=(2, 5))
If the request establishes a connection within 2 seconds and receives data within 5 seconds of the connection being established, then the response will be returned as it was before. If the request times out, then the function will raise a Timeout exception:
from requests. exceptions import Timeout
response = (”, timeout=1)
except Timeout:
print(‘The request timed out’)
print(‘The request did not time out’)
Your program can catch the Timeout exception and respond accordingly.
The Session Object
Until now, you’ve been dealing with high level requests APIs such as get() and post(). These functions are abstractions of what’s going on when you make your requests. They hide implementation details such as how connections are managed so that you don’t have to worry about them.
Underneath those abstractions is a class called Session. If you need to fine-tune your control over how requests are being made or improve the performance of your requests, you may need to use a Session instance directly.
Sessions are used to persist parameters across requests. For example, if you want to use the same authentication across multiple requests, you could use a session:
from getpass import getpass
# By using a context manager, you can ensure the resources used by
# the session will be released after use
with ssion() as session:
= (‘username’, getpass())
# Instead of (), you’ll use ()
response = (”)
# You can inspect the response just like you did before
print(response. headers)
Each time you make a request with session, once it has been initialized with authentication credentials, the credentials will be persisted.
The primary performance optimization of sessions comes in the form of persistent connections. When your app makes a connection to a server using a Session, it keeps that connection around in a connection pool. When your app wants to connect to the same server again, it will reuse a connection from the pool rather than establishing a new one.
Max Retries
When a request fails, you may want your application to retry the same request. However, requests will not do this for you by default. To apply this functionality, you need to implement a custom Transport Adapter.
Transport Adapters let you define a set of configurations per service you’re interacting with. For example, let’s say you want all requests to to retry three times before finally raising a ConnectionError. You would build a Transport Adapter, set its max_retries parameter, and mount it to an existing Session:
from apters import HTTPAdapter
from requests. exceptions import ConnectionError
github_adapter = HTTPAdapter(max_retries=3)
session = ssion()
# Use `github_adapter` for all requests to endpoints that start with this URL
(”, github_adapter)
except ConnectionError as ce:
When you mount the HTTPAdapter, github_adapter, to session, session will adhere to its configuration for each request to Timeouts, Transport Adapters, and sessions are for keeping your code efficient and your application resilient.
You’ve come a long way in learning about Python’s powerful requests library.
You’re now able to:
Make requests using a variety of different HTTP methods such as GET, POST, and PUT
Customize your requests by modifying headers, authentication, query strings, and message bodies
Inspect the data you send to the server and the data the server sends back to you
Work with SSL Certificate verification
Use requests effectively using max_retries, timeout, Sessions, and Transport Adapters
Because you learned how to use requests, you’re equipped to explore the wide world of web services and build awesome applications using the fascinating data they provide.
Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Making HTTP Requests With Python

Frequently Asked Questions about python http header

How do I make a HTTP header in Python?

Use requests. get() to add headers using requests Create a dictionary using the syntax {key: value} where key is the header name and value is the header content. Call requests. get(url, headers=headers_dict) with headers_dict as the dictionary from the previous step to send the headers to url .

What are HTTP headers python?

The request and response between client and server involves header and body in the message. Headers contain protocol specific information that appear at the beginning of the raw message that is sent over TCP connection. The body of the message is separated from headers using a blank line.

What is an HTTP header?

HTTP headers let the client and the server pass additional information with an HTTP request or response. An HTTP header consists of its case-insensitive name followed by a colon ( : ), then by its value. … Response headers hold additional information about the response, like its location or about the server providing it.Oct 3, 2021

Leave a Reply

Your email address will not be published. Required fields are marked *