How to Download an Image Using Python – Towards Data …
Learn how to download image files using Python modules like request, urllib and GiphyRecently, I was working with a remote system and needed to download some images that my code will eventually process. I could have used curl or wget on my terminal for downloading files. But, I wanted the entire process to be automated for the led me to the question:How can I download an Image using Python? In this tutorial, I will cover several modules that can be used for downloading files in Python(specifically images). The modules covered are: requests, wget, and urllib. Disclaimer: Do not download or use any image that violates its copyright quests is a neat and user-friendly HTTP library in Python. It makes sending HTTP/1. 1 requests extremely seems to be the most stable and recommended method for downloading any type of file using GiphyHere is the entire AuthorDon’t Worry. Let’s break it down will start by importing the necessary modules and will also set the Image requests # to get image from the webimport shutil # to save it locallyimage_url = “We use slice notation to separate the filename from the image link. We split the Image URL using forward-slash( /) and then use [-1] to slice the last lename = (“/”)[-1]The get() method from the requests module will be used to retrieve the image. r = (image_url, stream = True)Use stream = True to guarantee no, we will create the file locally in binary-write mode and use the copyfileobj() method to write our image to the file. # Set decode_content value to True, otherwise the downloaded image file’s size will be = True# Open a local file with wb ( write binary) open(filename, ‘wb’) as f: pyfileobj(, f)We can also add certain conditionals to check if the image was retrieved successfully using Request’s Status can also improve further by adding progress bars while downloading large files or a large number of files. Here is a good quests is the most stable and recommended method for downloading any type of file using from the python requests module, we can also use the python wget module for is the python equivalent of GNU ’s quite straightforward to AuthorThe standard Python library for accessing websites via your program is urllib. It is also used by the requests rough urllib, we can do a variety of things: access websites, download data, parse data, send GET and, POST can download our image using just a few lines of code:We used the urlretrieve method to copy the required web resource to a local is important to note that on some systems and a lot of websites, the above code will result in an error: HTTPError: HTTP Error 403: is because a lot of websites don’t appreciate random programs accessing their data. Some programs can attack the server by sending a large number of requests. This prevents the server from is why these websites can either:Block you and you will receive HTTP Error you different or NULL can overcome this by modifying user-agent, a variable sent with our request. This variable, by default, tells the website that the visitor is a python modifying this variable, we can act as if the website is being accessed on a standard web browser by a normal can read more about it here.
How To Download Multiple Images In Python | Sempioneer
Learning Outcomes
Python Imports
Method One: How To Download Multiple Images From A Python List
Method Two: How To Download Multiple Images From Many HTML Web Pages
How To Speed Up Your Image Downloads
ThreadPoolExecutor()
Async Programming!
How To Download 1 File Asychronously
How To Download Multiple Python Files Inside Of A Python File ()
Create A Python File
To learn how to download multiple images in Python using synchronous and asynchronous code.
Automatically downloading images from a number of your HTML pages is an essential skill, in this guide you’ll be learning 4 methods on how to download images using Python!
Let’s begin with the easiest example, if we already have a list of image URLs then we can follow this process:
Change into a directory where we would like to store all of the images.
Make a request to download all of the images, one by one.
We will also include error handling so that if a URL no longer exists the code will still work.
Python Imports! pip install tldextract
Requirement already satisfied: tldextract in /opt/anaconda3/lib/python3. 7/site-packages (2. 2. 2)
Requirement already satisfied: requests>=2. 1. 0 in /opt/anaconda3/lib/python3. 7/site-packages (from tldextract) (2. 22. 0)
Requirement already satisfied: setuptools in /opt/anaconda3/lib/python3. 7/site-packages (from tldextract) (20200309)
Requirement already satisfied: requests-file>=1. 4 in /opt/anaconda3/lib/python3. 7/site-packages (from tldextract) (1. 5. 1)
Requirement already satisfied: idna in /opt/anaconda3/lib/python3. 8)
Requirement already satisfied: urllib3! =1. 25. 0,! =1. 1, <1. 26, >=1. 21. 1 in /opt/anaconda3/lib/python3. 7/site-packages (from requests>=2. 0->tldextract) (1. 8)
Requirement already satisfied: chardet<3. 0, >=3. 0. 2 in /opt/anaconda3/lib/python3. 0->tldextract) (3. 4)
Requirement already satisfied: certifi>=2017. 4. 17 in /opt/anaconda3/lib/python3. 0->tldextract) (2019. 11. 28)
Requirement already satisfied: six in /opt/anaconda3/lib/python3. 7/site-packages (from requests-file>=1. 4->tldextract) (1. 14. 0)
import requests
import os
import subprocess
import quest
from bs4 import BeautifulSoup
import tldextract! mkdir all_images! ls
Changing into the directory of the folder called all_images, this can be done by either:
cd all_images
(‘path’)
(‘all_images’)! pwd
/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images
In order to download the multiple images, we’ll use the requests library. We’ll also create a python list to store any broken image URLs that didn’t return a 200 status code:
broken_images = []
image_urls = [”,
”]
for img in image_urls:
# We can split the file based upon / and extract the last split within the python list below:
file_name = (‘/’)[-1]
print(f”This is the file name: {file_name}”)
# Now let’s send a request to the image URL:
r = (img, stream=True)
# We can check that the status code is 200 before doing anything else:
if atus_code == 200:
# This command below will allow us to write the data to a file as binary:
with open(file_name, ‘wb’) as f:
for chunk in r:
(chunk)
else:
# We will write all of the images back to the broken_images list:
(img)
This is the file name:
☝️ See how simple that is! ☝️
If you check your folder, you will have now downloaded all of the images that contained a status code of 200!
If we don’t yet have the exact image URLs, we will need to do the following:
Download the HTML content of every web page.
Extract all of the image URLs for every page.
Create the file names.
Check to see if the image status code is 200.
Write all of images to your local computer.
This website has some relative image URLs. Therefore we will need to ensure that our code can handle for the following two types of image source URLs:
Exact Filepath: Relative Filepath: /html-and-css/links-and-images/
web_pages = [”,
”,
We will also extract the domain of every URL whilst we loop over the webpages like so:
for page in webpages:
domain_name = tldextract. extract(page). registered_domain
url_dictionary = {}
for page in web_pages:
# 1. Extracting the domain name of the web page:
print(f”The domain name: {domain_name}”)
# 2. Request the web page:
r = (page)
# 3. Check to see if the web page returned a status_200:
# 4. Create a URL dictionary entry for future use:
url_dictionary[page] = []
# 5. Parse the HTML content with BeautifulSoup and look for image tags:
soup = BeautifulSoup(ntent, ”)
# 6. Find all of the images per web page:
images = ndAll(‘img’)
# 7. Store all of the images
url_dictionary[page](images)
print(‘failed! ‘)
The domain name:
Now let’s double check and filter our dictionary so that we only look at web pages where there was at least 1 image tag:
for key, value in ():
if len(value) > 0:
print(f”This domain: {key} has more than 1 image on the web page. “)
This domain: has more than 1 image on the web page.
An easier way to write the above code would be via a dictionary comprehension:
cleaned_dictionary = {key: value for key, value in () if len(value) > 0}
We can now clean all of the image URLs inside of every dictionary key and change all of the relative URL paths to exact URL paths.
Let’s start by printing out all of the different image sources to see how we might need to clean up the data below:
for key, images in ():
for image in images:
print([‘src’])
For the scope of this tutorial, I have decided to:
Remove the logo links with the //
Add on the domain to the relative URLs
all_images = []
# 1. Creating a clean_urls and domain name for every page:
clean_urls = []
domain_name = tldextract. extract(key). registered_domain
# 2. Looping over every image per url:
# 3. Extracting the source (src) with
source_image_url = [‘src’]
# 4. Clean The Data
if artswith(“//”):
pass
elif domain_name not in source_image_url and ” not in source_image_url:
url = ” + domain_name + source_image_url
(url)
(source_image_url)
print(all_images[0:5])
[”, ”, ”, ”, ”]
After cleaning the image URLs, we can now refer to method one for downloading the images to our computer!
This time let’s convert it into a function:
def extract_images(image_urls_list:list, directory_path):
# Changing directory into a specific folder:
(directory_path)
# Downloading all of the images
for img in image_urls_list:
# Let’s try both of these versions in a loop [ and
url_paths_to_try = [img, place(”, ”)]
for url_image_path in url_paths_to_try:
print(url_image_path)
try:
except Exception as e:
pass! pwd
path = ‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images’
extract_images(image_urls_list=all_images,
directory_path=path)
Fantastic!
Now there are some things that we didn’t necessarily cover for which include:
only image urls.
But for the most part, you’ll be able to download images in bulk!
When working with 100’s or 1000’s of URLs its important to avoid using a synchronous approach to downloading images. An asynchronous approach means that we can download multiple web pages or multiple images in parallel.
This means that the overall execution time will be much quicker!
The ThreadPoolExecutor is one of python’s built in I/O packages for creating an asynchronous behaviour via multiple threads. In order to utilise it, we will make sure that the function will only work on a single URL.
Then we will pass the image URL list into multiple workers
def extract_single_image(img):
return “Completed”
return “Failed”
all_images[0:5]
[”,
The below code will create a new directory and then make it the current active working directory:
(‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc’)
except FileExistsError as e:
print(‘The file path already exists! ‘)
import concurrent. futures
# We can use a with statement to ensure threads are cleaned up promptly
with readPoolExecutor(max_workers=5) as executor:
# Start the load operations and mark each future with its URL
future_to_url = {(extract_single_image, image_url) for image_url in all_images}
for future in _completed(future_to_url):
url = future_to_url[future]
data = ()
except Exception as exc:
print(‘%r generated an exception:%s’% (url, exc))
You should’ve downloaded the images but at a much faster rate!
Just like JavaScript, Python 3. 6+ comes bundled with native support for co-routines called asyncio. Similar to NodeJS, there is a method available to you for creating custom event loops for async code.
We will also need to download an async code HTTP requests library called aio! pip install aio
We will also download aiofiles that allows us to write multiple image files asynchronously:! pip install aiofiles
import aio
import aiofiles
import asyncio
(‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc_event_loop’)
(‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc_event_loop’)! pwd
/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc_event_loop
print(all_images[0:1])
[”]
single_image = ”
async with ientSession() as session:
async with (single_image) as resp:
# 1. Capturing the image file name like we did before:
single_image_name = (‘/’)[-1]
# 2. Only proceed further if the HTTP response is 200 (Ok)
if == 200:
async with (single_image_name, mode=’wb’) as f:
await (await ())
await ()
We will need to structure our code slightly different for the async version to work across multiple files:
We will have a fetch function to query every image URL.
We will have a main function that creates, then executes a series of co-routines.
async def fetch(session, url):
async with (url) as resp:
url_name = (‘/’)[-1]
async with (url_name, mode=’wb’) as f:
async def main(image_urls:list):
tasks = []
headers = {
“user-agent”: “Mozilla/5. 0 (compatible; Googlebot/2. 1; +)”}
async with ientSession(headers=headers) as session:
for image in image_urls:
(await fetch(session, url))
data = await (*tasks)
main(all_images)
☝️☝️☝️ Notice how when we call this function, it doesn’t actually run and produces a co-routine! ☝️☝️☝️
We can then use asyncio as method for executing all of the fetch callables that need to be completed:
If you receive this type of error when running the following command:
(main(all_images))
It is likely because you’re trying to run asyncio within an event loop which is not natively possible. (Jupyter notebook runs in an event loop! ).
Let’s save the variable containing our URLs to a file:
with open(”, ‘w’) as f:
for item in all_images:
(f”{item}n”)
Then you will need to create a python file and add the following code to it:
# Package / Module Imports
# 1. Choose A Path – You will need to change this to your desired directory:
path = ‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc_event_loop’
# 2. Changing directory into that specific path:
# 3. Reading the URLs from the text file:
with open(”, ‘r’) as f:
image_urls = ()(‘n’)
# 2. Creating the async functions:
(await fetch(session, image))
# 3. Executing all of the asyncio tasks:
(main(image_urls))
print(e)
Then run the python script in either your terminal / command line with:
python3
Let’s break down what’s happening in the above code snippet:
We are importing all of the relevant packages for async programming with files.
Then we create a new directory.
After creating the new folder we change that folder to be the active working directory.
We then read the variable data which was previously saved from the file called
Then we create a series of co-routines and execute them within a main() function with asyncio.
As these co-routines are executed every file is asynchronously saved to your computer.
Finally let’s clear up and delete all of the folders to clean up our environment:
folders_to_delete = [
‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc_event_loop’,
‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images’,
‘/Users/jamesaphoenix/Desktop/Imran_And_James/Python_For_SEO/6_downloading_multiple_images/all_images_asnyc’]
import shutil
for folder in folders_to_delete:
print(f”Deleting this folder directory: {folder}”)
print(‘——‘)
(folder)
Being able to download images with python allows you to extend your automation capabilities and what other programs, APIs etc you might use that image data with!
Hopefully you now feel confident about downloading images within Python ❤
How to open an image from the URL in PIL? – GeeksforGeeks
In this article, we will learn How to open an image from the URL using the PIL module in python. For the opening of the image from a URL in Python, we need two Packages urllib and Pillow(PIL). Approach:Install the required libraries and then import them. To install use the following commands:pip install pillowCopy the URL of any URL with file name in quest. urlretrieve() () method to open last show the image using () Code: Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics. To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Courseimport questfrom PIL import quest. urlretrieve(‘Image url’, “file_name”)img = (“file_name”)()Example:Output:
Frequently Asked Questions about python download images
How do I download an image from Python?
How to download an image using requests in Pythonresponse = requests. get(“https://i.imgur.com/ExdKOOz.png”)file = open(“sample_image.png”, “wb”)file. write(response. content)file.
How do I download and save an image from Python?
Use urllib. request. urlretrieve() to save an image from a URL Call urllib. request. urlretrieve(url, filename) with url as the URL the image will be downloaded from and filename as the name of the file the image will be saved to on the local filesystem.
How do you download multiple images in python?
Method Two: How To Download Multiple Images From Many HTML Web PagesDownload the HTML content of every web page.Extract all of the image URLs for every page.Create the file names.Check to see if the image status code is 200.Write all of images to your local computer.