How to Scrape Span Tags in Python

In this tutorial, we will learn how to scrape Span tags in Python using a powerful and easy-to-use library called BeautifulSoup. Span tags are commonly used in HTML to group inline elements, and oftentimes, the information enclosed within these tags are of high importance for data analysis.

Whether you’re a data scientist looking to gather data to train your models, a student wanting to programmatically retrieve information from a website for a project, or simply interested in web scraping, this tutorial is meant for you!

Step 1: Install Required Libraries

Begin by ensuring that you have the necessary libraries installed. For this tutorial, we’ll need requests, to send HTTP requests to a URL, and BeautifulSoup, for parsing HTML content. You can install them via pip:

1
pip install requests beautifulsoup4

Step 2: Making a GET Request

First, we need to download the page contents. We can do this using the requests library. For example, assuming we want to scrape content from https://www.example.com:

1
2
3
4
import requests
 
url = "https://www.example.com"
response = requests.get(url)

We can then print out the status code of the response to ensure that the request was successful:

1
print(response.status_code)

Step 3: Parsing HTML with BeautifulSoup

Now that we have the HTML content, we can use BeautifulSoup to parse it and extract the information held within the span tags:

1
2
3
4
from bs4 import BeautifulSoup
 
soup = BeautifulSoup(response.text, 'html.parser')
spans = soup.find_all('span')

The ‘find_all()’ method returns all span elements on the page. You can then iterate over these to access the information you need.

Step 4: Extracting Information from Span Tags

Let’s go ahead and print the text enclosed within each of our identified span tags:

1
2
for span in spans:
    print(span.text)

Here’s the complete code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import requests
from bs4 import BeautifulSoup
 
url = "https://www.example.com"
response = requests.get(url)
 
if response.status_code == 200:
    print("Successful GET request!")
 
    soup = BeautifulSoup(response.text, 'html.parser')
    spans = soup.find_all('span')
 
    for span in spans:
        print(span.text)
else:
    print("Failed GET request!")

Output:

Successful GET request!

Conclusion

You’ve successfully learned how to scrape Span tags in Python! With this knowledge, you’re now able to extract valuable information from websites for your various data retrieval needs. Of course, always be respectful and make sure to abide by the website’s scraping policies, and don’t scrape data at a disruptive rate. Happy scraping!