Web Scraping and Automation

Python offers powerful libraries like BeautifulSoup and requests that simplify web scraping. With just a few lines of code, you can extract data from websites and save it for analysis or other purposes.

# Example: Web scraping with BeautifulSoup
import requests
from bs4 import BeautifulSoup

url = “https://example.com”
response = requests.get(url)

if response.status_code == 200:
soup = BeautifulSoup(response.content, “html.parser”)
title = soup.title.text
print(“Website Title:”, title)
else:
print(“Failed to fetch data!”)

Web scraping’s potential goes beyond data extraction. You can automate repetitive tasks such as form filling, content posting, or price monitoring with the help of Python.

# Example: Automating form filling with Selenium
from selenium import webdriver

driver = webdriver.Chrome(executable_path=”path/to/chromedriver”)

url = “https://example.com/login”
driver.get(url)

# Fill the login form
username_field = driver.find_element_by_name(“username”)
password_field = driver.find_element_by_name(“password”)

username_field.send_keys(“your_username”)
password_field.send_keys(“your_password”)

# Click the login button
login_button = driver.find_element_by_name(“login_button”)
login_button.click()

# Perform other automated tasks
# …

driver.quit()