Selenium + pandas web scraping tutorial for Python (2021) but also I'm late to pick up a cat
A tutorial for scraping web pages using Selenium, using Python and saving as a CSV with pandas. Covers selecting by tag names and classes, and compares find_element vs find_elements. After we cover a little bit of the basics, we show how to take our scraped data, create a pandas dataframe, and save it as a CSV file.
I made this as quick class review for my students who wanted extra Selenium scraping content.
CAT BONUS: I run a cat rescue and had to go pick up a cat from the vet. She just had spay surgery and did really well!
2021 BONUS: We use the new method of find_element(By.CLASS_NAME, "...") instead of the older deprecated find_element_by_class_name. Not very exciting but the error "find_element_by_* commands are deprecated. Please use find_element() instead" can be scary.
======
LINKS
Notebook with links here: https://gist.github.com/jsoma/803b2b2...
Webdriver manager: If you haven't used Webdriver Manager it's a real dream. "pip install webdriver-manager" or visit https://pypi.org/project/webdriver-ma... for details
=====
General-purpose import code I paste in is:
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from webdriver_manager.chrome import ChromeDriverManager
=====
CHAPTERS
00:00 Starting up Selenium
00:28 Scraping by class name
02:40 Scraping by tag name
04:15 Scraping with duplicate tags
07:07 Scraping a table cells
08:23 Scraping into a dictionary
09:50 Scraping multiple table rows
14:05 Scraping into list of dicts
17:37 Scraping a page into pandas
19:55 Save results into a CSV
Watch video Selenium + pandas web scraping tutorial for Python (2021) but also I'm late to pick up a cat online without registration, duration hours minute second in high quality. This video was added by user Jonathan Soma 01 December 2021, don't forget to share it with your friends and acquaintances, it has been viewed on our site 3,620 once and liked it 75 people.