Selenium + pandas web scraping tutorial for Python (2021) but also I'm late to pick up a cat
A tutorial for scraping web pages using Selenium, using Python and saving as a CSV with pandas. Covers selecting by tag names and classes, and compares find_element vs find_elements. After we cover a little bit of the basics, we show how to take our scraped data, create a pandas dataframe, and save it as a CSV file.
I made this as quick class review for my students who wanted extra Selenium scraping content.
CAT BONUS: I run a cat rescue and had to go pick up a cat from the vet. She just had spay surgery and did really well!
2021 BONUS: We use the new method of find_element(By.CLASS_NAME, "...") instead of the older deprecated find_element_by_class_name. Not very exciting but the error "find_element_by_* commands are deprecated. Please use find_element() instead" can be scary.
======
LINKS
Notebook with links here: https://gist.github.com/jsoma/803b2b2...
Webdriver manager: If you haven't used Webdriver Manager it's a real dream. "pip install webdriver-manager" or visit https://pypi.org/project/webdriver-ma... for details
=====
General-purpose import code I paste in is:
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from webdriver_manager.chrome import ChromeDriverManager
=====
CHAPTERS
00:00 Starting up Selenium
00:28 Scraping by class name
02:40 Scraping by tag name
04:15 Scraping with duplicate tags
07:07 Scraping a table cells
08:23 Scraping into a dictionary
09:50 Scraping multiple table rows
14:05 Scraping into list of dicts
17:37 Scraping a page into pandas
19:55 Save results into a CSV
Смотрите видео Selenium + pandas web scraping tutorial for Python (2021) but also I'm late to pick up a cat онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Jonathan Soma 01 Декабрь 2021, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 3,620 раз и оно понравилось 75 людям.