mcmaster_scraper
A Python library for fetching product tables from a McMaster-Carr URL as a DataFrame for complex filtering and calculations.
Features
- Caches data locally to speed up future calls
- Supports both sync/async APIs
- Works in Python files and Jupyter notebooks
- Includes convenience functions to quickly retrieve product tables from multiple URLs
- Typed functions for type-checking compatibility
Install
McMaster-Scraper is available on PyPi:
pip install mcmaster-scraper
McMaster-Scraper requires Playwright to fetch the product tables. It is already included as a dependency. However, you will need to install the browsers manually:
playwright install
Quick Start
To use the Sync API, import the sync_api module and call get_products_from_url(s):
from mcmaster_scraper.sync_api import get_products_from_url
url = "https://www.mcmaster.com/products/screws/socket-head-screws-2~/steel-socket-head-screws~~/"
data = get_products_from_url(url) # Returns a DataFrame with all the products from the URL
... # Do stuff with the DataFrame (filter, perform calculations, etc.)
Using the Async API is similar, import the async_api module and await the function call:
from mcmaster_scraper.async_api import get_products_from_url
url = "https://www.mcmaster.com/products/screws/socket-head-screws-2~/steel-socket-head-screws~~/"
data = await get_products_from_url(url) # Returns a DataFrame with all the products from the URL
... # Do stuff with the DataFrame (filter, perform calculations, etc.)