Web scraping tool

Soldato
Joined
27 Dec 2005
Posts
17,296
Location
Bristol
I need to scrape a couple of webpages for data that's publicly accessible (tho it's behind a login, if that matters).

Both are just a list of profiles. The main page has a list of everyone's names, and their subsequent/individual page has further details they've opted to share. So I need a scraper that will follow each link and export the data to a spreadsheet. I've tried using a couple but they don't have the "follow link" functionality and so they only scrape the names.

Any recommendations? I can write PHP and can run something locally if that helps.
 
Back
Top Bottom