Web Scraping

Libraries

• Scrapy (Python) http://www.scrapy.org/

On line services

• Yahoo Query Language http://developer.yahoo.com/yql/, http://yuilibrary.com/yui/docs/yql/
  • exemple query état des stations Vélo Toulouse : http://query.yahooapis.com/v1/public/yql?q=select%20%2A%20from%20xml%20where%20url%20%3D%20%22http%3A%2F%2Fwww.velo.toulouse.fr%2Fservice%2Fstationdetails%2Ftoulouse%2F1%22&format=json&diagnostics=true&callback=
• Bobik http://usebobik.com/api/docs
• http://morph.io
• https://scraperwiki.com (freemium)

Sites:

• The ScrapingHub, turn web content into useful data

Articles:

• Web Scraping to Create Open Data

• WebShell http://webshell.io/, available apis: http://webshell.io/explorer