• code snippets
  • all code snippets/
  • popular code snippets/
  • your code snippets
• extras
• blog
• about snipplr

• Big /
• Small

scrapy

Member since 09/01/2012

Website: http://scrapy.org/

Scrapy is a fast high-level screen scraping and web crawling framework for Python, used to crawl websites and extract structured data from their pages.

1 snippets

View their favorites

Their Tags

PROfile

All time reputation: 337

Achievement

scrapy's Recent Snippets Tagged 'random'

• All / 
• JavaScript / 
• HTML / 
• PHP / 
• CSS / 
• Ruby / 
• Objective C

1. Use a random user agent for each request

   Python random user agent useragent middleware scrapy
   posted on September 1, 2012 by scrapy