<p>Web scraping is one of those subjects that often appears in python discussions. There are many ways to do this, and there doesn't seem to be one best way. There are fully fledged frameworks like <a class="reference external" href="scrapy">http://scrapy.org">scrapy and more lightweight libraries like <a class="reference external" href="mechanize.">http://wwwsearch.sourceforge.net/mechanize/">mechanize. Do-it-yourself solutions are also popular: one can ...</p>