Showing posts with label Scrap. Show all posts
Showing posts with label Scrap. Show all posts

Wednesday, June 13, 2007

scrap google search results use hpricot

require 'rubygems'
require 'cgi'
require 'open-uri'
require 'hpricot'

q = %w{david cain pc}.map { |w| CGI.escape(w) }.join("+")
url = "http://www.google.com/search?q=#{q}"
doc = Hpricot(open(url).read)
urls = (doc/"div[@class='g'] a")
urls.each {|url| puts url['href']}