This makes scraping twitter for multiple search terms a piece of cake.

Just start a new Python scraper, and type this at the top:

from scraperwiki import swimport swimport('twitter_search').search(['some_phrase', 'from:some_user'])

Set your scraper to run daily and it’ll automatically fill its datastore with the latest tweets.

Forked from ScraperWiki

Contributors asdasdasdasdasd123

Last run failed with status code 1.

Console output of last run

Oh dear, failed to scrape http://search.twitter.com/search.json?q=trending&rpp=100&lang=en&page=1 Traceback (most recent call last): File "/repo/scraper.py", line 29, in <module> search(queries = ["trending","highschool"]) File "/repo/scraper.py", line 15, in search results_json = simplejson.loads(scraperwiki.scrape(base_url)) File "/usr/local/lib/python2.7/dist-packages/scraperwiki-0.3.7-py2.7.egg/scraperwiki/utils.py", line 31, in scrape f = urllib2.urlopen(req) File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 406, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 519, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 444, in error return self._call_chain(*args) File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2 . HTTPError : HTTP Error 401: Unauthorized

Statistics

Average successful run time: less than 5 seconds

Total run time: less than 10 seconds

Total cpu time used: less than 5 seconds

Total disk space used: 23.8 KB

History

  • Manually ran revision 6b49e151 and failed .
    nothing changed in the database
  • Manually ran revision bd36088e and failed .
    nothing changed in the database
  • Manually ran revision 60fb9919 and failed .
    nothing changed in the database
  • Manually ran revision e6105f57 and failed .
    nothing changed in the database
  • Manually ran revision 5dffce98 and completed successfully .
    nothing changed in the database
  • Forked from ScraperWiki

Scraper code

Python

twitter_search / scraper.py