oxygala / basic_twitter_scraper_2

Basic Twitter Scraper

Scrapes search.twitter.com

Search Twitter to find the latest news and world events faster. Find popular people, hashtags and photos for any topic you can imagine.


This is a basic twitter scraper where you can change the search terms and store it in the datastore. Feel free to fork.

Forked from ScraperWiki

Contributors oxygala

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Oh dear, failed to scrape http://search.twitter.com/search.json?q=%23opendata&rpp=100&lang=en&page=1

Data

Downloaded 3 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (55.8 MB) Use the API

rows 10 / 216538

text id from_user created_at
first @fixmystreet fr @mysociety now @FixMyTransport http://t.co/9re4iom we need real @open311 in #seattle #opengov #opendata #transit
108990595585949696
MrDataFerret
@Gov Walker of WI can't be pleased w poor audit of state contract sunshine site http://t.co/pByo2g1 #opengov #opendata #wisconsin
108986525211037696
MrDataFerret
" #OpenData Geeks to the Rescue" http://t.co/osK0iNu > nice work by @dbhume & @data_bc team. good step 2ward gr8r #opengov #dbchack
108984924614307841
asterix
RT @chilobbyists: Official statement from Rahm's office on the release of more lobbyist data and our collaboration: http://t.co/ENpUMjY #opengov #opendata
108982291694493696
billgatewood
RT @WorldBank: Where should we take #opendata next? Tell us. Live 9/13 1800 GMT http://ow.ly/6hCvz
108980355138199552
Mitalikm
RT @WorldBank: Where should we take #opendata next? Tell us. Live 9/13 1800 GMT http://ow.ly/6hCvz
108976950365798400
abmakulec
RT @chilobbyists: Official statement from Rahm's office on the release of more lobbyist data and our collaboration: http://t.co/ENpUMjY #opengov #opendata
108976121231577088
rougeux
RT @WorldBank: Where should we take #opendata next? Tell us. Live 9/13 1800 GMT http://ow.ly/6hCvz
108976005695283200
avilarenata
RT @vinsumner: #AmsterdamSmartCity add your information http://t.co/1z5SzDI about the city , #opendata , #picnic11
108975904633524224
spolliaro
RT @worldbank: Where should we take #opendata next? Tell us. Live 9/13 1800 GMT http://t.co/h9PV2iO
108975758717886464
MGPSI

Statistics

Average successful run time: half a minute

Total run time: half a minute

Total cpu time used: less than 5 seconds

Total disk space used: 55.8 MB

History

  • Manually ran revision aafb8bc2 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Forked from ScraperWiki

Scraper code

Python

basic_twitter_scraper_2 / scraper.py