SuzanaK / basic_twitter_scraper_with_geocode

Twitter Scraper with geocoding and date

Scrapes search.twitter.com

Search Twitter to find the latest news and world events faster. Find popular people, hashtags and photos for any topic you can imagine.


This is an example scraper of how to get tweets within a special area and time frame.

Made as a response to this Stackoverflow Question:
http://stackoverflow.com/questions/16472463/scraperwiki-twitter-query/16473317#16473317

Forked from ScraperWiki

Contributors SuzanaK

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Traceback (most recent call last): File "scraper.py", line 27, in <module> for result in result_json['results']: KeyError: 'results'

Data

Downloaded 1 time by SuzanaK

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (18 KB) Use the API

rows 10 / 15

text to_user_id_str in_reply_to_status_id id profile_image_url_https source geocode id_str from_user_name metadata profile_image_url to_user from_user from_user_id to_user_id iso_language_code to_user_name created_at location in_reply_to_status_id_str from_user_id_str place geo
@WillMcAvoyACN vs. Europe (exl Eastern Europe) & a few areas of Asia. But what is your guess as to WHY violent crime for us is worse?
617825073
332643922750869505
332644719144337408
<a href="http://twitter.com/download/iphone">Twitter for iPhone</a>
37.781157,-122.398720,25mi
332644719144337408
James Wilder
{'result_type': 'recent'}
WillMcAvoyACN
jamesdwilder
55937627
617825073
en
Will McAvoy
Thu, 09 May 2013 23:54:15 +0000
San Francisco, CA
332643922750869505
55937627
Have you taught abroad in #Vienna?! Help us out and write a teach guide for @GoOverseas! #TEFL #ESL #Austria #Europe http://t.co/rqP3sIOgdY
332643913460510720
<a href="http://bufferapp.com">Buffer</a>
37.781157,-122.398720,25mi
332643913460510720
Rachel Overseas
{'result_type': 'recent'}
TeachReviews
371015517
en
Thu, 09 May 2013 23:51:03 +0000
Berkeley, CA
371015517
@AutoPublication Are you in Canada or Europe?
152360033
332630126565158912
332639936018776064
<a href="http://itunes.apple.com/us/app/twitter/id409789998?mt=12">Twitter for Mac</a>
37.781157,-122.398720,25mi
332639936018776064
aerbook
{'result_type': 'recent'}
AutoPublication
aerbook
183055043
152360033
en
AutoPublication
Thu, 09 May 2013 23:35:15 +0000
San Francisco, CA
332630126565158912
183055043
RT @Crunchyroll: News: "Dead or Alive 5 Ultimate" Confirmed for North America and Europe http://t.co/utfovVcNJ0
332639402784337920
<a href="http://twitter.com/download/iphone">Twitter for iPhone</a>
37.781157,-122.398720,25mi
332639402784337920
Ryan Gendreau
{'result_type': 'recent'}
evilshadowx
36159823
en
Thu, 09 May 2013 23:33:08 +0000
Wishing I was in Japan
36159823
RT @Crunchyroll: News: "Dead or Alive 5 Ultimate" Confirmed for North America and Europe http://t.co/utfovVcNJ0
332639387978436608
<a href="http://levelupstudio.com/plume">Plume for Android</a>
37.781157,-122.398720,25mi
332639387978436608
Kristina
{'result_type': 'recent'}
ChibiUFO
35598437
en
Thu, 09 May 2013 23:33:04 +0000
Canada-Land
35598437
News: "Dead or Alive 5 Ultimate" Confirmed for North America and Europe http://t.co/utfovVcNJ0
332639128321667072
<a href="http://www.crunchyroll.com">Crunchyroll</a>
37.781157,-122.398720,25mi
332639128321667072
Crunchyroll
{'result_type': 'recent'}
Crunchyroll
28023330
en
Thu, 09 May 2013 23:32:02 +0000
San Francisco, CA
28023330
Europe stocks end higher after US data http://t.co/oCUF0K6ZRL
332638394066808832
<a href="http://www.snsanalytics.com">SNS Analytics</a>
37.781157,-122.398720,25mi
332638394066808832
MarketWatch News
{'result_type': 'recent'}
MarketWatchNews
200896018
en
Thu, 09 May 2013 23:29:07 +0000
San Francisco
200896018
Happy Europe Day! Register .EU, your European Identity, for just $2.99 during our #sale! http://t.co/2PbPYmZoVH @Euregistry
332636914261831680
<a href="http://twitter.com/">web</a>
37.781157,-122.398720,25mi
332636914261831680
Dynadot
{'result_type': 'recent'}
Dynadot
89060745
en
Thu, 09 May 2013 23:23:15 +0000
San Mateo, CA
89060745
I think I might go to Central Europe next summer :o
332635594591174657
<a href="http://twitter.com/download/iphone">Twitter for iPhone</a>
37.781157,-122.398720,25mi
332635594591174657
CarouselSniperVictim
{'result_type': 'recent'}
EricParkerAsc
870195673
en
Thu, 09 May 2013 23:18:00 +0000
870195673
{'type': 'CITY', 'id': '51d80bae0d6cb453', 'full_name': 'Fairview'}
{'type': 'Point', 'coordinates': [37.6866996929, -122.0547432772]}
Yet another brazen robbery. - Focus Turns to 3 Suspects in Belgian Diamond Robbery, via @nytimes http://t.co/sbJ5y2j11P
332634567586488320
<a href="http://itunes.apple.com/us/app/nytimes/id284862083?mt=8&uo=4">NYTimes on iOS</a>
37.781157,-122.398720,25mi
332634567586488320
Dushka Zapata
{'result_type': 'recent'}
DushkaAmateur
20890666
en
Thu, 09 May 2013 23:13:55 +0000
San Francisco
20890666

Statistics

Total run time: less than 5 seconds

Total cpu time used: less than 5 seconds

Total disk space used: 37.5 KB

History

  • Manually ran revision bf5147e7 and failed .
    nothing changed in the database
    1 page scraped
  • Forked from ScraperWiki