anishsugathan / wikipedia_power_plants

Wikipedia Power Plants

Scrapes live.dbpedia.org, www.example.com, and www.gov.uk

DBpedia-Live | DBpedia


This scraper retrieves all the pages for power plants on Wikipedia which are organized using the hierarchy of categories starting at Category:Power_stations_by_country. This is achieved via a single SPARQL query which performs a hierarchical category traversal using the endpoint at live.dbpedia.org.

Forked from ScraperWiki

Contributors anishsugathan

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... creating data object about to run query creating data object about to run query Traceback (most recent call last): File "scraper.py", line 161, in <module> parse_csv(csv_url(queryString)) File "scraper.py", line 28, in parse_csv data.write(scraperwiki.scrape(url)) File "/app/.heroku/src/scraperwiki/scraperwiki/utils.py", line 31, in scrape f = urllib2.urlopen(req) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 410, in open response = meth(req, response) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 523, in http_response 'http', request, response, code, msg, hdrs) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 448, in error return self._call_chain(*args) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/app/.heroku/python/lib/python2.7/urllib2.py", line 531, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 504: Gateway Time-out

Data

Downloaded 1 time by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.18 MB) Use the API

rows 10 / 3725

category latitude plant longitude longEw latS longS latNs latD longM latM longD geoLat geoLong locale country installedCapacity generationUnits primaryFuel owner categories status latDMS longDMS averageAnnualGen secondaryFuel
8.16888888888889
77.7125
E
8
45
N
8
42
10
77
8.16888888888889
77.7125
E
8
45
N
8
42
10
77
37.4333333333333
138.6
E
N
37
36
26
138
42.9136123657227
-115.070556640625
42.91361236572266
-115.070556640625
40.7077789306641
-74.0047225952148
40.70777893066406
-74.00472259521484
48.3416666666667
-114.014166666667
Flathead County, near Columbia Falls, Montana, USA
428.0
Hydroelectric power plants in Montana
48.34166666666667
-114.0141666666667
Wind farms in North Dakota, Wind farms in South Dakota
43.4883346557617
-85.6322250366211
43.48833465576172
-85.63222503662109
Hydroelectric power plants in Michigan

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.18 MB) Use the API

rows 0 / 0

Statistics

Total run time: 1 minute

Total cpu time used: less than 5 seconds

Total disk space used: 1.22 MB

History

  • Manually ran revision 4e6151eb and failed .
    nothing changed in the database
    5 pages scraped
  • Forked from ScraperWiki

Scraper code

Python

wikipedia_power_plants / scraper.py