Mortimerp9 / flickr_camera_finder

flickr camera finder

Scrapes www.flickr.com

Flickr, a Yahoo company | Flickr - Photo Sharing!


This scraper liberates the data from Flickr Camera Finder (http://www.flickr.com/cameras/), extracting a list of: * Camera brands * Camera models for each brands * numbers of users on Flickr when the camera model was first crawled.

The scraper also extracts "yesterday" stats for each camera: * number of users that uploaded photos yesterday * number of photos that were uploaded yesterday so that the evolution of the cameras can be plotted. Note that this doesn't do anything smart about the date as it stores the date of the crawl.

Please keep in mind the following note by Flickr: “The graphs are only accurate to the extent that we can automatically detect the camera used to take the photo or shoot the video (about 2/3rds of the time). That is not usually possible with cameraphones, therefore they are under-represented.”

Anyone is welcomed to contribute, just ask me.

Scraping history

  • ran on scraperwiki from 2011-10-01 to 2013-04-07 almost daily.
  • big hiatus when scraperwiki updated their API and went read-only.
  • running since 2015-06-04 on morph.io mostly daily.

You can find the data as an sqlite db or a csv file at https://morph.io/Mortimerp9/flickrcamerafinder

Forked from ScraperWiki

Contributors Mortimerp9

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running...

Data

Downloaded 3 times by cybersholt

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (84.3 MB) Use the API

rows 10 / 41

url name
/cameras/generalimaging/
General Imaging
/cameras/acer/
Acer
/cameras/apple/
Apple
/cameras/benq/
BenQ
/cameras/blackberry/
BlackBerry
/cameras/canon/
Canon
/cameras/casio/
Casio
/cameras/concord/
Concord
/cameras/docomo/
DoCoMo
/cameras/epson/
Epson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (84.3 MB) Use the API

rows 10 / 2470

name url camera_type brand_url record_date initial_count optical_zoom megapixels lcd_size media_type
CE-5330
/cameras/acer/ce-5330/
Point & Shoot
/cameras/acer/
2011-10-01
132691
Unknown
Unknown
Unknown
Unknown
CE-6430
/cameras/acer/ce-6430/
Point & Shoot
/cameras/acer/
2011-10-01
17946
Unknown
Unknown
Unknown
Unknown
CI-6330
/cameras/acer/ci-6330/
Point & Shoot
/cameras/acer/
2011-10-01
72747
Unknown
Unknown
Unknown
Unknown
CI-8330
/cameras/acer/ci-8330/
Point & Shoot
/cameras/acer/
2011-10-01
8467
Unknown
Unknown
Unknown
Unknown
CL-6300
/cameras/acer/cl-6300/
Point & Shoot
/cameras/acer/
2011-10-01
4337
Unknown
Unknown
Unknown
Unknown
CP-8531
/cameras/acer/cp-8531/
Point & Shoot
/cameras/acer/
2011-10-01
9355
Unknown
Unknown
Unknown
Unknown
CP-8660
/cameras/acer/cp-8660/
Point & Shoot
/cameras/acer/
2011-10-01
10040
Unknown
Unknown
Unknown
Unknown
CP-X530
/cameras/acer/cp-x530/
Point & Shoot
/cameras/acer/
2011-10-01
1780
Unknown
Unknown
Unknown
Unknown
CR-8530
/cameras/acer/cr-8530/
Point & Shoot
/cameras/acer/
2011-10-01
69794
Unknown
Unknown
Unknown
Unknown
CS-5531
/cameras/acer/cs-5531/
Point & Shoot
/cameras/acer/
2011-10-01
26571
Unknown
Unknown
Unknown
Unknown

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (84.3 MB) Use the API

rows 1 / 1

value_blob type name
1531628027
integer
lastcrawl

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (84.3 MB) Use the API

rows 10 / 827197

url date photo_count camera_url daily_users
/cameras/htc/droid_incredible/
2011-10-01
2068344
/cameras/htc/elf_v2/
2011-10-01
6113
/cameras/htc/droideris/
2011-10-01
731295
/cameras/htc/evo4g/
2011-10-01
2652169
/cameras/htc/excalibur/
2011-10-01
324155
/cameras/htc/excalibur_v2/
2011-10-01
9853
/cameras/htc/gene/
2011-10-01
8055
/cameras/htc/gene_i/
2011-10-01
9232
/cameras/htc/hd7/
2011-10-01
27675
/cameras/htc/herald/
2011-10-01
78404

Statistics

Average successful run time: 3 minutes

Total run time: 16 days

Total cpu time used: about 1 hour

Total disk space used: 84.3 MB

History

  • Auto ran revision 316c31d2 and completed successfully .
    1 record added, 1 record removed in the database
    87 pages scraped
  • Auto ran revision 316c31d2 and completed successfully .
    2 records added, 2 records removed in the database
    90 pages scraped
  • Auto ran revision 316c31d2 and completed successfully .
    1 record added, 1 record removed in the database
    87 pages scraped
  • Auto ran revision 316c31d2 and completed successfully .
    1 record added, 1 record removed in the database
    87 pages scraped
  • Auto ran revision 316c31d2 and completed successfully .
    1 record added, 1 record removed in the database
    87 pages scraped
  • ...
  • Forked from ScraperWiki

Show complete history