masterofpun / r_datasets_traffic

r/datasets/about/traffic (hoping this goes somewhere in the long run)

Scrapes www.reddit.com

reddit: the front page of the internet


This is a scraper that runs on Morph. To get started see the documentation

Contributors masterofpun

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.5.1  $ pip install -r requirements.txt  Collecting requests==2.10.0 (from -r requirements.txt (line 8))  Downloading requests-2.10.0-py2.py3-none-any.whl (506kB)  Installing collected packages: requests  Successfully installed requests-2.10.0   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 1 time by masterofpun

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (44 KB) Use the API

rows 10 / 2713

time uniques pageviews
1470589200
31
97
1470592800
21
86
1470596400
19
35
1470600000
18
33
1470603600
16
45
1470607200
25
65
1470610800
22
44
1470614400
14
36
1470618000
21
29
1470621600
21
36

Statistics

Average successful run time: less than a minute

Total run time: about 2 hours

Total cpu time used: less than a minute

Total disk space used: 76 KB

History

  • Auto ran revision 1127cc54 and completed successfully .
    36 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    23 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    21 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    13 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    31 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    6 records added, 2 records updated in the database
    2 pages scraped
  • Manually ran revision 1127cc54 and completed successfully .
    20 records added, 1 record updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    11 records added, 2 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    97 records added in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    27 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    65 records added, 2 records updated in the database
    4 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    15 records added, 3 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    28 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    22 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    31 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    23 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    25 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    97 records added in the database
  • Auto ran revision 1127cc54 and completed successfully .
    57 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    13 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    34 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    5 records added, 3 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    30 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    19 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    42 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    14 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    34 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    8 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    35 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    23 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    21 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    32 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    26 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    13 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    97 records added in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran and failed .
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    5 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    35 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    18 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    22 records added, 2 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    28 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    34 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    25 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    21 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    16 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    27 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    14 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    36 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    28 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    52 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    18 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    19 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    29 records added, 2 records updated in the database
    3 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    77 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    15 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    33 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    12 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    19 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    34 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    16 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    32 records added, 1 record updated in the database
    987 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    11 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    86 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    13 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    19 records added, 3 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    29 records added, 2 records updated in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    41 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    5 records added, 1 record updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    27 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    18 records added, 3 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    23 records added, 2 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    46 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    18 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    97 records added in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    18 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    25 records added, 7 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    20 records added, 7 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    26 records added, 1 record updated in the database
    4 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    44 records added, 2 records updated in the database
    1 page scraped
  • Manually ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and failed .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    19 records added, 2 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    41 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    40 records added, 6 records updated in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    97 records added in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    26 records added, 2 records updated in the database
    1 page scraped
  • Auto ran revision 1127cc54 and completed successfully .
    7 records added, 2 records updated in the database
    28 pages scraped
  • Manually ran revision 1127cc54 and completed successfully .
    41 records added, 2 records updated in the database
    1 page scraped
  • Manually ran revision 1127cc54 and completed successfully .
    nothing changed in the database
  • Auto ran revision 1127cc54 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 1127cc54 and completed successfully .
    20 records added, 2 records updated in the database
    2013 pages scraped
  • Manually ran revision 1127cc54 and completed successfully .
    97 records added in the database
    1 page scraped
  • Created on morph.io

Scraper code

Python

r_datasets_traffic / scraper.py