spendright / scrape-urls

Scrape URLs from SpendRight data for Facebook link, Twitter handle etc.

Scrapes www.thurrock.gov.uk

Thurrock Council


Contributors davidmarin

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.6  $ pip install -r requirements.txt  Collecting Unidecode==0.04.9 (from -r requirements.txt (line 2))  /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.  SNIMissingWarning  /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.  InsecurePlatformWarning  Downloading Unidecode-0.04.9.tar.gz (196kB)  Collecting beautifulsoup4==4.1.3 (from -r requirements.txt (line 3))  Downloading beautifulsoup4-4.1.3.tar.gz (58kB)  Collecting dumptruck==0.1.6 (from -r requirements.txt (line 4))  Downloading dumptruck-0.1.6.tar.gz  Collecting html5lib==0.90 (from -r requirements.txt (line 5))  Downloading html5lib-0.90.tar.gz (86kB)  Collecting requests==1.0.4 (from -r requirements.txt (line 6))  Downloading requests-1.0.4.tar.gz (336kB)  Installing collected packages: Unidecode, beautifulsoup4, dumptruck, html5lib, requests  Running setup.py install for Unidecode: started  Running setup.py install for Unidecode: finished with status 'done'  Running setup.py install for beautifulsoup4: started  Running setup.py install for beautifulsoup4: finished with status 'done'  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py install for html5lib: started  Running setup.py install for html5lib: finished with status 'done'  Running setup.py install for requests: started  Running setup.py install for requests: finished with status 'done'  Successfully installed Unidecode-0.4.9 beautifulsoup4-4.1.3 dumptruck-0.1.6 html5lib-0.90 requests-1.0.4   ! Hello! It looks like your application is using an outdated version of Python.  ! This caused the security warning you saw above during the 'pip install' step.  ! We recommend 'python-2.7.12', which you can specify in a 'runtime.txt' file.  ! -- Much Love, Heroku.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Traceback (most recent call last): File "scraper.py", line 23, in <module> from srs.db import create_table_if_not_exists ImportError: No module named srs.db

Data

Downloaded 356 times by spendright-scrapers MikeRalphson Sohuth bigsmaug

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (766 KB) Use the API

rows 10 / 4565

Statistics

Average successful run time: about 10 hours

Total run time: 2 months

Total cpu time used: 1 day

Total disk space used: 995 KB

History

  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    80 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    81 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    144 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    788 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    75 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    68 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    940 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    1915 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    191 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    81 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and failed .
    nothing changed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2279 records added, 2279 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2273 records added, 2273 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2265 records added, 2265 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2275 records added, 2275 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    955 records added, 955 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2280 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2275 records added, 2275 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2282 records added, 2282 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2281 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2289 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2289 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2285 records added, 2284 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2283 records added, 2283 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2284 records added, 2284 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2285 records added, 2285 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2277 records added, 2276 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2219 records added, 2219 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2290 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2281 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2288 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2287 records added, 2287 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2281 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2283 records added, 2283 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2292 records added, 2292 records removed in the database
  • Auto ran revision 91ffedb1 and completed successfully .
    2143 records added, 2142 records removed in the database
    2497 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2288 records removed in the database
  • Auto ran revision 91ffedb1 and failed .
    2018 records added, 2018 records removed in the database
    6494 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2294 records added, 2294 records removed in the database
    7603 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2290 records removed in the database
    7581 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2291 records added, 2290 records removed in the database
    15152 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2286 records added, 2286 records removed in the database
    7543 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2288 records removed in the database
    7559 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2300 records added, 2300 records removed in the database
    7575 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2291 records added, 2291 records removed in the database
    15092 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2292 records added, 2292 records removed in the database
    7533 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2289 records removed in the database
    7526 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2293 records added, 2291 records removed in the database
    7525 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2299 records added, 2298 records removed in the database
    7591 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2288 records removed in the database
    7548 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2300 records added, 2300 records removed in the database
    7579 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2301 records added, 2301 records removed in the database
    7517 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2288 records removed in the database
    7516 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2293 records added, 2293 records removed in the database
    7511 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2291 records added, 2291 records removed in the database
    7518 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2296 records added, 2296 records removed in the database
    7538 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2313 records added, 2313 records removed in the database
    7524 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2307 records added, 2306 records removed in the database
    7503 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2317 records added, 2317 records removed in the database
    7515 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2316 records added, 2315 records removed in the database
    7520 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2315 records added, 2315 records removed in the database
    7532 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2317 records added, 2316 records removed in the database
    7485 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2324 records added, 2324 records removed in the database
    7491 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2327 records added, 2326 records removed in the database
    7536 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2328 records added, 2326 records removed in the database
    7488 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2326 records added, 2326 records removed in the database
    7490 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2336 records added, 2335 records removed in the database
    7501 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2341 records added, 2341 records removed in the database
    7535 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2332 records added, 2332 records removed in the database
    7487 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2342 records added, 2342 records removed in the database
    7512 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2335 records added, 2335 records removed in the database
    7484 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2342 records added, 2341 records removed in the database
    7518 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2340 records added, 2340 records removed in the database
    7485 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2336 records added, 2335 records removed in the database
    7470 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2340 records added, 2340 records removed in the database
    7462 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2344 records added, 2344 records removed in the database
    14958 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2342 records added, 2342 records removed in the database
    7457 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2359 records added, 2358 records removed in the database
    7525 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2356 records added, 2340 records removed in the database
    7500 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2362 records added, 2362 records removed in the database
    7488 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2364 records added, 2364 records removed in the database
    7496 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2364 records added, 2364 records removed in the database
    7530 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2370 records added, 2369 records removed in the database
    7536 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2366 records added, 2366 records removed in the database
    15052 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2370 records added, 2367 records removed in the database
    7519 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2364 records added, 2362 records removed in the database
    7520 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2367 records added, 2367 records removed in the database
    7528 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2372 records added, 2372 records removed in the database
    7545 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2370 records added, 2367 records removed in the database
    7529 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2367 records added, 2366 records removed in the database
    7499 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2370 records added, 2370 records removed in the database
    7510 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2366 records added, 2366 records removed in the database
    7488 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2369 records added, 2369 records removed in the database
    7505 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2355 records added, 2355 records removed in the database
    7435 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2355 records added, 2355 records removed in the database
    7483 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2373 records added, 2373 records removed in the database
    7486 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2359 records added, 2358 records removed in the database
    7478 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2366 records added, 2365 records removed in the database
    7476 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2366 records added, 2365 records removed in the database
    7482 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2359 records added, 2359 records removed in the database
    7455 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2367 records added, 2367 records removed in the database
    7504 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2364 records added, 2363 records removed in the database
    7497 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2363 records added, 2363 records removed in the database
    7465 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2362 records added, 2362 records removed in the database
    7470 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2365 records added, 2365 records removed in the database
    7493 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2372 records added, 2372 records removed in the database
    7508 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2367 records added, 2367 records removed in the database
    7495 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2361 records added, 2361 records removed in the database
    7525 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2363 records added, 2363 records removed in the database
    7505 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2370 records added, 2370 records removed in the database
    7495 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2369 records added, 2369 records removed in the database
    7499 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2371 records added, 2370 records removed in the database
    7522 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2355 records added, 2355 records removed in the database
    7486 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2373 records added, 2373 records removed in the database
    7494 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2376 records added, 2376 records removed in the database
    7490 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2380 records added, 2379 records removed in the database
    7485 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2376 records added, 2376 records removed in the database
    7488 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2380 records added, 2380 records removed in the database
    7475 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2375 records added, 2375 records removed in the database
    7513 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2376 records added, 2376 records removed in the database
    7467 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2376 records added, 2375 records removed in the database
    7476 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2379 records added, 2379 records removed in the database
    7488 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2371 records added, 2371 records removed in the database
    7480 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2373 records added, 2373 records removed in the database
    7463 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2374 records added, 2373 records removed in the database
    7460 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2378 records added, 2378 records removed in the database
    7471 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2364 records added, 2364 records removed in the database
    7409 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2383 records added, 2382 records removed in the database
    7458 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2386 records added, 2386 records removed in the database
    7485 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2392 records added, 2391 records removed in the database
    7506 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2380 records added, 2379 records removed in the database
    7521 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2375 records added, 2375 records removed in the database
    7502 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2384 records added, 2384 records removed in the database
    7489 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2386 records added, 2386 records removed in the database
    7518 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2388 records added, 2388 records removed in the database
    7471 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2383 records added, 2381 records removed in the database
    7497 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2381 records added, 2381 records removed in the database
    7501 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2380 records added, 2380 records removed in the database
    7507 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2212 records added, 2212 records removed in the database
    6920 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2379 records added, 2379 records removed in the database
    9331 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2371 records added, 2371 records removed in the database
    7493 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2390 records added, 2388 records removed in the database
    7513 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2349 records added, 2349 records removed in the database
    7402 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2388 records added, 2349 records removed in the database
    7509 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2349 records added, 2348 records removed in the database
    7374 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2360 records added, 2359 records removed in the database
    7426 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2356 records added, 2356 records removed in the database
    7435 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2371 records added, 2370 records removed in the database
    7426 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2358 records added, 2358 records removed in the database
    7411 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2369 records added, 2369 records removed in the database
    7430 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2373 records added, 2370 records removed in the database
    7427 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2365 records added, 2331 records removed in the database
    7402 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2332 records added, 2330 records removed in the database
    7326 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2334 records added, 2332 records removed in the database
    7332 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2333 records added, 2333 records removed in the database
    7344 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2331 records added, 2331 records removed in the database
    7328 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2331 records added, 2320 records removed in the database
    7325 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2325 records added, 2325 records removed in the database
    7326 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2336 records added, 2336 records removed in the database
    7341 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2335 records added, 2334 records removed in the database
    7319 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2334 records added, 2334 records removed in the database
    7351 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2336 records added, 2335 records removed in the database
    7327 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2341 records added, 2309 records removed in the database
    7332 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2338 records added, 2335 records removed in the database
    7365 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2343 records added, 2343 records removed in the database
    7334 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2337 records added, 2292 records removed in the database
    7332 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2297 records added, 2247 records removed in the database
    7302 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2295 records added, 2295 records removed in the database
    7245 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2290 records removed in the database
    7246 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2296 records added, 2296 records removed in the database
    7230 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2290 records added, 2290 records removed in the database
    14467 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2298 records added, 2298 records removed in the database
    21820 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2296 records added, 2289 records removed in the database
    7254 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2289 records added, 2289 records removed in the database
    21590 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2295 records added, 2295 records removed in the database
    7225 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2295 records added, 2295 records removed in the database
    28891 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2283 records added, 2283 records removed in the database
    7214 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2283 records added, 2281 records removed in the database
    7195 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2288 records added, 2287 records removed in the database
    7205 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2297 records added, 2297 records removed in the database
    7222 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2267 records removed in the database
    7187 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2279 records added, 2279 records removed in the database
    7151 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2281 records added, 2280 records removed in the database
    21489 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2276 records added, 2276 records removed in the database
    14319 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2282 records added, 2280 records removed in the database
    14335 pages scraped
  • Auto ran revision 91ffedb1 and completed successfully .
    2288 records added, 2288 records removed in the database
    7173 pages scraped
  • Manually ran revision 91ffedb1 and completed successfully .
    2291 records added, 2288 records removed in the database
    14344 pages scraped
  • Auto ran revision 9e69c452 and failed .
    2287 records added, 1265 records removed in the database
    7203 pages scraped
  • Manually ran revision 093ef2d9 and completed successfully .
    3199 records added in the database
    5841 pages scraped
  • Created on morph.io

Scraper code

Python

scrape-urls / scraper.py