Brandhunt / Brandhunt_produpdate_scraper_phantomjs_center_py

Collects all scraped information from all related headless browser scrapers on Morph.IO into one solitary database


This is a scraper that runs on Morph. To get started see the documentation

Contributors Brandhunt

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.6.2  ! Requested runtime (python-3.6.2) is not available for this stack (cedar-14).  ! Aborting. More info: https://devcenter.heroku.com/articles/python-support

Data

Downloaded 110721 times by Brandhunt

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (10.8 MB) Use the API

rows 10 / 3967

productid url domain price salesprice domainmisc prodlogurls prodlogurl finalimgurls validimgurls imgurls notfound notavailable removeon404 soldoutfix soldouthtmlfix catstoaddresult attributes sizetypemapsqls
14957
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
[[{"term_id": 56, "name": "Shirts", "slug": "shirts", "taxonomy": "product_cat", "ancestors": []}, false]]
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 115, "name": "White", "slug": "color-white", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}, {"name": "Sizetype", "options": [[{"term_id": 2527, "name": "Shirts/Suits", "slug": "sizetype-shirtssuits", "taxonomy": "pa_sizetype"}, false]], "position": 4, "visible": 1, "variation": 1}]
["", "", "", ""]
15123
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 456, "name": "Burgundy", "slug": "color-burgundy", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15163
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15225
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15227
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 1330, "name": "Army", "slug": "color-army", "taxonomy": "pa_color"}, false], [{"term_id": 501, "name": "Blue/Army/Black", "slug": "color-bluearmyblack", "taxonomy": "pa_color"}, false], [{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false], [{"term_id": 120, "name": "Blue", "slug": "color-blue", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15229
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15231
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15233
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15235
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 115, "name": "White", "slug": "color-white", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15237
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
[[{"term_id": 32, "name": "Dresses", "slug": "dresses", "taxonomy": "product_cat", "ancestors": []}, false], [{"term_id": 55, "name": "Scarves", "slug": "scarves", "taxonomy": "product_cat", "ancestors": [17]}, false], [{"term_id": 17, "name": "Accessories", "slug": "accessories", "taxonomy": "product_cat", "ancestors": []}, false]]
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 1183, "name": "Ash", "slug": "color-ash", "taxonomy": "pa_color"}, false], [{"term_id": 503, "name": "Dress Gordon", "slug": "color-dress-gordon", "taxonomy": "pa_color"}, false], [{"term_id": 682, "name": "Tan", "slug": "color-tan", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}, {"name": "Sizetype", "options": [[{"term_id": 2034, "name": "Dresses", "slug": "sizetype-dresses", "taxonomy": "pa_sizetype"}, false], [{"term_id": 2033, "name": "Accessories", "slug": "sizetype-accessories", "taxonomy": "pa_sizetype"}, false]], "position": 4, "visible": 1, "variation": 1}]
["", "", "", ""]

Statistics

Average successful run time: 35 minutes

Total run time: about 2 months

Total cpu time used: 8 days

Total disk space used: 10.9 MB

History

  • Auto ran revision 122cc15b and failed .
    nothing changed in the database
  • Auto ran revision 122cc15b and failed .
    nothing changed in the database
  • Auto ran revision 122cc15b and failed .
    nothing changed in the database
  • Auto ran revision 122cc15b and failed .
    nothing changed in the database
  • Auto ran revision 122cc15b and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history