Brandhunt / Brandhunt_produpdate_scraper_phantomjs_py_module_1_v2

Collects product information to be updated from product URLs stored at Brandhunt.se - Only products that require usage of headless browser! Module scraper 1 for centrailzed main scraper


This is a scraper that runs on Morph. To get started see the documentation

Contributors Brandhunt

Last run failed with status code 999.

Data

Downloaded 67058 times by Brandhunt

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (4.22 MB) Use the API

rows 10 / 1531

productid url domain price salesprice domainmisc prodlogurls prodlogurl finalimgurls validimgurls imgurls notfound notavailable removeon404 soldoutfix soldouthtmlfix catstoaddresult attributes sizetypemapsqls
14957
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
[[{"term_id": 56, "name": "Shirts", "slug": "shirts", "taxonomy": "product_cat", "ancestors": []}, false]]
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 115, "name": "White", "slug": "color-white", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}, {"name": "Sizetype", "options": [[{"term_id": 2527, "name": "Shirts/Suits", "slug": "sizetype-shirtssuits", "taxonomy": "pa_sizetype"}, false]], "position": 4, "visible": 1, "variation": 1}]
["", "", "", ""]
15123
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 456, "name": "Burgundy", "slug": "color-burgundy", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15163
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15225
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15227
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 1330, "name": "Army", "slug": "color-army", "taxonomy": "pa_color"}, false], [{"term_id": 501, "name": "Blue/Army/Black", "slug": "color-bluearmyblack", "taxonomy": "pa_color"}, false], [{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false], [{"term_id": 120, "name": "Blue", "slug": "color-blue", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15229
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15231
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15233
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 117, "name": "Black", "slug": "color-black", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15235
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
""
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 115, "name": "White", "slug": "color-white", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}]
["", "", "", ""]
15237
bergbergstore.com
1
1
""
""
""
""
""
true
false
true
false
false
[[{"term_id": 32, "name": "Dresses", "slug": "dresses", "taxonomy": "product_cat", "ancestors": []}, false], [{"term_id": 55, "name": "Scarves", "slug": "scarves", "taxonomy": "product_cat", "ancestors": [17]}, false], [{"term_id": 17, "name": "Accessories", "slug": "accessories", "taxonomy": "product_cat", "ancestors": []}, false]]
[{"name": "Brand", "options": [[{"term_id": -1, "name": "Berg & Berg", "slug": "berg-berg", "taxonomy": "pa_brand"}, true]], "position": 1, "visible": 1, "variation": 1}, {"name": "Color", "options": [[{"term_id": 1183, "name": "Ash", "slug": "color-ash", "taxonomy": "pa_color"}, false], [{"term_id": 503, "name": "Dress Gordon", "slug": "color-dress-gordon", "taxonomy": "pa_color"}, false], [{"term_id": 682, "name": "Tan", "slug": "color-tan", "taxonomy": "pa_color"}, false]], "position": 2, "visible": 1, "variation": 1}, {"name": "Sex", "options": [[{"term_id": 141, "name": "Male", "slug": "male", "taxonomy": "pa_sex"}, false]], "position": 3, "visible": 1, "variation": 1}, {"name": "Sizetype", "options": [[{"term_id": 2034, "name": "Dresses", "slug": "sizetype-dresses", "taxonomy": "pa_sizetype"}, false], [{"term_id": 2033, "name": "Accessories", "slug": "sizetype-accessories", "taxonomy": "pa_sizetype"}, false]], "position": 4, "visible": 1, "variation": 1}]
["", "", "", ""]

Statistics

Average successful run time: about 2 hours

Total run time: 3 months

Total cpu time used: 7 days

Total disk space used: 4.24 MB

History

  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • ...
  • Created on morph.io

Show complete history