g0vhkio / district-minor-works-crawler

Crawler of https://www.dmw.gov.hk/tc_chi/results.php

Scrapes www.dmw.gov.hk

District Minor Works

Contributors howawong

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Python app detected [1G ! The latest version of Python 3 is python-3.6.2 (you are using python-3.6.4, which is unsupported). [1G ! We recommend upgrading by specifying the latest version (python-3.6.2). [1G Learn More: https://devcenter.heroku.com/articles/python-runtimes [1G-----> Installing python-3.6.4 [1G-----> Installing pip [1G-----> Installing requirements with pip [1G Obtaining scraperwiki from git+http://github.com/andylolz/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 1)) [1G Cloning http://github.com/andylolz/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki [1G Collecting requests==2.18.4 (from -r /tmp/build/requirements.txt (line 3)) [1G Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB) [1G Collecting beautifulsoup4==4.6.0 (from -r /tmp/build/requirements.txt (line 4)) [1G Downloading https://files.pythonhosted.org/packages/9e/d4/10f46e5cfac773e22707237bfcd51bbffeaf0a576b0a847ec7ab15bd7ace/beautifulsoup4-4.6.0-py3-none-any.whl (86kB) [1G Collecting six (from scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl [1G Collecting sqlalchemy (from scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/aa/cc/348eec885d81f7260b07d961b3ececfc0aa82f7d4a8f45ff997e0d3f44ba/SQLAlchemy-1.2.11.tar.gz (5.6MB) [1G Collecting alembic (from scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/96/c7/a4129db460c3e0ea8fea0c9eb5de6680d38ea6b6dcffcb88898ae42e170a/alembic-1.0.0-py2.py3-none-any.whl (158kB) [1G Collecting chardet<3.1.0,>=3.0.2 (from requests==2.18.4->-r /tmp/build/requirements.txt (line 3)) [1G Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) [1G Collecting idna<2.7,>=2.5 (from requests==2.18.4->-r /tmp/build/requirements.txt (line 3)) [1G Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB) [1G Collecting certifi>=2017.4.17 (from requests==2.18.4->-r /tmp/build/requirements.txt (line 3)) [1G Downloading https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl (147kB) [1G Collecting urllib3<1.23,>=1.21.1 (from requests==2.18.4->-r /tmp/build/requirements.txt (line 3)) [1G Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB) [1G Collecting Mako (from alembic->scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz (564kB) [1G Collecting python-dateutil (from alembic->scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB) [1G Collecting python-editor>=0.3 (from alembic->scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/65/1e/adf6e000ea5dc909aa420352d6ba37f16434c8a3c2fa030445411a1ed545/python-editor-1.0.3.tar.gz [1G Collecting MarkupSafe>=0.9.2 (from Mako->alembic->scraperwiki->-r /tmp/build/requirements.txt (line 1)) [1G Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz [1G Installing collected packages: chardet, idna, certifi, urllib3, requests, six, sqlalchemy, MarkupSafe, Mako, python-dateutil, python-editor, alembic, scraperwiki, beautifulsoup4 [1G Running setup.py install for sqlalchemy: started [1G Running setup.py install for sqlalchemy: finished with status 'done' [1G Running setup.py install for MarkupSafe: started [1G Running setup.py install for MarkupSafe: finished with status 'done' [1G Running setup.py install for Mako: started [1G Running setup.py install for Mako: finished with status 'done' [1G Running setup.py install for python-editor: started [1G Running setup.py install for python-editor: finished with status 'done' [1G Running setup.py develop for scraperwiki [1G Successfully installed Mako-1.0.7 MarkupSafe-1.0 alembic-1.0.0 beautifulsoup4-4.6.0 certifi-2018.8.24 chardet-3.0.4 idna-2.6 python-dateutil-2.7.3 python-editor-1.0.3 requests-2.18.4 scraperwiki six-1.11.0 sqlalchemy-1.2.11 urllib3-1.22 [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Total number of pages: 80 Finished page 1. Finished page 2. Finished page 3. Finished page 4. Finished page 5. Finished page 6. Finished page 7. Finished page 8. Finished page 9. Finished page 10. Finished page 11. Finished page 12. Finished page 13. Finished page 14. Finished page 15. Finished page 16. Finished page 17. Finished page 18. Finished page 19. Finished page 20. Finished page 21. Finished page 22. Finished page 23. Finished page 24. Finished page 25. Finished page 26. Finished page 27. Finished page 28. Finished page 29. Finished page 30. Finished page 31. Finished page 32. Finished page 33. Finished page 34. Finished page 35. Finished page 36. Finished page 37. Finished page 38. Finished page 39. Finished page 40. Finished page 41. Finished page 42. Finished page 43. Finished page 44. Finished page 45. Finished page 46. Finished page 47. Finished page 48. Finished page 49. Finished page 50. Finished page 51. Finished page 52. Finished page 53. Finished page 54. Finished page 55. Finished page 56. Finished page 57. Finished page 58. Finished page 59. Finished page 60. Finished page 61. Finished page 62. Finished page 63. Finished page 64. Finished page 65. Finished page 66. Finished page 67. Finished page 68. Finished page 69. Finished page 70. Finished page 71. Finished page 72. Finished page 73. Finished page 74. Finished page 75. Finished page 76. Finished page 77. Finished page 78. Finished page 79. Finished page 80.

Data

Downloaded 1 time by g0vhkio

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (211 KB) Use the API

rows 10 / 798

project_id category location department run_date year details contact
C&W-DMW159
休憩處
位於中西區山道54-56號的休憩處
民政事務總署
2018-09-04
2010
約三十平方米休憩處 兩張長椅 一張雲石枱連兩張櫈
中西區民政事務處熱線: 2852 3481
C&W-DMW241
蔭棚
位於中西區堅尼地城吉席街近泓都2座的蔭棚
民政事務總署
2018-09-04
2013
兩座約三米長附有一張長椅的蔭棚六個花盆
中西區民政事務處熱線: 2189 2819
C&W-DMW002
涼亭
位於中西區薄扶林水塘道近山頂的涼亭
民政事務總署
2018-09-04
2009
一座約十二平方米附有三張長椅的涼亭
中西區民政事務處熱線: 2189 2819
C&W-DMW165
避雨亭
位於中西區中環林士街碼頭座椅避雨亭
民政事務總署
2018-09-04
2011
一座約十四米長避雨亭
中西區民政事務處熱線: 2189 2819
C&W-DMW375
避雨亭
位於中西區加列山道明德國際醫院對出的避雨亭
民政事務總署
2018-09-04
2015
一座約三米長避雨亭
中西區民政事務處熱線: 2852 3481
C&W-DMW226
避雨亭
位於中西區加惠民道加惠臺對面的避雨亭
民政事務總署
2018-09-04
2013
一座約四米長避雨亭
中西區民政事務處熱線: 2852 3481
C&W-DMW007
避雨亭
位於中西區士美菲路13號小巴站避雨亭
民政事務總署
2018-09-04
2009
一座約四米長避雨亭
中西區民政事務處熱線: 2189 2819
C&W-DMW155
避雨亭
位於中西區泓都的士站避雨亭
民政事務總署
2018-09-04
2011
一座約四米長避雨亭
中西區民政事務處熱線: 2189 2819
C&W-DMW216
避雨亭
位於中西區羅便臣道4號巴士站避雨亭
民政事務總署
2018-09-04
2013
一座約六米長附有九張座椅的避雨亭
中西區民政事務處熱線: 2189 2819
C&W-DMW124
避雨亭
位於中西區近般咸道及高街交界的避雨亭
民政事務總署
2018-09-04
2011
一座約四米長避雨亭
中西區民政事務處熱線: 2852 3481

Statistics

Average successful run time: 19 minutes

Total run time: 20 minutes

Total cpu time used: 2 minutes

Total disk space used: 251 KB

History

  • Manually ran revision 7b8fa3e3 and completed successfully .
    798 records added in the database
    879 pages scraped
  • Manually ran revision d431fb75 and failed .
    171 pages scraped
  • Manually ran revision d431fb75 and failed .
  • Manually ran revision d431fb75 and failed .
  • Created on morph.io