g0vhkio / legco-council-meeting-vote-bot

Bot for crawling https://www.legco.gov.hk/php/detect-votes.php?term=yr17-18&meeting=cm


Contributors g0vhkio howawong

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 3 is python-3.6.2 (you are using python-3.6.4, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-3.6.2).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-3.6.4 -----> Installing pip -----> Installing requirements with pip  Collecting requests==2.18.4  Downloading requests-2.18.4-py2.py3-none-any.whl (88 kB)  Collecting pytz==2017.3  Downloading pytz-2017.3-py2.py3-none-any.whl (511 kB)  Collecting pillow==5.2.0  Downloading Pillow-5.2.0-cp36-cp36m-manylinux1_x86_64.whl (2.0 MB)  Collecting slackclient==1.2.0  Downloading slackclient-1.2.0.tar.gz (14 kB)  Collecting textwrap3==0.9.1  Downloading textwrap3-0.9.1-py2.py3-none-any.whl (12 kB)  Collecting lxml==4.1.1  Downloading lxml-4.1.1-cp36-cp36m-manylinux1_x86_64.whl (5.6 MB)  Collecting simplejson==3.16.0  Downloading simplejson-3.16.0.tar.gz (81 kB)  Obtaining scraperwiki from git+http://github.com/andylolz/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 8))  Cloning http://github.com/andylolz/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Collecting idna<2.7,>=2.5  Downloading idna-2.6-py2.py3-none-any.whl (56 kB)  Collecting chardet<3.1.0,>=3.0.2  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)  Collecting urllib3<1.23,>=1.21.1  Downloading urllib3-1.22-py2.py3-none-any.whl (132 kB)  Collecting websocket-client<1.0a0,>=0.35  Downloading websocket_client-0.57.0-py2.py3-none-any.whl (200 kB)  Collecting six<2.0a0,>=1.10  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)  Collecting sqlalchemy  Downloading SQLAlchemy-1.3.19-cp36-cp36m-manylinux2010_x86_64.whl (1.3 MB)  Collecting alembic  Downloading alembic-1.4.3-py2.py3-none-any.whl (159 kB)  Collecting Mako  Downloading Mako-1.1.3-py2.py3-none-any.whl (75 kB)  Collecting python-editor>=0.3  Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)  Collecting python-dateutil  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)  Collecting MarkupSafe>=0.9.2  Downloading MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl (27 kB)  Building wheels for collected packages: slackclient, simplejson  Building wheel for slackclient (setup.py): started  Building wheel for slackclient (setup.py): finished with status 'done'  Created wheel for slackclient: filename=slackclient-1.2.0-py2.py3-none-any.whl size=16924 sha256=39bcec05c98851eab2edd8f5232f6ca5de986866be66ec9a20d7d0bfff658776  Stored in directory: /tmp/pip-ephem-wheel-cache-ufo1bec1/wheels/98/3b/e0/34fc1e36412d56bfc3453032836897b02518e18dec9e071e3a  Building wheel for simplejson (setup.py): started  Building wheel for simplejson (setup.py): finished with status 'done'  Created wheel for simplejson: filename=simplejson-3.16.0-cp36-cp36m-linux_x86_64.whl size=118872 sha256=4d0c6e50e48199e3d50cb4ca918a0e25615e776373991f1682ca4938ad289162  Stored in directory: /tmp/pip-ephem-wheel-cache-ufo1bec1/wheels/fc/f1/eb/61fc974e24fc8c2fca0b524ab400636baced588708d25b16b4  Successfully built slackclient simplejson  Installing collected packages: idna, chardet, certifi, urllib3, requests, pytz, pillow, six, websocket-client, slackclient, textwrap3, lxml, simplejson, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, scraperwiki  Running setup.py develop for scraperwiki  Successfully installed Mako-1.1.3 MarkupSafe-1.1.1 alembic-1.4.3 certifi-2020.6.20 chardet-3.0.4 idna-2.6 lxml-4.1.1 pillow-5.2.0 python-dateutil-2.8.1 python-editor-1.0.4 pytz-2017.3 requests-2.18.4 scraperwiki simplejson-3.16.0 six-1.15.0 slackclient-1.2.0 sqlalchemy-1.3.19 textwrap3-0.9.1 urllib3-1.22 websocket-client-0.57.0   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... http://www.legco.gov.hk/php/detect-votes.php?term=yr19-20&meeting=cm http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20191023.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20191127.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20191204.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20191211.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20191218.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200115.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200325a.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200429.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200506.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200513.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200520.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200521.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200527.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200603.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200610.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200617.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200708.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/counmtg/voting/cm_vote_20200715.xml already loaded to API server. http://www.legco.gov.hk/php/detect-votes.php?term=yr19-20&meeting=esc http://www.legco.gov.hk/yr19-20/chinese/fc/esc/results/esc_vote_20200108.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/esc/results/esc_vote_20200506.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/esc/results/esc_vote_20200520.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/esc/results/esc_vote_20200603.xml already loaded to API server. http://www.legco.gov.hk/php/detect-votes.php?term=yr19-20&meeting=pwsc http://www.legco.gov.hk/yr19-20/chinese/fc/pwsc/results/pwsc_vote_20191106.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/pwsc/results/pwsc_vote_20191120.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/pwsc/results/pwsc_vote_20200115.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/pwsc/results/pwsc_vote_20200511.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/pwsc/results/pwsc_vote_20200601.xml already loaded to API server. http://www.legco.gov.hk/php/detect-votes.php?term=yr19-20&meeting=hc http://www.legco.gov.hk/yr19-20/chinese/hc/voting/hc_vote_20191122.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/hc/voting/hc_vote_20191129.xml already loaded to API server. http://www.legco.gov.hk/php/detect-votes.php?term=yr19-20&meeting=fc http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20191011.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20191108.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20191206.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20191213.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200221.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200228.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200306.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200311.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200313.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200320.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200418.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200515.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200522.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200529.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200612.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200619.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200626.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200630.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200702.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200703.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200706.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200707.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200710.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200713.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200714.xml already loaded to API server. http://www.legco.gov.hk/yr19-20/chinese/fc/fc/results/fc_vote_20200717.xml already loaded to API server.

Statistics

Average successful run time: 2 minutes

Total run time: about 9 hours

Total cpu time used: 7 minutes

Total disk space used: 104 KB

History

  • Auto ran revision 35da34ca and completed successfully .
    55 records added, 55 records removed in the database
  • Auto ran revision 35da34ca and completed successfully .
    55 records added, 55 records removed in the database
  • Auto ran revision 35da34ca and completed successfully .
    55 records added, 55 records removed in the database
  • Auto ran revision 35da34ca and completed successfully .
    55 records added, 55 records removed in the database
  • Auto ran revision 35da34ca and completed successfully .
    55 records added, 55 records removed in the database
  • ...
  • Created on morph.io

Show complete history