g0vhkio / legco-council-meeting-question-bot

Bot for crawling https://www.legco.gov.hk/yr16-17/chinese/counmtg/question/ques1617.htm

README.md

legco-council-meeting-question-bot

Bot for crawling https://www.legco.gov.hk/yr16-17/chinese/counmtg/question/ques1617.htm

Contributors g0vhkio howawong

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 3 is python-3.6.2 (you are using python-3.6.4, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-3.6.2).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-3.6.4 -----> Installing pip -----> Installing requirements with pip  Collecting requests==2.18.4  Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)  Collecting pytz==2017.3  Downloading https://files.pythonhosted.org/packages/a3/7f/e7d1acbd433b929168a4fb4182a2ff3c33653717195a26c1de099ad1ef29/pytz-2017.3-py2.py3-none-any.whl (511kB)  Collecting pillow==5.2.0  Downloading https://files.pythonhosted.org/packages/d1/24/f53ff6b61b3d728b90934bddb4f03f8ab584a7f49299bf3bde56e2952612/Pillow-5.2.0-cp36-cp36m-manylinux1_x86_64.whl (2.0MB)  Collecting slackclient==1.2.0  Downloading https://files.pythonhosted.org/packages/15/92/ad635dd1755c9943e989adbce84a554d2f9f4006aeba669f4b9286b71d91/slackclient-1.2.0.tar.gz  Collecting textwrap3==0.9.1  Downloading https://files.pythonhosted.org/packages/e8/78/3bb4b819ae7ab81ba0759788b28671d7ae13186544c333f973e6c91a6726/textwrap3-0.9.1-py2.py3-none-any.whl  Collecting lxml==4.1.1  Downloading https://files.pythonhosted.org/packages/81/da/4fabb01d59f3d2b256e75ffee9516ea849e23ddebea8235caad570fe7332/lxml-4.1.1-cp36-cp36m-manylinux1_x86_64.whl (5.6MB)  Collecting simplejson==3.16.0  Downloading https://files.pythonhosted.org/packages/e3/24/c35fb1c1c315fc0fffe61ea00d3f88e85469004713dab488dee4f35b0aff/simplejson-3.16.0.tar.gz (81kB)  Obtaining scraperwiki from git+http://github.com/andylolz/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 8))  Cloning http://github.com/andylolz/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/andylolz/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting idna<2.7,>=2.5  Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)  Collecting urllib3<1.23,>=1.21.1  Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)  Collecting chardet<3.1.0,>=3.0.2  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Collecting certifi>=2017.4.17  Downloading https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl (154kB)  Collecting websocket-client<1.0a0,>=0.35  Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)  Collecting six<2.0a0,>=1.10  Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl  Collecting sqlalchemy  Downloading https://files.pythonhosted.org/packages/34/5c/0e1d7ad0ca52544bb12f9cb8d5cc454af45821c92160ffedd38db0a317f6/SQLAlchemy-1.3.11.tar.gz (6.0MB)  Collecting alembic  Downloading https://files.pythonhosted.org/packages/84/64/493c45119dce700a4b9eeecc436ef9e8835ab67bae6414f040cdc7b58f4b/alembic-1.3.1.tar.gz (1.1MB)  Collecting Mako  Downloading https://files.pythonhosted.org/packages/b0/3c/8dcd6883d009f7cae0f3157fb53e9afb05a0d3d33b3db1268ec2e6f4a56b/Mako-1.1.0.tar.gz (463kB)  Collecting python-editor>=0.3  Downloading https://files.pythonhosted.org/packages/c6/d3/201fc3abe391bbae6606e6f1d598c15d367033332bd54352b12f35513717/python_editor-1.0.4-py3-none-any.whl  Collecting python-dateutil  Downloading https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227kB)  Collecting MarkupSafe>=0.9.2  Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl  Building wheels for collected packages: slackclient, simplejson, sqlalchemy, alembic, Mako  Building wheel for slackclient (setup.py): started  Building wheel for slackclient (setup.py): finished with status 'done'  Created wheel for slackclient: filename=slackclient-1.2.0-py2.py3-none-any.whl size=16925 sha256=9c40b891cede5fd43c9102bc08080d22db718d4a270bb8b9f2b490cb114deb33  Stored in directory: /tmp/pip-ephem-wheel-cache-1pyf_1pd/wheels/69/1e/ea/17ec4eebc19ec60e448a945c30ec07b6626b5669c1a1b477b7  Building wheel for simplejson (setup.py): started  Building wheel for simplejson (setup.py): finished with status 'done'  Created wheel for simplejson: filename=simplejson-3.16.0-cp36-cp36m-linux_x86_64.whl size=118875 sha256=3da4afcb27d654103c1826ea25cdf52fe1af144ecbd4346ff51ddd0d84e21f1c  Stored in directory: /tmp/pip-ephem-wheel-cache-1pyf_1pd/wheels/5d/1a/1e/0350bb3df3e74215cd91325344cc86c2c691f5306eb4d22c77  Building wheel for sqlalchemy (setup.py): started  Building wheel for sqlalchemy (setup.py): finished with status 'done'  Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.11-cp36-cp36m-linux_x86_64.whl size=1197083 sha256=993bb42ca8eef354c29aa560ff1f98ee24d074ca8c468b7d03f78a0b15a37fdd  Stored in directory: /tmp/pip-ephem-wheel-cache-1pyf_1pd/wheels/a3/67/7d/6c41104a1a08ff1a25e260d3edec3ac19203141d1aaa2f0975  Building wheel for alembic (setup.py): started  Building wheel for alembic (setup.py): finished with status 'done'  Created wheel for alembic: filename=alembic-1.3.1-py2.py3-none-any.whl size=144523 sha256=28fe599582c2b7e9dac0c462e5276aa903fa33ac2c313a17d0bf9f721939df0b  Stored in directory: /tmp/pip-ephem-wheel-cache-1pyf_1pd/wheels/b2/d4/19/5ab879d30af7cbc79e6dcc1d421795b1aa9d78f455b0412ef7  Building wheel for Mako (setup.py): started  Building wheel for Mako (setup.py): finished with status 'done'  Created wheel for Mako: filename=Mako-1.1.0-cp36-none-any.whl size=75363 sha256=1c47764899bdfcc9e57ea7b8520c9c13a92b97099b0e34ada910133f17a86ca6  Stored in directory: /tmp/pip-ephem-wheel-cache-1pyf_1pd/wheels/98/32/7b/a291926643fc1d1e02593e0d9e247c5a866a366b8343b7aa27  Successfully built slackclient simplejson sqlalchemy alembic Mako  Installing collected packages: idna, urllib3, chardet, certifi, requests, pytz, pillow, six, websocket-client, slackclient, textwrap3, lxml, simplejson, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, scraperwiki  Running setup.py develop for scraperwiki  Successfully installed Mako-1.1.0 MarkupSafe-1.1.1 alembic-1.3.1 certifi-2019.9.11 chardet-3.0.4 idna-2.6 lxml-4.1.1 pillow-5.2.0 python-dateutil-2.8.1 python-editor-1.0.4 pytz-2017.3 requests-2.18.4 scraperwiki simplejson-3.16.0 six-1.13.0 slackclient-1.2.0 sqlalchemy-1.3.11 textwrap3-0.9.1 urllib3-1.22 websocket-client-0.56.0   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... 19 https://www.legco.gov.hk/yr19-20/chinese/counmtg/question/ques1920.htm 2019-10-23 https://www.info.gov.hk/gia/general/201910/23/P2019102300359.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300393.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300530.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300579.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300519.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300435.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300357.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300457.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300218.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300246.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300380.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300350.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300303.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300267.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300504.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300616.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300628.htm https://www.info.gov.hk/gia/general/201910/23/P2019102200483.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300552.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300500.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300381.htm https://www.info.gov.hk/gia/general/201910/23/P2019102300300.htm 2019-10-30 https://www.info.gov.hk/gia/general/201910/30/P2019103000356.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000471.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000393.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000448.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000403.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000408.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000350.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000378.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000297.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000310.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000454.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000209.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000267.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000223.htm https://www.info.gov.hk/gia/general/201910/30/P2019102900497.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000449.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000367.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000354.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000458.htm https://www.info.gov.hk/gia/general/201910/30/P2019103000331.htm 2019-11-06 https://www.info.gov.hk/gia/general/201911/06/P2019110600364.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600511.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600506.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600538.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600443.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600446.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600522.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600580.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600228.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600480.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600393.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600308.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600249.htm https://www.info.gov.hk/gia/general/201911/06/P2019110500274.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600269.htm https://www.info.gov.hk/gia/general/201911/06/P2019110500360.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600806.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600336.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600362.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600496.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600373.htm https://www.info.gov.hk/gia/general/201911/06/P2019110600286.htm 2019-11-13 https://www.info.gov.hk/gia/general/201911/13/P2019111300648.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300524.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300538.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300546.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300526.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300529.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300585.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300570.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300356.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300448.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300400.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300429.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300476.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300660.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300650.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300474.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300482.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300541.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300484.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300375.htm?fontSize=1 https://www.info.gov.hk/gia/general/201911/13/P2019111300387.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300556.htm https://www.info.gov.hk/gia/general/201911/13/P2019111300319.htm 2019-11-20 https://www.info.gov.hk/gia/general/201911/20/P2019112000445.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000332.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000324.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000354.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000352.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000232.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000327.htm https://www.info.gov.hk/gia/general/201911/20/P2019112000294.htm Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded Already uploaded

Data

Downloaded 1 time by g0vhkio

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (248 KB) Use the API

rows 10 / 1276

key date link individual
784834a4eaf6c13013ba745701591bc6
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400399.htm
范國威議員
684b4e44ed0d9bb6d0dff411e5b4f763
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400505.htm
盧偉國議員
873b9633d02157d29845cfb5937e1c92
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400540.htm
陳恒鑌議員
bddea1da3eed2cd41a1be281aefd6407
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400528.htm
何啟明議員
160ef23abe14df2495a2ca47c8bd3621
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400637.htm
郭偉強
dca7711b2efa9226ff81931e661fd5c4
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400613.htm
葛珮帆議員
96eb6f987f06a47ec16125502b840eb2
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400442.htm
林卓廷議員
692eceecafcf09c33214443470e78eda
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400316.htm
謝偉銓議員
da08d32989187f53b3754d05c8fd8fb0
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400455.htm
吳永嘉議員
9829545820ffb420c75e0c2570a7bad5
2018-10-24
https://www.info.gov.hk/gia/general/201810/24/P2018102400639.htm
李國麟議員

Statistics

Average successful run time: 4 minutes

Total run time: about 1 hour

Total cpu time used: 3 minutes

Total disk space used: 288 KB

History

  • Auto ran revision 53d39072 and completed successfully .
    95 records added, 95 records removed in the database
  • Auto ran revision 53d39072 and completed successfully .
    95 records added, 87 records removed in the database
  • Auto ran revision 53d39072 and completed successfully .
    87 records added, 87 records removed in the database
  • Auto ran revision 53d39072 and completed successfully .
    87 records added, 87 records removed in the database
  • Auto ran revision 53d39072 and completed successfully .
    87 records added, 87 records removed in the database
  • ...
  • Created on morph.io

Show complete history