This is a scraper that runs on Morph. To get started see the documentation
Contributors
Last run completed successfully
.
Console output of last run
Injecting configuration and compiling...
[1G-----> Python app detected
[1G-----> Stack changed, re-installing runtime
[1G-----> Installing runtime (python-2.7.9)
[1G-----> Installing dependencies with pip
[1G Collecting requests==2.8.1 (from -r requirements.txt (line 1))
[1G Downloading requests-2.8.1-py2.py3-none-any.whl (497kB)
[1G Collecting pyquery==1.2.9 (from -r requirements.txt (line 2))
[1G Downloading pyquery-1.2.9.zip (45kB)
[1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 3))
[1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki
[1G Collecting lxml>=2.1 (from pyquery==1.2.9->-r requirements.txt (line 2))
[1G Downloading lxml-3.4.4.tar.gz (3.5MB)
[1G Building lxml version 3.4.4.
[1G Building without Cython.
[1G Using build configuration of libxslt 1.1.28
[1G /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
[1G warnings.warn(msg)
[1G Collecting cssselect (from pyquery==1.2.9->-r requirements.txt (line 2))
[1G Downloading cssselect-0.9.1.tar.gz
[1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 3))
[1G Downloading dumptruck-0.1.6.tar.gz
[1G Installing collected packages: dumptruck, cssselect, lxml, scraperwiki, pyquery, requests
[1G Running setup.py install for dumptruck
[1G Running setup.py install for cssselect
[1G Running setup.py install for lxml
[1G Building lxml version 3.4.4.
[1G Building without Cython.
[1G Using build configuration of libxslt 1.1.28
[1G /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
[1G warnings.warn(msg)
[1G building 'lxml.etree' extension
[1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-MlgtgY/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
[1G gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
[1G building 'lxml.objectify' extension
[1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-MlgtgY/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -w
[1G gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/objectify.so
[1G Running setup.py develop for scraperwiki
[1G Creating /app/.heroku/python/lib/python2.7/site-packages/scraperwiki.egg-link (link to .)
[1G Adding scraperwiki 0.3.7 to easy-install.pth file
[1G Installed /app/.heroku/src/scraperwiki
[1G Running setup.py install for pyquery
[1G
[1G Successfully installed cssselect-0.9.1 dumptruck-0.1.6 lxml-3.4.4 pyquery-1.2.9 requests-2.8.1 scraperwiki
[1G
[1G-----> Discovering process types
[1G Procfile declares types -> scraper
Injecting scraper and running...
1 100
2 100
3 100
4 100
5 100
6 100
7 100
8 100
9 100
10 100
11 100
12 100
13 100
14 100
15 100
16 100
17 100
18 100
19 100
20 100
21 100
22
100
23 100
24 100
25 100
26 100
27 100
28 100
29 100
30 100
31 100
32 100
33 100
34 100
35
100
36 100
37 100
38 100
39
100
40 100
41 100
42 100
43 100
44 100
45
100
46 100
47
100
48 100
49 100
50 100
51 100
52 100
53 100
54 100
55 100
56 100
57
100
58 100
59 100
60 100
61 100
62 100
63 100
64 100
65 100
66 100
67 100
68 100
69 100
70 100
71 100
72 100
73 100
74 100
75 100
76 100
77 100
78
100
79 100
80 100
81 100
82 100
83 100
84 100
85 100
86 100
87 100
88 100
89 100
90
100
91 100
92 100
93 100
94 100
95 100
96 100
97 100
98 100
99 100
100 100
101 100
102 100
103 100
104 100
105 100
106 100
107 100
108 100
109 100
110 100
111 100
112 100
113 100
114 100
115 100
116 100
117 100
118 100
119 100
120 100
121 100
122 100
123 100
124 100
125 100
126 100
127 100
128 100
129 100
130 100
131 100
132 100
133 100
134 100
135 100
136 100
137 100
138 100
139 100
140 64
141 0
Data
Downloaded
11 times
by
To download data sign in with GitHub
rows 10 / 13964
| id |
|---|
|
ISRCTN16261285
|
|
ISRCTN11198115
|
|
ISRCTN90871183
|
|
ISRCTN17963123
|
|
ISRCTN10304032
|
|
ISRCTN18642659
|
|
ISRCTN11570646
|
|
ISRCTN15957905
|
|
ISRCTN16804726
|
|
ISRCTN22549991
|
Statistics
Average successful run time: 8 minutes
Total run time: 26 minutes
Total cpu time used: less than a minute
Total disk space used: 704 KB
History
-
Manually
ran
revision
8abaa285
and
completed successfully
.
13964 records added in the database141 pages scraped
-
Manually
ran
revision
3e603ba3
and
completed successfully
.
nothing changed in the database141 pages scraped
-
Manually
ran
revision
5bf367de
and
failed
.
nothing changed in the database
-
Manually
ran
revision
83ddc1a7
and
completed successfully
.
nothing changed in the database
- Created on morph.io