miedzinski / hltv

HLTV matches


This is a scraper that runs on Morph. To get started see the documentation

This scraper

Aims to collect CSGO matches from HLTV results page.

To get to know what's scraped see hltv.models.

License

This software is available under The MIT License, see the LICENSE file for more information.

Contributors miedzinski

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.9  $ pip install -r requirements.txt  Collecting py-dateutil==2.2 (from -r requirements.txt (line 1))  Downloading py-dateutil-2.2.tar.gz (64kB)  Collecting Scrapy==1.0.3 (from -r requirements.txt (line 2))  Downloading Scrapy-1.0.3-py2-none-any.whl (290kB)  Collecting SQLAlchemy==1.0.9 (from -r requirements.txt (line 3))  Downloading SQLAlchemy-1.0.9.tar.gz (4.7MB)  Collecting six (from py-dateutil==2.2->-r requirements.txt (line 1))  Downloading six-1.10.0-py2.py3-none-any.whl  Collecting lxml (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading lxml-3.7.2-cp27-cp27m-manylinux1_x86_64.whl (6.8MB)  Collecting service-identity (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading service_identity-16.0.0-py2.py3-none-any.whl  Collecting pyOpenSSL (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading pyOpenSSL-16.2.0-py2.py3-none-any.whl (43kB)  Collecting queuelib (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading queuelib-1.4.2-py2.py3-none-any.whl  Collecting cssselect>=0.9 (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading cssselect-1.0.1-py2.py3-none-any.whl  Collecting Twisted>=10.0.0 (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading Twisted-16.6.0.tar.bz2 (3.0MB)  Collecting w3lib>=1.8.0 (from Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading w3lib-1.16.0-py2.py3-none-any.whl  Collecting pyasn1 (from service-identity->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading pyasn1-0.1.9-py2.py3-none-any.whl  Collecting pyasn1-modules (from service-identity->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading pyasn1_modules-0.0.8-py2.py3-none-any.whl  Collecting attrs (from service-identity->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading attrs-16.3.0-py2.py3-none-any.whl  Collecting cryptography>=1.3.4 (from pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading cryptography-1.7.1.tar.gz (420kB)  Collecting zope.interface>=3.6.0 (from Twisted>=10.0.0->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading zope.interface-4.3.3.tar.gz (150kB)  Collecting constantly>=15.1 (from Twisted>=10.0.0->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading constantly-15.1.0-py2.py3-none-any.whl  Collecting incremental>=16.10.1 (from Twisted>=10.0.0->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading incremental-16.10.1-py2.py3-none-any.whl  Collecting idna>=2.0 (from cryptography>=1.3.4->pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading idna-2.2-py2.py3-none-any.whl (55kB)  Collecting enum34 (from cryptography>=1.3.4->pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading enum34-1.1.6-py2-none-any.whl  Collecting ipaddress (from cryptography>=1.3.4->pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading ipaddress-1.0.18-py2-none-any.whl  Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading cffi-1.9.1-cp27-cp27m-manylinux1_x86_64.whl (389kB)  Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyOpenSSL->Scrapy==1.0.3->-r requirements.txt (line 2))  Downloading pycparser-2.17.tar.gz (231kB)  Complete output from command python setup.py egg_info:  running egg_info  creating pip-egg-info/pycparser.egg-info  error: could not create 'pip-egg-info/pycparser.egg-info': No space left on device  ----------------------------------------  Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-MgivCQ/pycparser/

Data

Downloaded 213 times by wluxion koooooooooo hubrian astaunton ReagentX baabelfish amp530 Dierme MikeRalphson neoantox VIGGEEN Champ586 4ER0 Tom-Lo miedzinski iTsFILIPOficial BringMaster MatejVitek AlNar skramz1990 mariovrocha kaiserl94 vmzubarev larion93 sdieunidou Viiprogrammer denolfe Uhlvar KronKraken Technoblazed deskel dumdum85 TrueCarry

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (969 KB) Use the API

rows 10 / 2265

id hltv_id nickname
1
5185
DomiNate
2
3329
quix
3
8555
jayzaR
4
9034
dejvin
5
9261
Lekr0
6
7238
stonde
7
5479
suNny
8
7775
juho
9
7235
disturbed
10
7537
xartE

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (969 KB) Use the API

rows 10 / 1277

id name
1
Publiclir.se
2
MIXCAT
3
DenDD
4
Orgless
5
Space Soldiers
6
stfuNerd
7
EnVyUs
8
dignitas
9
Arcade
10
ESC

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (969 KB) Use the API

rows 10 / 6385

player_id team_id
5
1
1
1
2
1
3
1
4
1
6
2
7
2
8
2
9
2
10
2

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (969 KB) Use the API

rows 10 / 6290

hltv_id date tournament best_of score team1_id team2_id
2292323
2014-08-12
SLTV StarSeries X
1
1:0
939
448
2292324
2014-08-12
SLTV StarSeries X
1
1:0
939
799
2292325
2014-08-12
SLTV StarSeries X
1
0:1
121
10
2292327
2014-08-12
SLTV StarSeries X
1
0:0
754
23
2292328
2014-08-12
European Championship 2014 Qualification Round 1
3
0:2
168
247
2292335
2014-08-14
ESL One Cologne 2014
1
0:1
15
944
2292336
2014-08-14
ESL One Cologne 2014
1
0:1
60
754
2292337
2014-08-14
ESL One Cologne 2014
1
0:1
766
57
2292338
2014-08-14
ESL One Cologne 2014
1
1:0
612
11
2292339
2014-08-14
ESL One Cologne 2014
1
0:1
30
748

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (969 KB) Use the API

rows 10 / 12073

id match name score
1
2298986
cache
8:16 (6:9; 2:7)
2
2298986
mirage
13:16 (8:7; 5:9)
3
2298986
cobblestone
16:11 (7:8; 9:3)
4
2298986
inferno
6:16 (4:11; 2:5)
5
2298986
train
6
2298933
mirage
16:10 (7:8; 9:2)
7
2298933
cobblestone
7:16 (6:9; 1:7)
8
2298933
cache
16:4 (14:1; 2:3)
9
2298943
cache
16:3 (12:3; 4:0)
10
2298943
train
16:5 (12:3; 4:2)

Statistics

Average successful run time: about 6 hours

Total run time: about 6 hours

Total cpu time used: 15 minutes

Total disk space used: 1010 KB

History

  • Manually ran revision c176fe24 and failed .
    nothing changed in the database
  • Manually ran revision 9922e987 and completed successfully .
    28290 records added in the database
    37702 pages scraped
  • Created on morph.io

Scraper code

Python

hltv / scraper.py