free flats in berlin

Forked from ScraperWiki

Contributors DoomyTheFroomy

The scraper is running. It was queued automatically .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.6, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.6 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 2))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:380: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  SNIMissingWarning,  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:139: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  InsecurePlatformWarning,  Collecting BeautifulSoup==3.2.0  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:139: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  InsecurePlatformWarning,  Downloading BeautifulSoup-3.2.0.tar.gz (31 kB)  Collecting Creoleparser==0.7.4  Downloading Creoleparser-0.7.4.zip (31 kB)  Collecting Genshi==0.6  Downloading Genshi-0.6.tar.gz (433 kB)  Collecting Jinja2==2.6  Downloading Jinja2-2.6.tar.gz (389 kB)  Collecting Markdown==2.2.0  Downloading Markdown-2.2.0.tar.gz (236 kB)  Collecting Pygments==1.4  Downloading Pygments-1.4.tar.gz (3.5 MB)  Collecting SQLAlchemy==0.6.6  Downloading SQLAlchemy-0.6.6.tar.gz (2.1 MB)  Collecting Twisted==11.1.0  Downloading Twisted-11.1.0.tar.bz2 (2.8 MB)  Collecting Unidecode==0.04.9  Downloading Unidecode-0.04.9.tar.gz (196 kB)  Collecting anyjson==0.3.3  Downloading anyjson-0.3.3.tar.gz (8.3 kB)  Collecting argparse==1.2.1  Downloading argparse-1.2.1.tar.gz (69 kB)  Collecting beautifulsoup4==4.1.3  Downloading beautifulsoup4-4.1.3.tar.gz (58 kB)  Collecting bitlyapi==0.1.1  Downloading bitlyapi-0.1.1.tar.gz (4.3 kB)  Collecting blinker==1.2  Downloading blinker-1.2.tar.gz (66 kB)  Collecting cartodb==0.6  Downloading cartodb-0.6.tar.gz (3.6 kB)  Collecting certifi==0.0.8  Downloading certifi-0.0.8.tar.gz (118 kB)  Collecting chardet==2.1.1  Downloading chardet-2.1.1.tar.gz (178 kB)  Collecting ckanclient==0.10  Downloading ckanclient-0.10.tar.gz (16 kB)  Collecting colormath==1.0.8  Downloading colormath-1.0.8.tar.gz (24 kB)  Collecting csvkit==0.3.0  Downloading csvkit-0.3.0.tar.gz (23 kB)  Collecting dataset==0.5.2  Downloading dataset-0.5.2.tar.gz (14 kB)  Collecting demjson==1.6  Downloading demjson-1.6.tar.gz (64 kB)  Collecting dropbox==1.4  Downloading dropbox-1.4.tar.gz (33 kB)  Collecting errorhandler==1.1.1  Downloading errorhandler-1.1.1.tar.gz (8.3 kB)  Collecting feedparser==5.0.1  Downloading feedparser-5.0.1.tar.bz2 (204 kB)  Collecting fluidinfo.py==1.1.2  Downloading fluidinfo.py-1.1.2.tar.gz (4.7 kB)  Collecting gdata==2.0.15  Downloading gdata-2.0.15.tar.gz (2.0 MB)  Collecting geopy==0.94.1  Downloading geopy-0.94.1.tar.gz (27 kB)  Collecting gevent==0.13.6  Downloading gevent-0.13.6.tar.gz (289 kB)  Collecting google-api-python-client==1.0beta8  Downloading google-api-python-client-1.0beta8.tar.gz (348 kB)  Collecting googlemaps==1.0.2  Downloading googlemaps-1.0.2.tar.gz (60 kB)  Collecting greenlet==0.3.2  Downloading greenlet-0.3.2.zip (50 kB)  Collecting html5lib==0.90  Downloading html5lib-0.90.tar.gz (86 kB)  Collecting httplib2==0.7.4  Downloading httplib2-0.7.4.tar.gz (106 kB)  Collecting imposm.parser==1.0.3  Downloading imposm.parser-1.0.3.tar.gz (31 kB)  Collecting jellyfish==0.2.0  Downloading jellyfish-0.2.0.tar.gz (14 kB)  Collecting mechanize==0.2.5  Downloading mechanize-0.2.5.tar.gz (383 kB)  Collecting mock==0.7.2  Downloading mock-0.7.2.tar.gz (896 kB)  Collecting networkx==1.6  Downloading networkx-1.6.tar.gz (707 kB)  Collecting ngram==3.3.0  Downloading ngram-3.3.0.tar.gz (8.8 kB)  Collecting nose==1.1.2  Downloading nose-1.1.2.tar.gz (729 kB)  Collecting oauth2==1.5.170  Downloading oauth2-1.5.170.tar.gz (18 kB)  Collecting oauth==1.0.1  Downloading oauth-1.0.1.tar.gz (9.5 kB)  Collecting oauthlib==0.1.2  Downloading oauthlib-0.1.2.tar.gz (24 kB)  Collecting openpyxl==1.5.7  Downloading openpyxl-1.5.7.tar.gz (67 kB)  Collecting ordereddict==1.1  Downloading ordereddict-1.1.tar.gz (2.1 kB)  Collecting pbkdf2==1.3  Downloading pbkdf2-1.3.tar.gz (6.4 kB)  Collecting pdfminer==20110515  Downloading pdfminer-20110515.tar.gz (4.1 MB)  Collecting pexpect==2.4  Downloading pexpect-2.4.tar.gz (113 kB)  Collecting pipe2py==0.9.2  Downloading pipe2py-0.9.2.tar.gz (57 kB)  Collecting pyOpenSSL==0.13  Downloading pyOpenSSL-0.13.tar.gz (250 kB)  Collecting pycrypto==2.5  Downloading pycrypto-2.5.tar.gz (426 kB)  Collecting pycurl==7.19.0  Downloading pycurl-7.19.0.tar.gz (70 kB)  Collecting pyephem==3.7.5.1  Downloading pyephem-3.7.5.1.tar.gz (703 kB)  Collecting pyparsing==1.5.6  Downloading pyparsing-1.5.6.tar.gz (1.4 MB)  Collecting pyth==0.5.6  Downloading pyth-0.5.6.tar.gz (24 kB)  Collecting python-Levenshtein==0.10.2  Downloading python-Levenshtein-0.10.2.tar.gz (45 kB)  Collecting python-dateutil==1.5  Downloading python-dateutil-1.5.tar.gz (233 kB)  Collecting python-gflags==2.0  Downloading python-gflags-2.0.tar.gz (65 kB)  Collecting python-modargs==1.2  Downloading python-modargs-1.2.tar.gz (6.3 kB)  Collecting python-stdnum==0.7  Downloading python-stdnum-0.7.tar.gz (113 kB)  Collecting pytz==2011k  Downloading pytz-2011k.tar.bz2 (166 kB)  Collecting rdflib==3.1.0  Downloading rdflib-3.1.0.tar.gz (249 kB)  Collecting requests-foauth==0.1.1  Downloading requests-foauth-0.1.1.tar.gz (1.9 kB)  Collecting requests==1.0.4  Downloading requests-1.0.4.tar.gz (336 kB)  Collecting selenium==2.5.0  Downloading selenium-2.5.0.tar.gz (2.4 MB)  Collecting simplejson==2.2.1  Downloading simplejson-2.2.1.tar.gz (49 kB)  Collecting suds==0.4  Downloading suds-0.4.tar.gz (104 kB)  Collecting tweepy==1.7.1  Downloading tweepy-1.7.1.tar.gz (19 kB)  Collecting tweetstream==1.1.1  Downloading tweetstream-1.1.1.tar.gz (6.9 kB)  Collecting w3lib==1.0  Downloading w3lib-1.0.tar.gz (6.1 kB)  Collecting xlrd==0.7.1  Downloading xlrd-0.7.1.tar.gz (118 kB)  Collecting xlutils==1.4.1  Downloading xlutils-1.4.1.tar.gz (40 kB)  Collecting xlwt==0.7.2  Downloading xlwt-0.7.2.tar.gz (114 kB)  Collecting xmltodict==0.4  Downloading xmltodict-0.4.tar.gz (3.3 kB)  Collecting zope.interface==3.8.0  Downloading zope.interface-3.8.0.tar.gz (111 kB)  Collecting lxml==2.3.3  Downloading lxml-2.3.3.tar.gz (3.1 MB)  Collecting chromium-compact-language-detector==0.031415  Downloading chromium_compact_language_detector-0.031415.tar.gz (2.2 MB)  Collecting icalendar==3.0.1b1  Downloading icalendar-3.0.1b1.tar.gz (39 kB)  Collecting pyquery==1.0  Downloading pyquery-1.0.tar.gz (17 kB)  Collecting scrapely==0.9  Downloading scrapely-0.9.tar.gz (24 kB)  Collecting Fom==0.9.8  Downloading Fom-0.9.8.tar.gz (70 kB)  Collecting PyYAML==3.10  Downloading PyYAML-3.10.tar.gz (241 kB)  Collecting Scrapy==0.14.1  Downloading Scrapy-0.14.1.tar.gz (719 kB)  Collecting adspygoogle.adwords==15.6.2  Downloading adspygoogle.adwords-15.6.2.tar.gz (166 kB)  Collecting nltk==3.0.2  Downloading nltk-3.0.2.tar.gz (991 kB)  Collecting pydot==1.0.2  Downloading pydot-1.0.2.tar.gz (18 kB)  Collecting M2Crypto==0.22.3  Downloading M2Crypto-0.22.3.tar.gz (74 kB)  Collecting dumptruck>=0.1.2  Downloading dumptruck-0.1.6.tar.gz (15 kB)  Collecting alembic>=0.6.2  Downloading alembic-1.4.2.tar.gz (1.1 MB)  Installing build dependencies: started  Installing build dependencies: finished with status 'done'  Getting requirements to build wheel: started  Getting requirements to build wheel: finished with status 'done'  Preparing wheel metadata: started  Preparing wheel metadata: finished with status 'done'  Collecting python-slugify>=0.0.6  Downloading python-slugify-4.0.0.tar.gz (8.8 kB)  Collecting distribute  Downloading distribute-0.7.3.zip (145 kB)  Collecting numpy  Downloading numpy-1.16.6-cp27-cp27m-manylinux1_x86_64.whl (17.0 MB)  Collecting Mako  Downloading Mako-1.1.2-py2.py3-none-any.whl (75 kB)  Collecting python-editor>=0.3  Downloading python_editor-1.0.4-py2-none-any.whl (4.9 kB)  Collecting text-unidecode>=1.3  Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)  Collecting MarkupSafe>=0.9.2  Downloading MarkupSafe-1.1.1-cp27-cp27m-manylinux1_x86_64.whl (24 kB)  Building wheels for collected packages: BeautifulSoup, Creoleparser, Genshi, Jinja2, Markdown, Pygments, SQLAlchemy, Twisted, Unidecode, anyjson, argparse, beautifulsoup4, bitlyapi, blinker, cartodb, certifi, chardet, ckanclient, colormath, csvkit, dataset, demjson, dropbox, errorhandler, feedparser, fluidinfo.py, gdata, geopy, gevent, google-api-python-client, googlemaps, greenlet, html5lib, httplib2, imposm.parser, jellyfish, mechanize, mock, networkx, ngram, nose, oauth2, oauth, oauthlib, openpyxl, ordereddict, pbkdf2, pdfminer, pexpect, pipe2py, pyOpenSSL, pycrypto, pycurl, pyephem, pyparsing, pyth, python-Levenshtein, python-dateutil, python-gflags, python-modargs, python-stdnum, pytz, rdflib, requests-foauth, requests, selenium, simplejson, suds, tweepy, tweetstream, w3lib, xlrd, xlutils, xlwt, xmltodict, zope.interface, lxml, chromium-compact-language-detector, icalendar, pyquery, scrapely, Fom, PyYAML, Scrapy, adspygoogle.adwords, nltk, pydot, M2Crypto, dumptruck, alembic, python-slugify, distribute  Building wheel for BeautifulSoup (setup.py): started  Building wheel for BeautifulSoup (setup.py): finished with status 'done'  Created wheel for BeautifulSoup: filename=BeautifulSoup-3.2.0-py2-none-any.whl size=31888 sha256=0aefa75bdd7c5307b31f04caef0cacfe76a9ea40031abd12e42c258c45fd0da2  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/07/ef/84/b06e7bf99ec3061d9a1d31ddfae45eee089c0cc6cb7741578b  Building wheel for Creoleparser (setup.py): started  Building wheel for Creoleparser (setup.py): finished with status 'done'  Created wheel for Creoleparser: filename=Creoleparser-0.7.4-py2-none-any.whl size=25838 sha256=3a7e15f0fee9977d8fc9d7290cf56fd4b6f3847a8ce07690849500aa51e9a0c8  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/df/a3/94/65ae363824b20787c55936be6c69ad0cfdb9713b4c2859f72a  Building wheel for Genshi (setup.py): started  Building wheel for Genshi (setup.py): finished with status 'done'  Created wheel for Genshi: filename=Genshi-0.6-py2-none-any.whl size=110612 sha256=f813396a0a82d3d4c79170f67f5e2ccba250a931d5639dec069b50e7ea87a7cb  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/5a/df/5b/f27840fc4b76eb60cd64043d2caf894b78656f11b3d016a856  Building wheel for Jinja2 (setup.py): started  Building wheel for Jinja2 (setup.py): finished with status 'done'  Created wheel for Jinja2: filename=Jinja2-2.6-py2-none-any.whl size=296733 sha256=7c180e827b1b3dc10544b6c9fd35cbafe5ff9d86ed59269e8ab696da6ac57f7a  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/02/79/ba/2a51fda3f97bbf2b6fe2d4b6a39cc66fdffe6267fbfd6b9e47  Building wheel for Markdown (setup.py): started  Building wheel for Markdown (setup.py): finished with status 'done'  Created wheel for Markdown: filename=Markdown-2.2.0-py2-none-any.whl size=135305 sha256=3d8adc32cbc0cac1ab138acf9fd7f220c0c8ddce52909e214efa9a7bbb2a8bdd  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/71/d6/5a/3cc9d2ddca0265fe6c6756c843e1f758df9c8757e1c5acfbcf  Building wheel for Pygments (setup.py): started  Building wheel for Pygments (setup.py): finished with status 'done'  Created wheel for Pygments: filename=Pygments-1.4-py2-none-any.whl size=452158 sha256=0c2df4465d794a2b5e0aa7089fcc4d7625043a0a12041708f62239c5fb08dd11  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/f2/91/89/5fd94f8626f0770fa40062916c9e86070434df930c192820d6  Building wheel for SQLAlchemy (setup.py): started  Building wheel for SQLAlchemy (setup.py): finished with status 'done'  Created wheel for SQLAlchemy: filename=SQLAlchemy-0.6.6-py2-none-any.whl size=584975 sha256=0f4bb4bf42dd1295b119ba54bd2d885216da73e5fb3526898fd0db8e56bde5b2  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/63/12/05/8f283ba6d09081fc8ca62da679df7d78767c93e9bb19048a61  Building wheel for Twisted (setup.py): started  Building wheel for Twisted (setup.py): finished with status 'done'  Created wheel for Twisted: filename=Twisted-11.1.0-cp27-cp27m-linux_x86_64.whl size=2716996 sha256=257b638a389052aa9cb7d6a47fe10508e66a16a4e1cea833cbd4e90c544cd59c  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/3c/de/16/f23444230f9439280f7b9b24125c0547b1eeddbe740490b826  Building wheel for Unidecode (setup.py): started  Building wheel for Unidecode (setup.py): finished with status 'done'  Created wheel for Unidecode: filename=Unidecode-0.4.9-py2-none-any.whl size=226425 sha256=8509a4e2197ca923a8d292316a1893bf3bd9ac340f6906fd4f2306c94b987827  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/ec/ea/2e/a63d73696df0572b0fca54fed83ba25ad62fd77867af43dd5c  Building wheel for anyjson (setup.py): started  Building wheel for anyjson (setup.py): finished with status 'done'  Created wheel for anyjson: filename=anyjson-0.3.3-py2-none-any.whl size=4952 sha256=1dfb32d7710d2a68badf2d4f46eb4c8f655070220f3afc9e7281eb1d5ab79f6f  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/51/e4/7b/e4ca0dd6d60916a0a20d94ad352a671675ed88f83a57830924  Building wheel for argparse (setup.py): started  Building wheel for argparse (setup.py): finished with status 'done'  Created wheel for argparse: filename=argparse-1.2.1-py2-none-any.whl size=21371 sha256=73cad48be805a6dd8e6bf279eb888588651b36f97d07fbc8df644c035312938f  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/9f/79/7e/e3f33e05010185975b7ade632c68bad70fcf71d1ff1adc1a02  Building wheel for beautifulsoup4 (setup.py): started  Building wheel for beautifulsoup4 (setup.py): finished with status 'done'  Created wheel for beautifulsoup4: filename=beautifulsoup4-4.1.3-py2-none-any.whl size=66446 sha256=e1e654a01095519d83d7cccf0e4a24fe890581ad0a38bd2001979080d643990c  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/d3/19/bb/b9d3a3ed20aa640d4424e824bf9664ca88da65578a8e0aa7dc  Building wheel for bitlyapi (setup.py): started  Building wheel for bitlyapi (setup.py): finished with status 'done'  Created wheel for bitlyapi: filename=bitlyapi-0.1.1-py2-none-any.whl size=5034 sha256=8a94727904b87ca8346e3d129ce2f1a47d02d47895035c9f22acf86ca4eb2097  Stored in directory: /tmp/pip-ephem-wheel-cache-BS94XE/wheels/34/46/3e/6aad460bb0a1eb7d676891ebc40da15e73b33141c087e1026b  Building wheel for blinker (setup.py): started

Data

Downloaded 1 time by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (18.2 MB) Use the API

rows 10 / 17373

Link id
Wolfshagener Straße 73 13187 Berlin Pankow
Kreuzstrasse 5 13187 Berlin Pankow
Mühlenstr. 74 13189 Berlin Pankow
Neumannstr. 20 13189 Berlin
Zillertalstraße 13187 Berlin Pankow
Mendelssohnstraße 3 10405 Berlin
Kollwitzstraße 28 10405 Berlin Prenzlauer Berg
Chrysanthemenstr. 8 10407 Berlin
Rodenbergstr. 35 10439 Berlin
Rodenbergstraße 14 10439 Berlin Prenzlauer Berg

Statistics

Average successful run time: half a minute

Total run time: 1 day

Total cpu time used: 14 minutes

Total disk space used: 18.2 MB

History

  • Auto ran revision 9e6fa6d2 and failed .
    nothing changed in the database
  • Auto ran revision 9e6fa6d2 and failed .
    nothing changed in the database
  • Auto ran revision 9e6fa6d2 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9e6fa6d2 and failed .
    nothing changed in the database
  • Auto ran revision 9e6fa6d2 and failed .
    nothing changed in the database
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

Python

streets_by_immo_search / scraper.py