mjr0fr4 / coffee-price

Get current coffee price from different sources

Scrapes www.federaciondecafeteros.org, www.banrep.gov.co, and www.ico.org

La Federaciónde Cafeteros actúa en diversos frentes para alcanzar su objetivo de elevar la calidad de vida de los cafeteros colombianos.

README.md

Gets and stores in morph.io coffee quotes from:

Federación Nacional de Cafeteros
International Coffee Organization
Banco de la República de Colombia

It gets the official colombian internal price for different cities, the colombian internal reference price, the quotes for the coffee commodity in the international markets and the exchange rate of the usdcop.

There are two files:

read_pdf.py
scraper.py

The first one - read_pdf.py - uses pdfMiner to read a pdf and get the text inside it. The second one - scraper.py - parses the text in the pdf and all the urls nedded to get the info.

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Executing at 2016-08-20 11:15:37 Retrieving pdf and getting text... Parsing raw text for reference price... Parsing raw text for cities price... Parsing raw text for pic price... Retrieving and parsing usdcop... Retrieving and parsing ico quotes...

Data

Downloaded 8 times by mjr0fr4 MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (52 KB) Use the API

rows 10 / 328

pamplona popayan ibague neiva armenia pereira buga santa_marta bucaramanga chinchina usdcop robustas suaves_colombianos naturales_brasileiros pasto otros_suaves ref manizales medellin date cucuta bogota valledupar
818000
820125
819125
818250
820750
819875
818375
821625
818750
819875
2943.97
89.27
156.45
135.77
818000
165.66
819500
819875
819125
2015-08-12
817875
820000
819250
790500
792625
791625
790750
793250
792375
790875
794125
791250
792375
2937.63
87.37
151.92
131.77
790500
160.98
792000
792375
791625
2015-08-13
790375
792500
791750
826000
828125
827125
826250
828750
827875
826375
829625
826750
827875
2937.63
87.37
151.92
131.77
826000
160.98
827500
827875
827125
2015-08-14
825875
828000
827250
834000
836125
835125
834250
836750
835875
834375
837625
834750
835875
2983.12
88.27
156.9
136.37
834000
165.88
835500
835875
835125
2015-08-15
833875
836000
835250
834000
836125
835125
834250
836750
835875
834375
837625
834750
835875
2983.12
88.27
156.9
136.37
834000
165.88
835500
835875
835125
2015-08-16
833875
836000
835250
834000
836125
835125
834250
836750
835875
834375
837625
834750
835875
2983.12
88.27
156.9
136.37
834000
165.88
835500
835875
835125
2015-08-17
833875
836000
835250
817625
819750
818750
817875
820375
819500
818000
821250
818375
819500
2983.12
87.97
154.7
135.07
817625
164.43
819125
819500
818750
2015-08-18
817500
819625
818875
826625
828750
827750
826875
829375
828500
827000
830250
827375
828500
3003.35
88.43
154.05
134.28
826625
164.71
828125
828500
827750
2015-08-19
826500
828625
827875
799000
801125
800125
799250
801750
800875
799375
802625
799750
800875
3027.20
86.49
150.21
130.68
799000
160.86
800500
800875
800125
2015-08-20
798875
801000
800250
799000
801125
800125
799250
801750
800875
799375
802625
799750
800875
3053.65
85.83
148.01
128.75
799000
158.21
800500
800875
800125
2015-08-21
798875
801000
800250

Statistics

Average successful run time: 1 minute

Total run time: about 13 hours

Total cpu time used: 16 minutes

Total disk space used: 82.6 KB

History

  • Auto ran revision 920cd03d and completed successfully .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision 920cd03d and completed successfully .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision 920cd03d and failed .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision 920cd03d and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 920cd03d and failed .
    nothing changed in the database
    2 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

coffee-price / scraper.py