This scraper grabs all the crappy spam postings from the Washington DC craigslist that are NOT actually located within the District proper.
This includes Silver Spring (which should be in “Southern Maryland” listings) and Arlington (which should be in the NoVa) listings) and creates the respective “Spam” flagging URLs then ‘clicks’ them. Thereby automatically bulk-flagging them as spam posts.

To Do:
- automate the flagging gesture (perhaps another HTTP requrest to the flagging URL?)

Forked from ScraperWiki

Contributors MichaelPhilippone

Last run failed with status code 1.

Console output of last run

Traceback (most recent call last): File "/repo/scraper.py", line 87, in <module> scrapes = scraperwiki.sqlite.select("* from 'swdata'") File "/usr/local/lib/python2.7/dist-packages/scraperwiki-0.3.7-py2.7.egg/scraperwiki/sqlite.py", line 41, in select result = dt.execute(sqlquery, data, commit = False) File "/usr/local/lib/python2.7/dist-packages/dumptruck-0.1.5-py2.7.egg/dumptruck/dumptruck.py", line 136, in execute self.cursor.execute(sql, *args) sqlite3 . OperationalError : no such table: swdata

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (123 KB) Use the API

rows 10 / 368

date_scraped title link location flag_url id
2013-02-14T05:28:37.955339
Studio Rosslyn,River Place East
Rosslyn Metro,Arlington
3575630121
2013-02-14T05:28:37.955339
Nearby to Sligo Creek Parkway, a short distance to Silver Spring Metro
Silver Spring
3616011393
2013-02-14T05:28:37.955339
Beautiful Single House
Silver Spring MD 20904White Oak area
3615942530
2013-02-14T05:28:37.955339
1 BR APT 10 MIN FROM COURTHOUSE METRO FREE STAY AND FURNITURE!!!
ARLINGTON
3615900124
2013-02-14T05:28:37.955339
Professional moving service call 571 501 4773 ...
VA, DC, MD
3591224546
2013-02-14T05:28:37.955339
Professional moving service call 571 501 4773 ...
MARYLAND,DC,VA
3596540890
2013-02-14T05:28:37.955339
Basement Room -Utilities inclu
McLean VA
3615849690
2013-02-20T12:11:01.351174
2 Bed inside noma area with upgrades
DC, NOMA
3606349772
2013-02-20T12:11:01.351174
1 Bedroom Apartment
Bethesda, MD
3630103111
2013-02-20T12:11:01.351174
This Apartment Outshines Them All!
Atlas, NoMa, Gallaudet
3629953726

Statistics

Total run time: less than 5 seconds

Total cpu time used: less than 5 seconds

Total disk space used: 144 KB

History

  • Manually ran revision 2114e85c and failed .
  • Forked from ScraperWiki

Scraper code

Python

MP_craigslist_flagger / scraper.py