davidk01 / hacker_news_front_page

Posts on the front page of HN


Testing out morph.io with a simple scraper for front page of HN.

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.5.0 -----> Installing dependencies using bundler version 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/......  Fetching version metadata from https://rubygems.org/.  Fetching public_suffix 4.0.7  Using bundler 1.15.2  Fetching coderay 1.1.2  Fetching connection_pool 2.2.5  Installing connection_pool 2.2.5  Installing coderay 1.1.2  Installing public_suffix 4.0.7  Fetching unf_ext 0.0.8.2  Installing unf_ext 0.0.8.2 with native extensions  Fetching httpclient 2.8.3  Installing httpclient 2.8.3  Fetching mime-types-data 3.2022.0105  Installing mime-types-data 3.2022.0105  Fetching net-http-digest_auth 1.4.1  Fetching mini_portile2 2.6.1  Installing mini_portile2 2.6.1  Fetching racc 1.6.0  Installing net-http-digest_auth 1.4.1  Installing racc 1.6.0 with native extensions  Fetching rubyntlm 0.6.3  Installing rubyntlm 0.6.3  Fetching webrick 1.7.0  Installing webrick 1.7.0  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Fetching method_source 0.9.0  Installing method_source 0.9.0  Fetching sqlite3 1.3.13  Installing sqlite3 1.3.13 with native extensions  Fetching net-http-persistent 4.0.1  Installing net-http-persistent 4.0.1  Fetching addressable 2.8.0  Installing addressable 2.8.0  Fetching mime-types 3.4.1  Installing mime-types 3.4.1  Fetching pry 0.11.3  Installing pry 0.11.3  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20190701  Installing domain_name 0.5.20190701  Fetching http-cookie 1.0.5  Installing http-cookie 1.0.5  Fetching nokogiri 1.12.5 (x86_64-linux)  Installing nokogiri 1.12.5 (x86_64-linux)  Fetching mechanize 2.8.5  Installing mechanize 2.8.5  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Fetching scraperwiki 3.0.2  Installing scraperwiki 3.0.2  Bundle complete! 4 Gemfile dependencies, 26 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (14.71s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Found some comments: Potential Issues of Using a USB Powerbank as a UPS (2021), 2 Creating DB item Found some comments: UML: My Part in Its Downfall, 10 Creating DB item Found some comments: We Burned Down Players’ Houses in Ultima Online, 149 Creating DB item Found some comments: Testing a Cheap ExpressCard to NVMe SSD Adapter on My ThinkPad T430, 36 Creating DB item Found some comments: Codebase as Database: Turning the IDE Inside Out with Datalog, 16 Creating DB item Found some comments: Ask HN: So you moved off Heroku, where did you go?, 59 Creating DB item Found some comments: The HTML5 Shiv (2011), 6 Creating DB item Found some comments: The 'attention economy' corrupts science, 108 Creating DB item Found some comments: The Fundamentals of Control Theory, 67 Creating DB item Found some comments: Junior Designers vs. Senior Designers, 9 Creating DB item Found some comments: VLC-developer VideoLAN sends legal notice to Indian ministries over ban, 4 Creating DB item Found some comments: DNS Providers as Piracy Fighters?, 10 Creating DB item Found some comments: Google “Pepper”: When a Brand Becomes More Popular Than the Meaning of the Word, 17 Creating DB item Found some comments: From Burned Out Tech CEO to Amazon Warehouse Associate, 269 Creating DB item Found some comments: DVD Drives Turned into Microscopes, 17 Creating DB item Found some comments: Hckr news – Hacker News sorted by time, 46 Creating DB item Found some comments: Paul McCartney's Freakish Memory, 115 Creating DB item Found some comments: VideoLAN has issued a legal notice to Indian Gov for banning their website, 6 Creating DB item Found some comments: Xerox PARC – Office Alto Commercial (1979) [video], 24 Creating DB item Found some comments: 'A fluke': The oldest webcam is still watching over San Francisco, 36 Creating DB item Found some comments: The technology behind Bella Hadid’s spray-on dress, 135 Creating DB item Found some comments: Raft Is So Fetch: The Raft Consensus Algorithm Explained Through Mean Girls, 50 Creating DB item Found some comments: ZFS 2.1.6 Released, 41 Creating DB item No comments yet for this entry: Summary: Contagious by Jonah Berger Creating DB item Found some comments: PS5 4.03 Kernel Exploit, 19 Creating DB item Found some comments: Postgres WASM, 157 Creating DB item Found some comments: A List of Hacker News's Undocumented Features and Behaviors, 27 Creating DB item Found some comments: The City of London’s oldest document is currently on public display, 36 Creating DB item No poits. Assuming job post: Meticulous (YC S21) Is Hiring #3 Founding Engineer in London Found some comments: Ask HN: How to find what I am really good at?, 2 Creating DB item Found some comments: Signals in Prod: Dangers and Pitfalls, 13 Creating DB item Found some comments: The Eleventh Commandment (2004), 2 Creating DB item Found some comments: Interior AI autofills photo of room with CG furniture/art, 18 Creating DB item Found some comments: What Can’t the Internet Handle in 2022? Apostrophes, 51 Creating DB item Found some comments: Researchers successfully convert methane gas into liquid methanol, 39 Creating DB item Found some comments: Show HN: I finished v5 of a JVM framework I've spent spent half a decade making, 120 Creating DB item Found some comments: “SNL stole my video”, 14 Creating DB item Found some comments: Vintage photos of the Romanovs at the 1903 Costume Ball, 8 Creating DB item Found some comments: Goal: Pass all 4259065 tests in sqllogictest in 1 week, 80 Creating DB item Found some comments: You can't buy a Raspberry Pi right now, 218 Creating DB item No comments yet for this entry: The Onion defends right to parody in Supreme Court brief Creating DB item No comments yet for this entry: Jaywalking is decriminalized in California under new law Creating DB item Found some comments: Brief of the Onion as Amicus Curiae in Support of Petitioner [pdf], 5 Creating DB item Found some comments: Mapping Wikipedia with BERT and UMAP, 1 Creating DB item Found some comments: Companies are paying huge sums to show their ads to bots, 318 Creating DB item Found some comments: Processing 40 TB of code from ~10M projects with a server and Go for $100 (2019), 13 Creating DB item Found some comments: Ask HN: Who is hiring? (October 2022), 379 Creating DB item Found some comments: Bitbucket Is Down, 8 Creating DB item Found some comments: “Creative” ways to use a $30 Wyze camera, 4 Creating DB item Found some comments: Tell HN: A disabled 40-year-old person founded a startup and makes a living, 147 Creating DB item Found some comments: Google’s “million’s of search results” are not being served, 118 Creating DB item Found some comments: Have lots of AWS accounts, 148 Creating DB item Found some comments: AITemplate, a revolutionary new inference engine by Meta AI, 18 Creating DB item Found some comments: High Efficiency Image File Format (HEIF/.heic), 60 Creating DB item Found some comments: Apple’s ad business set to boom on the back of its own anti-tracking crackdown, 588 Creating DB item Found some comments: Podcast on (Massive) Ad-Fraud, 17 Creating DB item No comments yet for this entry: Meta to terminate lease of 225 Park Ave in NYC Creating DB item Found some comments: Post-pandemic, the battle between Uber and Lyft is looking more one-sided, 160 Creating DB item Found some comments: For personal reasons, I do not browse the web from my computer. (2007), 4 Creating DB item Found some comments: I need structure in my life, 222 Creating DB item Found some comments: Highlights from Git 2.38, 37 Creating DB item Found some comments: Magma, a new storage engine for Couchbase, 12 Creating DB item Found some comments: Ask HN: Books for Reevaluation of Life, 6 Creating DB item Found some comments: The Schizoid Difference, 130 Creating DB item Found some comments: Cool HTML elements nobody uses, 102 Creating DB item Found some comments: Shrinking Mississippi River Puts American Farm Trade at Risks, 92 Creating DB item Found some comments: Rust Support Merged into Linux Kernel, 1 Creating DB item Found some comments: Ask HN: Who wants to be hired? (October 2022), 201 Creating DB item No comments yet for this entry: Ask HN: How to get into top tech companies during this hiring slowdown? Creating DB item Found some comments: The Anthropology of Farming Simulator, 98 Creating DB item Found some comments: Maigret: Collect a dossier on a person by username from thousands of sites, 187 Creating DB item Found some comments: mod_wasm: Run WebAssembly with Apache, 46 Creating DB item Found some comments: How We Read Now, 24 Creating DB item Found some comments: ASML's Secret: A view from inside the global semiconductor giant [video], 28 Creating DB item Found some comments: Man 3D Prints 110 Guns, Sells Them to Buyback Program for $21,000, 10 Creating DB item Found some comments: Fandom Acquires GameSpot, Giant Bomb, GameFAQs, TV Guide and Metacritic, 204 Creating DB item Found some comments: The Supreme Court is finally taking up Section 230, 1 Creating DB item Found some comments: New DNA analysis shows how cats spread around the world (2016), 23 Creating DB item Found some comments: How big is the capital hole at Credit Suisse?, 3 Creating DB item Found some comments: The PS5 Has Been Jailbroken, 365 Creating DB item Found some comments: “Privacy”.com–Yeah Right, 152 Creating DB item Found some comments: Why aren’t you using pretrained models?, 50 Creating DB item Found some comments: Ask hn: Has the Internet Become a Burden?, 25 Creating DB item Found some comments: Fishermen nearly won a tournament. Then weights were found in the fish, 1 Creating DB item Found some comments: The software that I love, 3 Creating DB item Found some comments: Svante Pääbo awarded Nobel Prize for studies of extinct human ancestors, 74 Creating DB item Found some comments: Cloning a Rare ISA Card to Use a Rare CD Drive [video], 42 Creating DB item Found some comments: Modern JavaScript Explained for Dinosaurs (2017), 1 Creating DB item No comments yet for this entry: Study kills glioblastoma by manipulating brain's astrocytes Creating DB item Found some comments: Aristotle influenced the computer (2017), 15 Creating DB item Found some comments: The next generation of Materialize, 36 Creating DB item Found some comments: US Cities Are Recycling Trees and Poop to Make Compost, 2 Creating DB item Found some comments: Xonsh: I don't remember how to write a for loop in Bash, 66 Creating DB item Found some comments: How a razor blade can be damaged as it cuts human hair (2020), 126 Creating DB item Found some comments: IceWM 3.0.0 Release, 1 Creating DB item Found some comments: Bay Area Bed Frame Grants, 9 Creating DB item No comments yet for this entry: Pekko (Open Source Akka fork) has been submitted to Apache Incubator Creating DB item Found some comments: Liquid Death Water Startup Valued at $700M After Funding Round, 5 Creating DB item Found some comments: Pop_OS Cosmic Desktop to Make Use of Iced Rust Toolkit Rather Than GTK, 67 Creating DB item No comments yet for this entry: Recording the Grateful Dead: The Culture of Tapers Creating DB item Found some comments: Researchers develop new rocket-propulsion system (2020), 12 Creating DB item Found some comments: Tell HN: Keep Your Phone on a Leash, 10 Creating DB item Found some comments: Matrix-webcam displays your webcam video feed in the console, 43 Creating DB item Found some comments: First make the change easy, then make the easy change (2021), 82 Creating DB item Found some comments: Google sued for violating MN law requiring video rental records to be destroyed, 1 Creating DB item Found some comments: Deepcake – Al solutions for video production, 20 Creating DB item Found some comments: New Pegasus spyware abuses identified in Mexico, 30 Creating DB item Found some comments: The lost art of handheld demakes, 14 Creating DB item Found some comments: Fake Men on Hinge, 153 Creating DB item Found some comments: Walgreens Turns to Prescription-Filling Robots to Free Up Pharmacists, 62 Creating DB item Found some comments: UK power station owner cuts down primary forests in Canada, 137 Creating DB item Found some comments: Autistic children with imaginary friends have better social skills, 50 Creating DB item Found some comments: The Chaos (1922), 25 Creating DB item Found some comments: Show HN: Dead Acquire – Sell your pre-revenue or dead side-project, 21 Creating DB item Found some comments: A study of lights at night suggests dictators lie about economic growth, 190 Creating DB item No comments yet for this entry: I add the SQLite amalgamation to my project, it adds Doom as a dependency Creating DB item Found some comments: Golden raises $40M Series B led by A16Z for decentralized knowledge protocol, 3 Creating DB item Found some comments: Toyota CEO talks about why he isn’t all-in on EVs, 981 Creating DB item Found some comments: How a Stable Diffusion prompt changes its output for the style of 1500 artists, 183 Creating DB item No comments yet for this entry: Wasoko and Zanzibar partner to launch an innovation hub Creating DB item Adding 119 items to db

Data

Downloaded 5 times by dingdreher MikeRalphson simondodson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (39.8 MB) Use the API

rows 10 / 162546

title url points author comments
Sorry to burst your bubble
123
prostoalex
57
The Gluglug – GNU/Linux laptop with libreboot firmware
118
weinzierl
52
As Beijing Becomes a Supercity, the Rapid Growth Brings Pains
12
doppp
1
Code Golf at Google
30
antichaos
18
De-duplicating Hacker News
28
shishir456
16
Purism – Privacy-Respecting Laptops
140
monort
86
Show HN: FAQT – A lightweight, personal knowledge base
116
siavosh
78
Currying in JavaScript
46
kevincennis
35
A mixed-format, mixed-architecture, mixed-sided diskette (2008)
105
bane
18
VnStat – A network traffic monitor for Linux and BSD
36
merah
6

Statistics

Average successful run time: 2 minutes

Total run time: 3 days

Total cpu time used: 39 minutes

Total disk space used: 39.9 MB

History

  • Auto ran revision 824a5451 and completed successfully .
    119 records added, 9 records removed in the database
  • Auto ran revision 824a5451 and completed successfully .
    119 records added, 77 records removed in the database
  • Auto ran revision 824a5451 and completed successfully .
    119 records added, 21 records removed in the database
  • Auto ran revision 824a5451 and completed successfully .
    119 records added, 13 records removed in the database
  • Auto ran revision 824a5451 and completed successfully .
    119 records added, 48 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

hacker_news_front_page / scraper.rb