gbballpack / public-proxy

Morph powered proxy scraper, forked from https://github.com/CookieMichal/us-proxy


public-proxy

This is a scraper that runs on Morph. To get started see the API

Proxy lists used

  • http://us-proxy.org/
  • http://sslproxies.org/
  • http://google-proxy.net/
  • http://free-proxy-list.net/
  • http://free-proxy-list.net/uk-proxy.html
  • http://free-proxy-list.net/anonymous-proxy.html

Among a continously changing list of proxy sources

Database format

| ip | port | type | code | country | anonymity | google | https | lastchecked | |-------------|---------|--------|---------|-----------------|-------------|---------------|---------------|------------------------------| | text | integer | text | text | text | text | text | text | text | | "123.5.1.4" | 8080 | "free" | "us" | "United States" | "anonymous" | "yes" or "no" | "yes" or "no" | "2016-03-29T17:34:54.814Z" |

Types of proxy servers

  • "free"
  • "ssl"
  • "us"
  • "uk"
  • "google"
  • "anonymous"
  • "github"

Types of proxy anonymity

  • Level 1 - Elite Proxy / Highly Anonymous Proxy: The web server can't detect whether you are using a proxy.
  • Level 2 - Anonymous Proxy: The web server can know you are using a proxy, but it can't know your real IP.
  • Level 3 - Transparent Proxy: The web server can know you are using a proxy and it can also know your real IP.

Contributors gbballpack

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): unspecified  engines.npm (package.json): unspecified (use default)   Resolving node version 6.x...  Downloading and installing node 6.17.1...  Using default npm version: 3.10.10  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@5.0.11 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   node-pre-gyp ERR! UNCAUGHT EXCEPTION  node-pre-gyp ERR! stack /tmp/build/node_modules/detect-libc/lib/detect-libc.js:76  node-pre-gyp ERR! stack const family = async () => {  node-pre-gyp ERR! stack ^  node-pre-gyp ERR! stack  node-pre-gyp ERR! stack SyntaxError: Unexpected token (  node-pre-gyp ERR! stack at createScript (vm.js:56:10)  node-pre-gyp ERR! stack at Object.runInThisContext (vm.js:97:10)  node-pre-gyp ERR! stack at Module._compile (module.js:549:28)  node-pre-gyp ERR! stack at Object.Module._extensions..js (module.js:586:10)  node-pre-gyp ERR! stack at Module.load (module.js:494:32)  node-pre-gyp ERR! stack at tryModuleLoad (module.js:453:12)  node-pre-gyp ERR! stack at Function.Module._load (module.js:445:3)  node-pre-gyp ERR! stack at Module.require (module.js:504:17)  node-pre-gyp ERR! stack at require (internal/module.js:20:19)  node-pre-gyp ERR! stack at Object.<anonymous> (/tmp/build/node_modules/@mapbox/node-pre-gyp/lib/util/versioning.js:8:21)  node-pre-gyp ERR! System Linux 5.12.2-x86_64-linode144  node-pre-gyp ERR! command "/tmp/build/.heroku/node/bin/node" "/tmp/build/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"  node-pre-gyp ERR! cwd /tmp/build/node_modules/sqlite3  node-pre-gyp ERR! node -v v6.17.1  node-pre-gyp ERR! node-pre-gyp -v v1.0.9  /tmp/build  +-- cheerio@1.0.0-rc.12  | +-- cheerio-select@2.1.0  | | +-- boolbase@1.0.0  | | +-- css-select@5.1.0  | | | `-- nth-check@2.1.1  | | +-- css-what@6.1.0  | | `-- domelementtype@2.3.0  | +-- dom-serializer@2.0.0  | | `-- entities@4.3.1  | +-- domhandler@5.0.3  | +-- domutils@3.0.1  | +-- htmlparser2@8.0.1  | +-- parse5@7.0.0  | `-- parse5-htmlparser2-tree-adapter@7.0.0  `-- request@2.88.2  +-- aws-sign2@0.7.0  +-- aws4@1.11.0  +-- caseless@0.12.0  +-- combined-stream@1.0.8  | `-- delayed-stream@1.0.0  +-- extend@3.0.2  +-- forever-agent@0.6.1  +-- form-data@2.3.3  | `-- asynckit@0.4.0  +-- har-validator@5.1.5  | +-- ajv@6.12.6  | | +-- fast-deep-equal@3.1.3  | | +-- fast-json-stable-stringify@2.1.0  | | +-- json-schema-traverse@0.4.1  | | `-- uri-js@4.4.1  | `-- har-schema@2.0.0  +-- http-signature@1.2.0  | +-- assert-plus@1.0.0  | +-- jsprim@1.4.2  | | +-- extsprintf@1.3.0  | | +-- json-schema@0.4.0  | | `-- verror@1.10.0  | | `-- core-util-is@1.0.2  | `-- sshpk@1.17.0  | +-- asn1@0.2.6  | +-- bcrypt-pbkdf@1.0.2  | +-- dashdash@1.14.1  | +-- ecc-jsbn@0.1.2  | +-- getpass@0.1.7  | +-- jsbn@0.1.1  | +-- safer-buffer@2.1.2  | `-- tweetnacl@0.14.5  +-- is-typedarray@1.0.0  +-- isstream@0.1.2  +-- json-stringify-safe@5.0.1  +-- mime-types@2.1.35  | `-- mime-db@1.52.0  +-- oauth-sign@0.9.0  +-- performance-now@2.1.0  +-- qs@6.5.3  +-- safe-buffer@5.2.1  +-- tough-cookie@2.5.0  | +-- psl@1.9.0  | `-- punycode@2.1.1  +-- tunnel-agent@0.6.0  `-- uuid@3.4.0   npm ERR! Linux 5.12.2-x86_64-linode144  npm ERR! argv "/tmp/build/.heroku/node/bin/node" "/tmp/build/.heroku/node/bin/npm" "install" "--unsafe-perm" "--userconfig" "/tmp/build/.npmrc"  npm ERR! node v6.17.1  npm ERR! npm v3.10.10  npm ERR! code ELIFECYCLE   npm ERR! sqlite3@5.0.11 install: `node-pre-gyp install --fallback-to-build`  npm ERR! Exit status 7  npm ERR!  npm ERR! Failed at the sqlite3@5.0.11 install script 'node-pre-gyp install --fallback-to-build'.  npm ERR! Make sure you have the latest version of node.js and npm installed.  npm ERR! If you do, this is most likely a problem with the sqlite3 package,  npm ERR! not with npm itself.  npm ERR! Tell the author that this fails on your system:  npm ERR! node-pre-gyp install --fallback-to-build  npm ERR! You can get information on how to open an issue for this project with:  npm ERR! npm bugs sqlite3  npm ERR! Or if that isn't available, you can get their info via:  npm ERR! npm owner ls sqlite3  npm ERR! There is likely additional logging output above.   npm ERR! Please include the following file with any support request:  npm ERR! /tmp/build/npm-debug.log  -----> Build failed   We're sorry this build is failing! You can troubleshoot common issues here:  https://devcenter.heroku.com/articles/troubleshooting-node-deploys   Some possible problems:   - Node version not specified in package.json  https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version   Love,  Heroku 

Data

Downloaded 5 times by boriszubenko isithebe serco18 yauz3

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (5.54 MB) Use the API

rows 10 / 46752

ip port type code country anonymity google https lastchecked
173.192.128.238
25
us
US
United States
elite proxy
no
no
2020-09-22T07:19:47.424Z
216.169.73.65
34679
us
US
United States
elite proxy
no
2021-08-16T04:11:10.370Z
24.172.82.94
53281
us
US
United States
elite proxy
no
2021-08-16T02:41:10.382Z
198.50.152.64
23500
us
US
United States
elite proxy
no
no
2021-05-16T05:25:50.075Z
162.243.108.129
8080
us
US
United States
anonymous
no
no
2020-11-14T12:42:29.462Z
198.199.120.102
8080
us
US
United States
anonymous
no
no
2020-11-14T12:42:29.464Z
102.129.249.120
8080
free
ZA
South Africa
anonymous
no
no
2021-08-17T02:22:18.502Z
159.8.114.37
25
free
FR
France
transparent
no
no
2021-07-05T17:49:07.126Z
80.48.119.28
8080
google
PL
Poland
elite proxy
yes
no
2021-08-13T14:01:33.386Z
207.154.231.213
8080
anonymous
DE
Germany
anonymous
no
no
2020-11-14T12:42:28.747Z

Statistics

Average successful run time: 3 minutes

Total run time: 1 day

Total cpu time used: about 1 hour

Total disk space used: 5.59 MB

History

  • Auto ran revision 5f5e831f and failed .
    nothing changed in the database
  • Auto ran revision 5f5e831f and failed .
    nothing changed in the database
  • Auto ran revision 5f5e831f and failed .
    nothing changed in the database
  • Auto ran revision 5f5e831f and failed .
    nothing changed in the database
  • Auto ran revision 5f5e831f and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Node.js

public-proxy / scraper.js