Announcing the development of a new open-source cross-market search engine

Hi /r/DarkNetMarkets,

I've been a long time follower of the darknet community and various hidden services (I still remember thoose nucleus days...).

When /u/gramsadmin announced the shutdown of Grams I immidiately started to work on my long time dream project, making a deepweb spider which collects data from hidden services. The cross-market search of Grams was broken for the past few months before the shutdown, so I started with scraping down markets. Currently I have a demo database of nearly 10k products (out of about 95k on Dream), 500 vendors, 50k ratings and a total purchase volume of ~ 3.1M USD from Dream Market (which is uploaded to a git repo, see the end of post).

The code name of the project is ounces (another common measurement unit on markets).

The program is currently in a very early state, the code structure is still not stable, and some features still missing. At the moment I'm busy writing the documentation, and making it ready to be deployed on my Raspberry Pi. After it I plan to add Tochka support.

I want share the source code under GPLv3 license. If you have some experience with Python programming and hidden service hosting (with probably a python-flask frontend) and would like to contribute to this project feel free to dm me here or send a message to the email linked to my pgp public key.

git repo: http://m6su7s3ir7dxggwg.onion/ounces/ounces

If you started calculating Dream's revenue from the numbers above, keep in mind that the max number of ratings displayed per product is 100 and I only calculated with the fix price of products (not dealing with bitcoin price changes), so their revenue is much bigger than that. However if you have plenty of time to play with the database, feel free to do it and give us interesting statistics ;)

My PGP key:

-----BEGIN PGP PUBLIC KEY BLOCK-----


mQINBFpEBNgBEACtmuQ8CutVHJKTx5mqUuFDCDd+8RwRpnYHKoYcjyPhvt0ZsJ0s

lhjVTiFq+Ga3KJcgvA6SFZHbIV2C9tfQ0tF51kvfLFTGl+JhnKWuP2EeEi8wuhJU

MkMTvak/Mwojhs5LH8uK1TFcQcAKTo3NuRNfR+osA66L93JwWUONWJWQ/B8ckIe1

sy1Uu1PoHY1VHYLCPQZKXlRoj3OwG7YouVZtj/VTaRZbTadD8fTkj6eFvXuhV3Mp

3SBosmO5fQIYnbqWCkIoWTSBXYTwtJVRbV0GNSdAXULyItYeJ1sLBkJk3R8n8n9M

1Fhy1a9rNJnObvS0M2SrGkYDgapYVBT83F3CnCFT6rQsHuwxv3nOrsKYM1GlDi0r

OWcnqr2mytPnd67Tz4eaKvfo0RmXJ87+6XjNecAM7si0Ody3jI13/U027JWANrXq

xRXG9IVYCnQDvTP2CsKDizVgB/LAwBYkpR6MkK7TH7/P8OQCUT+369fUdM18ySRW

WI13aaa27miNAogo2gtjc9P3PwFd0zKjJA+5s59Y5B4O3Gp/1rawAYE1sE0M5iE1

M0+XxPeZ8rIuOADDV/+jLEnKyrf1ZindOnf0KI3keck4WcaE2oEeUrSf5L3rOZtS

9c34FZXHCG6wXm/5dHPohFzVu2yazbG33RI8gaXDJ7U46pylpZuUu5KOHQARAQAB

tClvdW5jZXMgKG91bmNlcyBtYWluIGtleSkgPG91bmNlc0Bjb2NrLmxpPokCTgQT

AQoAOBYhBLfySbUKN0XB8ADKisrWjYwungJTBQJaRATYAhsDBQsJCAcDBRUKCQgL

BRYCAwEAAh4BAheAAAoJEMrWjYwungJTcpwP/1QOZ8AKHhh/ZFFbe8paKh5iTq0f

Z/qasX634Ss4vbisXsmIMrMwR94x10AmNTQM97SWcdgfDH1U05Mn6zFmV+dFQjtv

Q7/CkFJZf1ajE4KYAt2XFE3Buvc7vgymwQXakcja84ztZLs+K7heUdiBI/Pvu79r

PAijt+d6FZBu7UY1OJ0BobppX2jm7VDv3naSBXe7Ane5S6GHYcTaHGJfpMZwW56H

1ok3epVafFUyTgx0PtIw6oohTpjYzUapIbyulpWNKE4GXJxA0VPuE7fZMR4mwXPX

4wmYre53orVqmFovQNwWTSZjBQeDpl3lT02LbpKMzv4J8EB9fliGO+/cvBQtj2Rl

s8FJptBvEfRncnZFreZA1E9YqINoravkHX4tbt6ypkXNEUAZuZ5fA1jy/m5ws1YZ

QBkkyyEm6c4zYyWO3ByERYYvqRLfGqSupYLtmBeDNPPNudkKsA1LgfElJdFUmLzE

wz6QZbKlLvDzVU6Hmwtkb/N+sA4TvmF1q3lmQJUzrY0lyKy0TW5ufXVuYx32QaaL

mtiIpO4mSfnAyAVSC1qzHayJQ0qBY+Snuodm6SVHWRUiBBAhS06CYEZU7sKmoRKQ

JtViQb9QmFR7SYmn6BQieRZqI8cMDND9v7DVn/Y/DGKD0Tcs6+gAT20F2qjyRlOE

1weozlt2Ay+UszEduQINBFpEBNgBEADR+kKESlkxuX6WSIQ7wyl+6N8XjBuPgPqq

Lbf7BXGdC/TxfGdiS/cL5JHHTH9f4yktNp5K7vc7BQibWvuOPS4mJJELwEPJuQNS

nu6c8zw52ZFPX9aJAXiTMEEKz1Xk2l/pC1x8vCPKe+7Cp/BMqMUUTj7dyZstbIKL

PzmdCfWnTyLw3oZIC9jvgUk5zw2GlWbFwrPmAQDWD3yGuMNWfT8IySBIp+JqTDYk

g0K4MwJfoFdB3PDmDdW0lldp18cGltYmNjP73jAFOfRGHbHLDd6qq+f3HjWm/lBQ

HPyaZA57QqK72uVKf8ZC3hQJL10vdaCas2C4DQjc6/w7j/yg649BFP4zCCfOSUEa

70rHf7BWVkLwqfzwYzY2bWISRfITi4fQ/BWqGuLf5M26pRjQ34B3o1o1n/e95U2o

wzd2J79MNfugtWTilhTosnVLuE2s1R152Bs0uISzvt7yeMogJ1hVBrJm8mFOBEu3

DaENw4Iiu2CLZKVpcGLjR70GEhtrMneRXDw2NwMRof0ma3znb+YS+NM9Z9Ld8b9/

i/xx3HoqkWxhSJ5VrJ1lNhWJH+tAmnpkOAllGxbvRd74Uz2wFJrd+qjXmDNtcHap

X2gCMG5okcCtFzo7zkJIsxA4oRw3ObRiD03OLeNcI4oKrZKz46O5uWR4R7/ZNzbO

uI7I3jrhmwARAQABiQI2BBgBCgAgFiEEt/JJtQo3RcHwAMqKytaNjC6eAlMFAlpE

BNgCGwwACgkQytaNjC6eAlNOpQ/9H+Fzqwj+06cmqBi44ePew9v/VL9XNzLptoHc

qGkYHQeL9R7AVAL8kv77EsfCrODNHtO5ATpXHdH1HmGfo1iSmGLSr5ftOHQ38ig0

6W0qZvbY1AujikuDnkfPFrtDJx/p5+MnjBT9WFQ76zdol5/h3XTTITZ6R8apPL1d

tqRx9KSHBj/dYlTz6GWy0GOIAAR/kJPUKSUMNB0sd8ZmHKW3RbFk4IHb85efE6Bw

LVIekZ6g58JRSoFC2hVS7L7wQwNLm0nHbloG9QLBryRxAGkqSOqX6kOscXKKG3b6

tZe9I9tsnWRcdGbvcxBFAwE3FH8c3J1pECvFjmXwgzNStzJvvSmQFYasl1auHR65

DuDm3JXvkwS0rAsvFCvP/jWFrHaPYVMvRpgBRCHbqSkpu2HRIrKwVGS/FApBDw7M

f/KcpsfrrYFm8stAx7LQFxX73+db0IHtpkZy+WHxxNywlkF2CQndanTcO9n+zsUN

7oRKNiIoDnralsd45RhjN6l1A98X8WOQbt48etoBPhYQFmT+FKiUUyGykUnofFN4

EvWbOE6J7UIWv6BgO745OSKFqh5TXi/azo1qR4DGz3sZdfYKl23jgdooglR666os

cWXx6l8Aht5KXh5zMIDILYcyXOG0X5ZyEp/l3H74d/bUysZMFtCLQKfpcMHBZUyX

TxHaZwY=

=eQcL

-----END PGP PUBLIC KEY BLOCK-----


Comments


[8 Points] wombat2combat:

interesting, your scraping setup is similar to mine maybe we can join work. I will get in touch with you in the near future.


[7 Points] FuckYouNotHappening:

Grams

Ounces

You sir are venturing into /r/dadjokes territory ☺


[2 Points] MandyThatGirl:

What does this all mean?? So will I have a place to sell my prescription meds then.

<cradles her knees and starts rocking back and forth


[2 Points] JustAnotherDrugUser:

Excited to see what comes of this.


[1 Points] TheBookofTor:

I like it that you made it open source. We need more people like you.