Vice-Motherboard published an article about Gwen's 52gb DNM archive

You Can Now Download a Copy of Pretty Much Every Dark Web Market Ever Made

Written by JOSEPH COX

July 16, 2015 // 04:15 AM EST

The life of a dark web site can be fleeting.

As quickly as one drug site emerges, it can disappear shortly after, taking with it all sorts of information on what items were on offer, the posts on the site's forums, and plenty of other useful pieces of data for journalists or researchers, who are left empty handed in their attempts to keep tabs on the dark web.

Now that might change, thanks to an independent researcher who has released a colossal archive containing around 52Gb of data from over 80 different dark web marketplaces.

Since 2013, Gwern Branwen has collected the item listings, customer feedback, images, forum posts and much more of all English-language dark web markets. Branwen did this on a weekly or sometimes daily basis, according to a write-up on his site.

"This uniquely comprehensive collection is now publicly released as a 52GB (~1.6TB) collection covering 89 [Darknet Markets] & 37+ related forums, representing <4,438 mirrors, and is available for any research," he wrote.

Gwern Branwen has collected the item listings, customer feedback, images, forum posts and much more of all English-language dark web markets Most of the files in the collection are HTML or CSS, Branwen told Motherboard in a Twitter message, and the archive doesn't just contain Branwen's own data sets, but also those from several other people as well. These include academics such as Nicolas Christin from Carnegie Mellon University, and even dark web personalities such as the vendor known as "El Presidente," who made a backup of drug dealers' contact details and PGP keys in the event of marketplaces disappearing.

Branwen has contributed plenty of pieces of original research looking at the dark web markets. He's worked out the relative risk of running or participating in a market, judging by the number of known arrests from that particular market, and also mapped how long each market existed for.

A vibrant academic community exists around these sites too, with some researchers looking at the size of different marketplaces, and others investigating whether drugs on Silk Road were sold mostly to individuals, or to other drug dealers.

"I want to enable all sorts of research and analysis." Branwen's archive is now open for anyone to use, and he gives some suggestions for research that could springboard from his work. It might be possible to calculate the total number of sales per day on a market, and see whether there are any correlations with the Bitcoin price, he wrote. One could analyse the data for indicators of an "exit-scam," when a vendor or site owner disappears with a customer's cash. Or perhaps a researcher could collate all of the posts which talk about drug purity and see if any trends have developed over time.

Jamie Bartlett, author of the Dark Net, is already thinking of ways to use the data set, such as "working with some drugs charities to see what benefit there could be for the harm reduction groups and health professionals," he told Motherboard in an email. "It's such an incredible resource."

That's exactly what Branwen had in mind when publishing the massive collection.

"I want to enable all sorts of research and analysis," Branwen told Motherboard.

Edit: added link -> http://motherboard.vice.com/read/you-can-now-download-a-copy-of-pretty-much-every-dark-web-market-ever-made


Comments


[18 Points] None:

Great publicity for Gwern. He earned it.


[4 Points] None:

52GB (~1.6TB)

wut..?


[5 Points] -El_Presidente-:

even dark web personalities such as the vendor known as “El Presidente,”

Vendor? Seriously? Which orifice did Joseph Cox pull that from?


[2 Points] Vendor_BBMC:

Another darknet vendor sent a link to this post today, with a comment (and I quote) "Why would anybody think this is a good idea?"

I had to explain that Gwern likes to joke around. "Click the link. Do you think everybody would be wishing him bon voyage if he had really cracked up and dumped gigabytes of evidence where LE can access it? He was still asking for "marketplace scrapes" as recently as yesterday"

I have to admit, it gave me pause for thought. Everybody is congratulating him, for being so fastidious in his documentation of their online criminal activity. Maybe this is the best thing ever, because his research will enable us all to improve our OPSEC, and get away with all of our guilty history - totally scot-free. Somehow, the slate will be wiped clean, there will be no evidence of our drug buying and selling, and Gwern Branwen will go down in darknet history as a great hero.

I tried to picture this in the real world, because sometimes things are difficult to picture if they are "online". So here we go:-

Imagine Gwern standing outside a crack dealer's house, noting what was on offer every day, seeing who goes in and out, how long the place operated for, and taking the dealer's fingerprints / PGP key (to link him to crack dens in other cities).

Try to picture Gwern in an Italian restaurant, wearing a wire and doing a "scrape" of every bit of crime-related gossip, knowing that the made-guys use the restaurant as a kind of venue (or "forum" if you will) for narcotic-trade discussion.

If Gwern were technically-minded, he could place a microphone in every forum, and keep a tally of how much of each drug was sold by every drug dealer, along with 2 lines of testimony from every one of their customers that the drugs crossed borders, or didn't arrive, or "FE for trusted vendor. Will update"

Offline, the crack dealer might tap on Gwern's car window, and politely ask him what he was writing. "Oh, just calculating how much crack you've sold, anywhere. Since 2009. Its for my thesis"

"Don't worry. I would never talk to a reporter"

"Reporter? Ha ha ha! Its a good job I know you are joking. Otherwise, I would kill you"

That's OK then. Good luck with your degree. But not a word of this must ever get out. You know what happens to grasses"

"Of course not. This will go to the grave with me. I would never talk to the police. They could get a subpoena for all I care, they will never see a megabyte of the dirt I have on you and your customers"

"Can you imagine if I ever released my portfolio of criminal evidence publically? It would be used in every drug trial in every country, putting people behind bars for thousands of extra years, and earning the feds millions in asset forfeiture. It would be the biggest disaster in darknet history, worse than the seizure of the Silk Road server"

"If there was ever a movie about the darknet, I would be played by John Malkovich"

Good luck to Gwern Branwen - still cracking those April fools jokes right to the end. But don't forget how you all shat yourself when you thought the police had subpoena'd his "research"? I thought he was starting to crack up the last couple of weeks. I can always tell when I'm being lied to, and I was afraid he was going to do something stupid.

The fact you're all wishing him well rather than threatening to kill him reassures me that everything is alright, and the evidence is safe. (I don't like to click on links in Reddit). Gwern didn't become the darknet's first international supergrass. But I admit I panicked there for a moment. I nearly clicked on the link, half expecting to see it all there in black and white. But remember this:- Do you know for sure that Gwern is a great guy? What if he started asking for bitcoin contributions from our community? He could land every vendor and customer who ever used a darknet marketplace in the shit if he wasn't such a stand-up guy. It would only take one stolen copy to get out and he will have bombed darknet markets back to the stone age.

Just out of interest, has anybody clicked the link to see what's Gwern REALLY made public? I bet it was a single pdf file containing the words "Fuck you, police! Find another stool pigeon, suckers!"


[2 Points] Hank_Vendor:

I want to understand . . . Please. Someone tell me why this is a good idea?


[2 Points] Vendor_BBMC:

If he really DID crack up and dump evidence of everything we ever said, bought or sold since the early days of Silk Road where they can be used as evidence in criminal trials, it would certainly put vendors who don't delete your shipping details into perspective!

Gwern was publicly questioning my judgement recently, like he's the DNM professional and I'm the amateur. Talking about banning me for asking him questions.

I've seen the way some men act when they suffer "TOR fatigue", and I was worrying that gwern was displaying all the symptoms. He was clearly suffering under the burden of the vast archive of potential evidence he had accumulated on every one of us. Its a heavy load for one man's shoulders.

I admit I was worried that he was cracking under the strain, but couldn't quite bring himself to keep it safe until the statute of limitations renders it inadmissible (in the US at least. In the UK, it could be used to prove what I sold, and how much, for how long, even if I'm 90 when they kick my door in).

Sometimes I think I'm a better judge of character than I really am. I thought gwern was losing it and about to do something stupid without thinking it through.

Somebody has to document history, and it should be preserved for future generations. I tried to discourage gwern from retiring. But the more I think about it, the more I understand why gwern had to get out. Now, while he still can.

If his data ever leaked, it could set the darknet marketplace community back 5 years. He's worked so hard on it that it must be killing him knowing that he can never show it off without severely endangering current marketplace users. But in particular, vendors.

At least I could hand his marketplace scrapes to my accountant so he can calculate my back taxes if drugs are ever legalized!


[1 Points] None:

Nice. Applause all around.


[1 Points] CocaineNose:

Gwern is his first name?! Mind blown