Here's something lots of people like about Bitcoin: Governments can't control it. You can spend your Bitcoin the way you want to, and nobody can stop you. But here's the bad news: That's not true. It turns out that one of the things everybody believes about cryptocurrency is actually wrong. Really wrong. About a year ago, Trail of Bits was engaged by DARPA, the Defense Advanced Research Projects Agency to answer a question. DARPA wanted to know if one of the things that everybody "knows" about cryptocurrency is actually true: Are blockchains really decentralized? This is a key question for cryptocurrencies. And in this episode, we'll explain what the Trail of Bits team found.
Dan Guido is the CEO of Trail of Bits, a cybersecurity firm he founded in 2012 to address software security challenges with cutting-edge research. In his tenure leading Trail of Bits, Dan has grown the team to 80 engineers, led the team to compete in the DARPA Cyber Grand Challenge, built an industry-leading blockchain security practice, and refined open-source tools for the endpoint security market. In addition to his work at Trail of Bits, he’s active on the boards of four early-stage technology companies. Dan contributes to cybersecurity policy papers from RAND, CNAS, and Harvard. He runs Empire Hacking, a 1,500-member meetup group focused on NYC-area cybersecurity professionals. His latest hobby coding project -- AlgoVPN -- is the Internet's most recommended self-hosted VPN. In prior roles, Dan taught a capstone course on software exploitation at NYU as a faculty member and the Hacker in Residence, consulted at iSEC Partners (now NCC Group), and worked as an incident responder for the Federal Reserve System.
Evan Sultanik is a Principal Computer Security Researcher at Trail of Bits. A computer scientist with extensive experience both in industry (as a software engineer) and academia, Evan is an active contributor to open source software. He is author of more than two dozen peer-reviewed academic papers, and is particularly interested in intelligent, distributed/peer-to-peer systems. Evan is editor of and frequent contributor to the International Journal of PoC||GTFO.
Trent is a Principal Security Engineer and Research Practice Manager at Trail of Bits. He has worked in computer security since 2012 as a researcher and engineer at Assured Information Security in Rome, NY, and at the Georgia Tech Research Institute, where he served as the Threat Intelligence Branch Chief and the Associate Division Chief of Threat Intelligence & Analytics. Trent received his Ph.D. in computational physics from Emory University in Atlanta in 2014, and his dissertation work applied the renormalization group and Monte Carlo methods to study exact results on complex networks.
An accomplished information and physical security professional, Nick leads the Software Assurance practice at Trail of Bits, giving customers at some of the world's most targeted companies a comprehensive understanding of their security landscape. He is the creator of the Trail of Bits podcast, and does everything from writing scripts to conducting interviews to audio engineering to Foley (e.g. biting into pickles). Prior to Trail of Bits, Nick was Director of Cyber Intelligence and Investigations at the NYPD; the CSO of a blockchain startup; and VP of Operations at an industry analysis firm.
Story Editor: Chris Julin
Associate Editor: Emily Haavik
Executive Producer: Nick Selby
Executive Producer: Dan Guido
Rocky Hill Studios, Ghent, New York. Nick Selby, Engineer
Preuss-Projekt Tonstudio, Salzburg, Austria. Christian Höll, Engineer
Remote recordings: Whistler, BC (Nick Selby); Queens, NY (Emily Haavik)
Edited and Mastered by Chris Julin
Trail of Bits supports and adheres to the Tape Syncers United Fair Rates Card
Watch a video of this podcast.
Dispatches From Technology's Future, the Trail of Bits theme, Chris Julin
CANTO DELLE SCIACALLE, Cesare Pastanella
SHALLOW WATER - REMIX, Omri Smadar, Yehezkel Raz, Sivan Talmor
ALL IN YOUR STRIDE, ABE
LET IT RISE, Divine Attraction
ROAD LESS TRAVELED, The David Roy Collective
KILLING ME SOFTLY, Ty Simon
TECH TALK, Rex Banner
LOST ON EARTH, Marek Jakubowicz
SCAPES, Gray North
With the exception of any Copyrighted music herein, Trail of Bits Season 1 Episode 0; Immutable © 2022 by Trail of Bits is licensed under Attribution-NonCommercial-NoDerivatives 4.0 International. This license allows reuse: reusers may copy and distribute the material in any medium or format in unadapted form and for noncommercial purposes only (noncommercial means not primarily intended for or directed towards commercial advantage or monetary compensation), provided that reusers give credit to Trail of Bits as the creator. No derivatives or adaptations of this work are permitted. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
In “Are Blockchains Decentralized? Unintended Centralities in Distributed Ledgers,” Evan Sultanik, Trent Brunson, and nine other engineers on the Trail of Bits Research and Engineering and Software Assurance teams report their findings from the year-long project to examine Blockchain centrality.
Fluxture is a free and open source software crawling framework for Blockchains and peer-to-peer systems that Trail of Bits created to assist with the work described in this episode. We also link to the free and open source recursive dependency graphing tool It-Depends, which we will discuss in depth in the upcoming podcast episode that’s creatively titled, It-Depends.
The Are Blockchains Decentralized? Unintended Centralities in Distributed Ledgers paper cites more than 30 academic and commercial research papers.
There is literature about how malicious Tor exit nodes surveil and inject attacks into Tor-users’ traffic. You may also read comments about exit node manipulation by Tor network maintainers. One report states that On February 2, 2021, a single, malicious actor was able to fully manage 27 percent of Tor's exit capacity.
The reports “How Malicious Tor Relays are Exploiting Users in 2020 (Part I)" hypothesized that the entity behind a range of malicious tor relays would not to stop its activities anytime soon; the follow-up, "Tracking One Year of Malicious Tor Exit Relay Activities" continues the discussion.
Chris Julin has spent years telling audio stories and helping other people tell theirs. These days he works as a story editor and producer for news outlets like APM Reports, West Virginia Public Broadcasting, and Marketplace. He has also taught and mentored hundreds of young journalists as a professor. For the Trail of Bits podcast, he serves as story and music editor, sound designer, and mixing and mastering engineer.
For the past 10 years Emily Haavik has worked as a broadcast journalist in radio, television, and digital media. She’s spent time writing, reporting, covering courts, producing investigative podcasts, and serving as an editorial manager. She now works as an audio producer for several production shops including Us & Them from West Virginia Public Broadcasting and PRX, and APM Reports. For the Trail of Bits podcast, she helps with scripting, interviews, story concepts, and audio production.
NARRATOR (Nick Selby): Here's something lots of people like about Bitcoin: Governments can't control it. You can spend your Bitcoin the way you want to, and nobody can stop you.
But here's the bad news: That's not true. It turns out that one of the things everybody believes about cryptocurrency is actually wrong. Really wrong.
DAN GUIDO: Other people can make it impossible for you to transfer your cryptocurrency, and they can make it impossible for you to spend it at all.
NARRATOR: That’s Trail of Bits CEO, Dan Guido.
DAN GUIDO: this has really practical, real world impacts. If Russia wanted to stop people from donating to Ukraine, they could do it.
MUSIC: CANTO DELLE SCIACALLE Cesare Pastanella
NARRATOR: About a year ago, Trail of Bits was engaged by DARPA, the Defense Advanced Research Projects Agency to answer a question. DARPA wanted to know if one of the things that everybody "knows" about cryptocurrency is actually true: Are blockchains really decentralized? This is a key question for cryptocurrencies. And in this episode, we'll explain what the Trail of Bits team found.
I'll give you the short answer right now: No. Blockchains are not truly decentralized. And that's a big deal. We'll explain.
In our show notes, we include links to the full, un-redacted report, and supporting and analytical materials. The findings are reproducible, our research is open source, and freely distributable. So you can dig in for yourself.
One thing to be clear about: Trail of Bits thinks blockchain technologies are exciting, innovative, and can push the boundaries of current technology in exciting and useful ways. We’re not by any stretch of the imagination anti-blockchain. Our work is entirely about understanding and mitigating security risk and we think the risks here have been poorly described, and are often ignored – or even mocked – by those seeking to cash in in this decade’s gold rush.
Our team did some novel work. They built new tools and pursued original research. But the team also stood on the shoulders of the industry’s giants. They performed analysis and meta-analyses of prior findings that had never been aggregated. Putting it all together allowed us to make connections that hadn't been made before. The resulting report is a 30-thousand-foot view encompassing what’s known about blockchains in a way that we think is unique.
NARRATOR: All right. So let's look at centralization. It's a key concept for this whole discussion.
You probably know that cryptocurrencies like Bitcoin and Ethereum run on a blockchain, and a blockchain is a distributed ledger of transactions. We've been told that the ledger is immutable. It can't be changed. So when I sell you a Bitcoin today, the record is locked in for all time. That ledger of transactions is also decentralized. That means there isn't just one copy hidden away somewhere. It's public. So if someone alters their copy of the ledger, everyone else will know. This is an essential element of blockchain security. Everyone controls it, so no one controls it. And that means no one can manipulate it.
At least, that's what we've been told.
EVAN SULTANIK: You could manipulate the blockchain if you wanted.
NARRATOR: That’s Evan Sultanik.
EVAN SULTANIK: I’m a Principal Security Researcher at Trail of Bits. Everyone seems to take for granted the fact that blockchains are safe because they can't be manipulated. We took a look at the technological underpinnings that provide that and discovered that no, blockchains aren't fully decentralized, they aren't immutable. And With some modest resources, anyone can choose which transactions are approved in a blockchain.
NARRATOR: To really get this, we need to go deeper into this idea of "centrality."
There are actually a bunch of different kinds of Centrality, and you could go down a rabbit hole about each one. But for the purpose of the discussion in THIS episode, we're going to focus on just two kinds: Network Centrality, and Software Centrality.
MUSIC: SHALLOW WATER - REMIX, Omri Smadar, Yehezkel Raz, Sivan Talmor
First, Software Centrality.
Blockchains and the systems that they interact with are software. And like any software, it can contain coding errors and bugs. Some of those errors make software vulnerable.
In March, 2022, more than one in every five Bitcoin nodes were running dated versions of the Bitcoin Core client - versions that are known to be vulnerable to attack. So that's an example of Software Centrality.
TRENT BRUNSON: And with Network Centrality, we are interested in what would happen if a malicious internet service provider or nation-state decided to block or filter traffic to a blockchain.
NARRATOR: That’s Trent Brunson, he’s the Practice Director for Research and Engineering at Trail of Bits.
TRENT BRUNSON: So we look at whether the nodes are sufficiently geographically distributed such that they are uniformly distributed across the internet.
NARRATOR: But that network decentralization is challenged: The TOB team found that not only could a nation-state filter blockchain traffic. You could, too.
EVAN SULTANIK: If you don't like someone and you don't want any of their Bitcoin transactions to be mined into the Bitcoin blockchain and you have control over their network traffic, all you have to do is drop any bitcoin message that contains a transaction interacting with their account.
MUSIC ends
NARRATOR: It's pretty easy to set up a firewall to do that. It’s a trivial matter to just … redirect selected traffic to a “black hole”. So someone could remain in possession of their Bitcoin, see it in their wallet, but be unable to spend it.
EVAN SULTANIK: This is definitely something a government can do, particularly governments that have control over all of their Internet service providers. It is something that an Internet service provider could do.
NARRATOR: That's scary enough. Here’s an another problem:
EVAN SULTANIK: The majority of Bitcoin traffic these days is being routed over a network called Tor. And Tor is notoriously insecure.
NARRATOR: Tor is now the largest network provider in Bitcoin. In March, 2022, just about fifty-five percent of Bitcoin nodes were addressable ONLY via Tor. Every message that leaves Tor passes through an "exit node." You can think of an exit node as a computer server that bridges traffic from the Tor network to the regular Internet. Those exit nodes are vulnerable.
EVAN SULTANIK: Anyone can create a Tor exit node and filter all of the bitcoin traffic that goes through it. They can choose which traffic to drop and they can even manipulate messages. You can't manipulate the cryptographically signed portions, but you can certainly do things like decide, I don't think this transaction should make it to the Bitcoin network. And it will just disappear.
MUSIC: All in Your Stride
NARRATOR: Tor is billed as offering anonymity and privacy, so you might think it’s safer or “more secure” than the “regular internet”. In some ways, though, the opposite is true. Because governments, whistleblowers, and bad actors alike - are all watching Tor and its exit nodes - very closely.
Malicious Tor exit nodes surveil and inject attacks into Tor-users’ traffic. That means that not only can people monitor traffic from Tor exit nodes, but that they can manipulate it before passing it on to its destination.
EVAN SULTANIK: The Tor network is hostile. It's like sending your traffic through the bad part of town. You have no idea where it's going. You have no idea where the messages have been. And there's a storied history of people listening to Tor exit nodes to see what they can find.
NARRATOR: What Evan’s describing here is not theoretical; malicious activity at exit nodes is a known thing - we link in the Show Notes to comments on the topic by Tor network maintainers. One report we link to in the show notes states that On February 2, 2021, a single, malicious actor was able to fully manage 27 percent of Tor's exit capacity.
Incredibly, Bitcoin Consensus Network messages are NOT encrypted.
Most people think that the strong cryptography is why blockchains and currencies like Bitcoin are safe - the cryptography is strong, go the marketing claims, so your money is safe. The Trail of Bits team thinks the cryptography is fine:
EVAN SULTANIK: For all intents and purposes, the cryptography used in blockchain is secure.
MUSIC: LET IT RISE, Divine Attraction
NARRATOR: But you’ll notice that absolutely none of the security issues we are highlighting here have to do with Cryptographic integrity. For example, when it comes to immutability, consider that, in late August of 2021, a consensus issue related to changes in the most popular Ethereum client was exploited to cause a hard fork of the cryptocurrency. This is proof that the data — and, more important, code — deployed to a blockchain are not necessarily semantically immutable.
Not only can the state of the blockchain be retroactively changed through modifications to the blockchain’s software, but the semantics of individual transactions can change between when the transaction is initiated and when it is ultimately mined onto the blockchain thanks to software changes in the interim. Some blockchain platforms - like Polkadot and Substrate - also let certain parameters and code be updated (this means, ‘Changed’) through an on-chain governance process.
So we know that pretty much anyone can choose which transactions are approved in a blockchain. To understand the next finding, we need to talk about something called a 51 percent attack.
The way that a blockchain prevents fraud and manages its resources is that when a new transaction takes place, computers in a special network immediately work to verify it. Once the majority of those computers certify it as being correct, the transaction goes through and the data is locked into the blockchain. This is called reaching consensus.
So in order to get a fraudulent transaction through, you’d theoretically have to have 51 percent of the computers in that network verify it. To achieve that artificially, you’d need to gain control of that 51 percent. The traditional view has been that this would take so much money and so many resources, that it would be virtually impossible.
MUSIC: ROAD LESS TRAVELED, The David Roy Collective
But the Trail of Bits team found that in practice, even today, you actually don’t need that full 51 percent of Bitcoin mining power to launch this kind of attack.
EVAN SULTANIK: What reduces a 51% attack to less than 51% is the age of the knowledge that each of the consensus nodes has. So the more out of date your knowledge is, the more work that you're doing that doesn't actually contribute to consensus. And the slower the network is in reaching consensus, the less computing power you need to execute an attack on it.
NARRATOR: So a 51% attack doesn’t actually require, “51 percent of all the computing power in the consensus network” but instead, “51% of all the computers in the consensus network THAT ARE WORKING ON CONSENSUS RIGHT NOW.”
And remember: the 51% number assumes that the network is functioning without any inherent signal delay, or “network latency” - that’s almost never the case.
EVAN SULTANIK: Right now, for example, according to our most recent calculations, you only need about 49% of the computing power to take over Bitcoin. That 49% is just from natural network delays.
And if you could slow down the network beyond just the natural delays?
EVAN SULTANIK: Uh, let me get an exact number…
NARRATOR: With the accidental or nefarious introduction of further latency, the hashrate needed can plummet. With just a few minutes of delay, the takeover threshold drops to 40%, and with less than an hour it can be as low as 20%.
EVAN SULTANIK: then you would only need about 20% of the computational power to take over Bitcoin.
MUSIC KILLING ME SOFTLY, Ty Simon
NARRATOR: One often misunderstood aspect of how the Bitcoin network seeks to prevent hankypanky is called “Proof Of Work.” Simply stated, each node within the network has to solve a cryptographic puzzle, demonstrating they have done some work, in order to qualify to verify a transaction.
When you hear about all the energy consumption related to Bitcoin, it’s because Proof of Work, and “Mining” needs a LOT of resources.
The Ethereum network has been working to replace “proof of work” with a “proof of stake” mechanism that uses less computational power and thus less energy. Eventually the current Ethereum Mainnet will "merge" with the beacon chain proof-of-stake system, which will end use of proof-of-work for Ethereum. For example, in the Ethereum network, users can stake their ETH to become a validator in the network. Validators in ETH are the same as miners in proof-of-work schemes like Bitcoin: a validator helps with the order of transactions and creation of new blocks, so that all nodes can agree on the state of the network.
MUSIC out
And, it turns out this newer type of consensus mechanism - is actually *more vulnerable.
EVAN SULTANIK: We took a look at proof of stake, and the percentage went down. And even right now, there are some blockchains that have as few as three entities who, if they collude with each other, can choose to shut down the whole blockchain.
Think of each entity as a “wallet.”
EVAN SULTANIK: Polygon is one of these next generation proof of stake blockchains. It has like over $3 billion of staked assets. And there are only two wallets that control over a third of the staked assets. Whoever controls those two wallets, they can decide to shut down the whole thing.
NARRATOR: So what about Bitcoin? Today, the four most popular Bitcoin mining pools constitute over 51% of the hashrate of Bitcoin. In other words: the number of entities necessary to execute a 51% attack on Bitcoin was reduced from about 59,000 nodes, to four.
That’s less than 0.004% of the total nodes out there.
To be clear, some blockchains are more protected than others. But they’re ALL vulnerable.
EVAN SULTANIK: Solana, for example, which is another proof of stake blockchain that has over 37 billion in staked value. They need a minimum of 19 wallets to decide to deny service to the system. But that's still only 19 wallets out of thousands or hundreds of thousands in the ecosystem.
MUSIC: TECH TALK, Rex Banner
EVAN SULTANIK: … None of these numbers are good.
NARRATOR: We’ve just gone through some significant revelations about the technology that underpins Bitcoin and other cryptocurrencies. Now, what does this mean - for you? Here’s Trail of Bits CEO Dan Guido again.
DAN GUIDO: What it means is that another government, an ISP, somebody running a Tor exit node, can tell you how to spend your cryptocurrency.
NARRATOR: These vulnerabilities impact every person out there who has purchased Bitcoin - or who’s invested a portion of their retirement savings in it.
And, beyond personal savings, there are higher-level implications to think about.
DAN GUIDO: I think this is a real concern because there's a lot of governments around the world that want to make a central bank digital currency, and once they have it, they're not going to want other digital currencies to exist inside their borders. This allows those governments to censor transactions into other blockchain networks.
To Dan, one of the most alarming elements here - is not the findings themselves. It’s how difficult it was to dig them up.
DAN GUIDO: We had to spend months of time making novel tools to understand how the network operates, and that is not what you'd expect for something that carries cryptocurrency and value for billions of dollars and millions of people… Alongside the release of these papers, we've also released tools that allow us to investigate the network and determine when these sorts of things happen. But much more research is needed so that we can find out when people are censoring transactions, when the network operates in ways it's not supposed to, because right now it's way too difficult.
NARRATOR: We know we’ve dropped some pretty heavy information in this episode. And we’d love to give you some solutions. But if you’re not a security engineer, a politician, or a decision maker at any of these companies, the reality is that your power lies solely in your influence as a consumer.
DAN GUIDO: There's not a whole lot that average people can do to affect this problem besides demand that the people that are developing blockchain software and operating it invest in tools and techniques that help address this problem.
NARRATOR: Because this problem is not going away. Beyond cryptocurrency, today, blockchain technology is used extensively in agricultural supply chains. People are considering it for even more sensitive things in the future, like opioid prescription control.
Unless we tackle these nuanced and thorny technical issues head-on, the underlying flaws in the system are only going to create larger and larger risks as societal reliance on blockchain technologies increases.
EVAN SULTANIK: Blockchains are here to stay.
MUSIC: LOST ON EARTH, Marek Jakubowicz
NARRATOR: Here’s Evan Sultanik again:
EVAN SULTANIK: A lot of people in industry and people who are using these things every day thought they knew how they worked, but nobody really had proof. So we wanted to see how they actually work. We've seen all sorts of suspicious things happening in blockchains.
NARRATOR: Trail of Bits is optimistic about the future of blockchain, but that optimism is predicated on the economic imperative to face courageously the underlying technical challenges to advance the technology past its nascence.
DAN GUIDO: A lot of people are racing into this, and it's just like every other gold rush that we've seen in history. A lot of people aren't looking at the fundamentals of the technology. They're not determining if it's safe to build on. They're just building on it and being satisfied with it. But over time, this risk will build up to become existential and could threaten the security of the entire ecosystem.
--
NARRATOR: The people who worked on this podcast are Emily Haavik, Chris Julin, Evan Sultanik, Dan Guido, Josselin Feist, Trent Brunson, Opal Wright, and hi, I’m Nick Selby, I’m the Director of the Software Assurance Practice here at Trail of Bits.
Chris Julin made our theme music.
Trail of Bits helps secure some of the world's most targeted organizations and devices. We combine high-end security research with a real-world attacker mentality to reduce risk and fortify code. We believe the most meaningful security gains hide at the intersection of human intellect and computational power. Learn more at trailofbits dot com or on twitter, AT trailofbits; Dan Guido’s Twitter account is AT dguido, and I’m AT fuzztech.
--
DISCLAIMER: The research to which this podcast refers was conducted by Trail of Bits based upon work supported by DARPA under Contract No. HR001120C0084 (Distribution Statement A, Approved for Public Release: Distribution Unlimited). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Government or DARPA.