Ethereum proof-of-stake client bug caught and patched without incident

Ethereum developers identified a bug within the Besu Ethereum client that could have led to “consensus failure in networks with multiple EVM implementations.”

Gary Schulte reported the issue to the Hyperledger GitHub repository and was found by Martin Holst Swende. It is understood that “no production networks have transactions that would trigger this failure.”

Bug identified during The Merge code review

Swende documented that he found the bug while “doing some #ethereum fuzzing in preparation for #TheMerge.” In response to a CryptoSlate journalist, Swende stated that users running a Besu node would have become stuck and “not able to follow the canon chain.” Further, any “besu-dominated network could have been stopped in it’s tracks.”

They would have been stuck, not able to follow the canon chain. And/or, any besu-dominated network (non-eth-mainnet) could have been stopped in it’s tracks.

— M H (((Swende))) (@mhswende) September 27, 2022

The Besu client is the second most popular client on the Ethereum network behind Geth. According to data available via ethernodes.org, The Besu client is used by 7.81% of Ethereum mainnet clients.

Vulnerable Besu client versions

Version 22.7.1 of the Besu client contains a fix to ensure “excess gas will not be allocated to inner transaction calls and correcting the excess gas errors.”

Versions earlier than 22.1.3 will also “prevent incorrect execution,” however, Ethereum mainnet requires other features only available in later versions. Client versions 22.4.0 to 22.7.0 are currently considered vulnerable to the gas bug.

As a result, Besu client users on the mainnet must upgrade to the patched version.

Impact and resolution

Danno Ferrin created a full write-up of the issue in a Hackmd article published Sept. 21. Ferrin’s analysis stated that

“A flaw in handling unsigned data as signed data a properly coded smart contract can create a function call that will return more gas than was passed in.”

Further technical information regarding the bug can be found in Ferrin’s post. However, the main takeaway is that the bug was resolved without any issue on the Ethereum mainnet. For a bad actor to maliciously exploit the bug, they would have had to act in a precise manner.

“In order to elevate this to a chain-halting bug a deliberately crafted call was needed, involving some interactions with the EIP-150 “all but one 64th” rule and reserving a portion of available gas for the calling contract.”

If the bug was not found, any chain with high participation from the Besu client could have experienced a smart contract “infinite loop” whereby the contract would “truly execute forever.”

Ferrin stated that fuzzing enabled the developers to identify and patch the bug without issue. Fuzzing is a method used by software developers “that involves providing invalid, unexpected, or random data as inputs to a computer program.”

“The biggest lesson demonstrated by this exploit is that the comparison of trace data in a fuzzing execution catches more bugs than simply comparing the end results.”

The excess gas bug became a non-event due to the diligence of Ethereum developers dedicating themselves to protecting the network. However, the potential harm it could have caused showcases the complexity behind executing the merge without issues.

The bug was patched in version 22.7.1 using “a different conversion method that will “clamp” overflow values to the maximum expected values avoiding the signed translation issues.” Ferrin commented that users running nodes within the vulnerable range should update to the most recent version.