Polygon (MATIC) suffers network outage sparking a wave of angry users

Polygon devs yesterday alerted users to an issue regarding Tendermint implementation, leading to network downtime.

Network update:
The team is currently looking into an issue with the Tendermint implementation used by one of the two layers of Polygon PoS chain.
All users funds and data remain fully secure but there is a potential threat of Polygon PoS users experiencing downtime sporadically.

— Polygon Developers 💜 (@0xPolygonDevs) March 10, 2022

The dev team said they had deployed a “temporary hotfix” to reinstate block production several hours later. But the search for a longer-term fix is on, indicating the issue has not been fully resolved.

“We appreciate your support during this time and have an update: We have deployed a temporary hotfix to unblock the Bor chain and resume producing blocks on the Polygon chain. While this solution is temporary, the team is working on implementing a longer-term upgrade to fix the Heimdall issue.”

Although devs said funds and data are safe, users have voiced their frustration with incomplete transactions and missing funds.

Worse still, others have taken the incident to rekindle project rivalry.

What are Tendermint and Heimdall?

The Polygon mainnet is a hybrid Plasma Proof-of-Stake chain to Ethereum, made up of a Tendermint consensus validator layer and a Plasma sidechain for block production. Specifically, Polygon uses Peppermint as its “consensus engine,” which is a modified version of Tendermint – the framework that underpins the Cosmos chain.

Heimdall oversees this process by managing validators, block producer selection, spans, the state-sync mechanism between Ethereum and Matic, and other essential functions.

“The main chain Stake Manager contract works in conjunction with the Heimdall node to act as the trust-less stake management mechanism for the PoS engine, including selecting the Validator set, updating validators, etc.”

The Heimdall layer validates the blocks produced by the Bor chain, creates a Merkle tree of the block hashes, then publishes the Merkle root to the main chain.

Merkle trees are a data structure to encode blockchain data efficiently and securely. They are helpful because they offer data compression allowing individual transaction verification without processing the entire blockchain.

Based on the information from devs, it seems as though a fault at the Peppermint process level meant validators could not agree on the state of the network, thus halting block production. While the issue has been temporarily patched, there appears a deeper flaw at the Heimdall layer.

The community weighs in on the Polygon outage

Users have expressed annoyance over missing funds, stuck transactions, and error messages in response to the outage.

The incident has rehashed criticisms with Polygon. Last month, Justin Bons, the Chief Investment Officer at Cyber Capital, launched a scathing attack on Polygon, calling it insecure and centralized.

Bons’ primary focus was on the smart contract multi-sig setup, which would take just five people conspiring to compromise the entire network. This, Bons says, is “one of the largest hacks or exit scams just waiting to happen.”

2/14) The Polygon smart contract admin key is controlled by a 5 out of 8 multi-signature contract.

This means that polygon can gain complete control over Polygon with only 1 of the 4 outside parties conspiring.

The other 4 parties in the multisig where also selected by Polygon.

— Justin Bons (@Justin_Bons) February 12, 2022

Adding insult to injury, the most upvoted comment in this Reddit post spoke of the irony that, before the outage, some had claimed Polygon’s technology is superior to Cardano.

This week, Cardano has come under fire due to allegations its founder had faked his academic credentials. That aside, the Cardano chain has run continuously since its launch in September 2017.