Incentivized Testnet Stage 4 is Live! Join Today

February 21, 2022

Shardeum Solves Blockchain Trilemma

Blockchain's ultimate objective to reach worldwide adoption is to solve the scalability trilemma. Learn what it is and find how Shardeum solves...

Written By

Subbu

Home » Blog » Shardeum Innovations » Shardeum Solves Blockchain Trilemma

Getting your Trinity Audio player ready...

A new EVM-based layer 1 and community driven blockchain is on its way! How will it position itself among other layer 1 blockchains like Bitcoin, Ethereum, Cardano, Solana? How will it solve the challenges other smart contract platforms haven’t, retaining scalability, security and decentralization? Although you will get a clue from the name of the blockchain itself — Shardeum, there’s more than meets the eye. With a ground breaking technology and architecture, Shardeum is aiming to make decentralization affordable and accessible that will ultimately power the adoption of decentralized applications (dApps) at a truly global scale. In this article, we will find out how exactly Shardeum will achieve its goal at a high level.

As many of us know by now, established layer 1 networks like Bitcoin and Ethereum are slow and expensive. More recent layer 1 blockchains claim thousands of TPS (Transactions Per Second) with very low gas fees. However they sadly fall short in one or more of the following areas — scalability, security, decentralization and operational efficiency – which in reality results in network congestion and very high transaction fees especially once TPS is throttled during peak demand.

This is where layer 2 solutions such as Polygon, for example, find their value propositions. Remember, layer 1 blockchains are open source networks which allows such layer 2 solutions, that are faster and less expensive, to be built on top of them. Layer 2s have quite successfully enabled various use-cases that fuels today’s Web3 ecosystem. All the major utilities that you see in the blockchain industry were originally envisioned by layer 1 networks like Ethereum but they are now largely administered, so to say, through alternative scaling solutions like L2s and sidechains. Layer 2’s do this by adjusting their architectures that allows for higher scalability and lower gas fees. However, these adjustments don’t necessarily guarantee high security and/or decentralization – aspects that are fundamental to blockchain technology and what makes it stand out. And that is why experts often advocate addressing the scalability trilemma at the foundational (L1) level, as it will directly benefit any products built upon it.

Table of Contents

Blockchain’s Scalability Trilemma

Ethereum’s famed co-founder, Vitalik Buterin, coined the term, “scalability trilemma”. The scalability trilemma highlights the challenge faced by public blockchains, where achieving a balance among the three fundamental properties of security, scalability, and decentralization simultaneously is difficult. Vitalik maintains that, at best, only 2 out of the 3 properties can coexist at a given time sacrificing the 3rd one. Since security is prioritized typically, these platforms often make a choice between scalability and decentralization.

Now, even if the scalability issue is mitigated with layer 2 solutions and other scaling solutions, there have only been incremental improvements to meet the demands of the industry. While we aim to replace Web2 with Web3 and rightfully so, the former provides a largely seamless user experience we’re all accustomed to at global scale. In order to successfully replace Web2, Web3 needs to scale up in terms of throughput capacity for masses to realize and adopt the embedded values of permissionless blockchain technology that includes higher levels of inclusivity, data ownership, privacy and transparency. A case in point is, Visa – a global payment company – processing an average of 4000 TPS while public blockchains (including the more recent ones) processing an average of 400 TPS.

There’s More to the Scalability Issue

Today, social media apps host billions of activities and users on their networks. As users, we create and publish text, audio and video contents with significant ease on such platforms. An individual can perform numerous activities such as ‘like’, ‘share’, ‘subscribe’, and add emojis in response to contents published. Consider the throughput required for such activities produced by billions of people! The fact is, decentralized blockchains are not even close to offering the throughput centralized competitors offer today, and definitely struggle to offer a UX up to par with what society is accustomed to. This excerpt from a recent article on CoinDesk, properly summarizes the situation.

“How many blockchain games have lore or world building? Where is the community-driven content and engagement on social media? It’s difficult to even find YouTube or Twitch gameplay of crypto games that supposedly have hundreds of thousands of users.”

Since throughput directly impacts your ability to scale, it is no longer sustainable to overlook the scalability aspect in blockchains. Without the ability for L1 blockchain networks to scale, businesses and individuals will have to continue relying on big tech to manage our Personally Identifiable Information (or PII) and other user generated contents. Such entities are known to have repeatedly abused the power of hosting and controlling users’ data, which has been a catalyst for decentralization. Moreover lack of scalability inevitably drives up average transaction costs for end users creating a vicious cycle that limits world wide adoption.

What is the Root Cause of Blockchain Trilemma?

Self-Imposed Block Limits

In the wake of the 2008 financial crisis, Bitcoin emerged as a means to mitigate the adverse effects of traditional centralized institutions, such as banks, repeatedly leveraging their power to inflict widespread losses on the general public. Earlier blockchain networks developed their protocols focused on decentralization and security by essentially limiting their scalability potential. Without getting too technical, let me present an overview of what I mean.

Blockchain networks either have a fixed or varied block size depending on the traffic in the network. Larger block sizes allow more transactions to be included in a single block. This can increase the throughput of the network as each block can carry more transaction data. But that comes at the expense of decentralization and security. Bitcoin, for instance, have a smaller block size and longer block production time (complementing its resource intensive Proof-of-Work consensus mechanism) due to its rightful obsession with security. Further, Bitcoin does not have smart contract functionality as its community insists on keeping the oldest crypto no more than a peer-to-peer payment network and a store of value.

Ethereum, on the other hand, revolutionized blockchain technology through the introduction of a decentralized Virtual Machine (EVM) in 2015, which empowers the execution of smart contracts capable of automating diverse operations across multiple industries, all without the need for intermediaries. Public blockchains were simply not prepared to suddenly handle an industry that topped $1 trillion in 2021! This growth parallels the growth of big tech companies over the last 2 decades. But mind you, big tech didn’t have to concern themselves with self-imposed scaling limits or maintaining high decentralization.

Vertical Scalability and Sub-Optimal Architecture

Public networks, especially, with a smart contract platform preferred a varied block size and time. Together with lighter consensus mechanisms like Proof of Stake (PoS), they were able to process higher TPS while still capping the block size to limit malicious attack vectors and congestion. Remember, permissionless, open-source blockchains rely on public participants (nodes who use computer servers) to validate transactions. Therefore, consensus mechanism and self-limiting their architectures became indispensable.

Limiting TPS and finality eventually results in network congestion. In order to decongest and increase scalability, the networks opted to scale vertically similar to traditional systems. Scaling vertically includes significantly increasing the CPU, RAM and storage of existing servers which in turn mandate nodes to equip themselves with high hardware and/or staking requirements. At that point, only limited people can afford to run a node. When network congestion resulted in higher costs to operate a node coupled with the high demand for limited block space, nodes started prioritizing and processing transactions in order of highest fees paid instead of processing transactions in the order they were received leading to inflated fee markets. And since public blockchains record transactions in public ledgers which are open to anyone, a new unwelcome consequence began running amok – front-running and MEV. These disorders ultimately shot up the average transaction fees which has been a mainstay whenever blockchain networks saw high demand.

To be clear, there is no one silver bullet to solve the scalability trilemma. A practical solution lies in the human’s ages-old endeavor to research and develop innovative protocols. Together with numerous innovations and some of the proven novel techniques used by traditional networks, Shardeum demonstrates it can finally solve the trilemma problem and promote mass adoption of the Web3 ecosystem by individuals and institutions alike through an optimal architecture. Let’s see how!

How Does Shardeum Solve the Scalability Trilemma?

Sharding is not a new concept to the blockchain industry. On the contrary, it is a very well researched solution by top layer 1 blockchains such as Ethereum. In fact the tech industry have recognized sharding centralized databases as a prime solution for scalability in various applications for decades now. How does sharding help with scaling centralized networks? Below you will find an image – it’s a simplified illustration of sharding. In simple words, sharding breaks the job of validating and confirming transactions into small and manageable bits, or shards. While sharding is ultimately the best way to tackle the scalability issue, applying it to blockchain-based networks is not nearly as easy as applying it to centralized databases.

Sharding Shardeum blockchain — Source : devopedia.org/Oracle Docs 2020b, fig. 49-1

There are three main innovations, among others, that helps Shardeum to solve the blockchain trilemma which we will look into detail in the below paras.

Consensus Mechanism (Proof of Quorum)
Dynamic State Sharding
Autoscaling

Through Proof of Quorum, consensus and processing are done at the transaction level instead of the block level on Shardeum. Once you’ve done the consensus for the transactions, the data structure you use to store them does not matter. It matters for other networks because the data structure is tightly coupled with the consensus they employ. Once transactions are processed, the network batches them into groups and moves them to archive nodes.

And, through dynamic state sharding, the network will shard its state by evenly and dynamically distributing workload to all the nodes. This not only allows for parallel processing of transactions but also very low overhead for validator nodes as they will store only the state data of accounts they are handling. Just to be clear, dynamic state sharding is the most advanced and complex version of sharding. Dynamic state sharding will work hand in hand with Shardeum’s autoscaling feature allowing the network to automatically adjust the number and size of shards based on the current workload and historical data. And why are they important? Well, this is how Shardeum will get to maintain low transaction fees forever!

What is Dynamic State Sharding?

To begin with, the network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. This allows the system to pro-actively optimize performance and maintain high levels of scalability as it grows and evolves.

In contrast, static state sharding only enables a blockchain network to have static/pre-defined group of shards and transactions can only be processed sequentially after a minimum number of nodes join the network to create a new shard. There are two issues at play here. For one, this does not allow for growing dynamically in proportion to the varying demands in the network. Two, sequential processing slows down the network as a result of high latency arising from the time taken for new nodes to sync-up to the latest state of shards they’ve joined.

Technically, dynamic state sharding allows validator nodes on Shardeum to be assigned dynamic address ranges across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a different address range, but there is significant overlap between the addresses covered by nodes in respective shards creating sufficient redundancy. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus. This not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing.

Atomic Processing & Cross Shard Composability

Cross-shard communication allows for transactions to access and utilize data and state from different shards, enabling complex smart contract transactions to be executed in a sharded environment. Atomic composability is also important because it ensures that transactions are executed atomically, meaning that either all parts of a transaction are executed successfully or none of them. This is evident in complex transactions comprised of multiple, interconnected steps, each with its own objectives to achieve a common goal in the end. These steps must be executed successfully as a complete set to ensure the transaction achieves its intended outcome.

Without atomic composability, transactions could potentially fail or leave the blockchain in an inconsistent state, leading to security risks and reduced reliability. Shardeum will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the integrity and consistency of the blockchain.

Linear and Horizontal Scalability

With all the aforementioned elements in place, every node added to the network will increase the transaction throughput instantly – i.e. by simply adding more nodes from the network’s ‘standby‘ pool set during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly/horizontally. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security and low transaction fees irrespective of the demand in the network.

Consensus Mechanism

Now, let’s take a look into the consensus algorithm deployed on Shardeum. Transactions are verified/updated through an innovative consensus mechanism called PoQ or Proof-of-Quorum. Nodes in PoQ validate the transactions individually as soon as they are received in FCFS basis. This is followed by gossiping the transactions to all the other nodes within a consensus group on the network instead of every node on the network. Every node will be made aware that all other nodes within that group have knowledge about a specific transaction. This enables a trustless collection of votes (or quorum) in the form of receipts. And when there are more than 50% of the receipts, each transaction is confirmed/updated on the network with instant finality. Individual transactions, as such, will be grouped together before they are passed on to the archive nodes.

Shardeum will operate by combining both PoQ and PoS (Proof of Stake) for consensus to increase network security. Nodes aiming to operate on the network will need to stake a minimum number of network coins so potential misbehaviors are penalized. Shardeum will use PoS primarily as a sybil deterrence mechanism. The consensus algorithm will also play a key role in assigning a ‘node ID’ randomly to standby nodes before they join the network. Note, standby nodes are a type of nodes on Shardeum waiting for their turn to be validator nodes. Standby nodes’ use-case is not just limited to accommodating more capacity when the demand surges. With the help of node IDs, Shardeum will also auto-rotate the validator nodes with standby nodes regularly to make it even more difficult for bad actors to take over the network at any given point in time.

Autoscaling & Anyone Can Operate a Node

Autoscaling, essentially, enables the network to independently scale up or down its capacity proportional to the demand. Auto-scaling is crucial because when you build a network, it should ideally be able to self-govern the number of nodes it needs to properly incentivize. Shardeum’s protocol auto-detects the current capacity on the network and will work its way towards an ideal number of active validator nodes either by adding or removing from/to standby nodes pool and adjusting its shard size accordingly. This way, the network can conserve resources when the traffic is on the lower side.

Further, Shardeum will make it easy for average people to join the network and operate a node with minimal resources and compute requirements to maximize decentralization. As mentioned previously, the validator nodes on the network, who are responsible to process transactions sent by end users, would need to store only the current state within a shard while the historical data will be offloaded to archive nodes on the network. With reasonable staking amount and affordable hardware, you can also run a node on the network and keep it safe in return for network tokens. This helps the network to achieve another core objective — growing horizontally with its community while solving the blockchain trilemma.

In Conclusion

Shardeum’s guiding principle is OCC which is short for Open, Collaborative and Community Driven. Anyone anywhere in the world can join Shardeum and its movement to operate transparently and help reach its milestones. Further, the network is EVM-based which is developer friendly and moreover as a developer, you will never have to worry about rising gas fees again!

Shardeum co-founders make it clear that they not trying to compete with other L1 networks. Instead, Shardeum is keen to be the light at the end of tunnel for existing and future Web3 platforms by disrupting the way blockchain is utilized, or rather, under-utilized today. The project is hyper-focused to deliver a pivotal transformation in the Web3 movement and it recognizes today’s youth are willing to be part of an active generation and make steady progress towards a more equitable world.

Shardeum will also seek to learn from its peers, while inspiring them to introduce decentralization across our societies and industries at scale. With that, we will come closer to Web3’s larger agenda – when a trustless society becomes reality, where you don’t have to necessarily bank (no pun intended) on an intermediary to include you or exclude you. Shardeum will open-source its code in 2024 ahead of its mainnet. The project also has released its whitepaper in Q4 of 2023 which you can find here.