Explore the Future of Web3: Shardeum's Whitepaper Released!

Shardeum Solves Blockchain Trilemma

Shardeum Solves Blockchain Trilemma

Shardeum is an EVM based sharded L1 blockchain, and in this blog we unpack how its breakthrough protocol will solve scalability trilemma....

Back to top
  • Share

A new EVM-based layer 1 and community driven blockchain is on its way! How will it position itself among other layer 1 blockchains like Bitcoin, Ethereum, Cardano, Solana? How will it solve the challenges other smart contract platforms haven’t, retaining scalability, security and decentralization? Although you will get a clue from the name of the blockchain itself — Shardeum, there’s more than meets the eye. With a ground breaking technology and architecture, Shardeum is aiming to enable the adoption of dapps at a truly global scale. In this article, we will unpack the interesting mechanics behind the project.

As many of us know by now, established layer 1 networks like Bitcoin and Ethereum are slow and expensive. More recent layer 1 blockchains claim thousands of TPS (Transactions Per Second) with very low gas fees. However they sadly fall short in one or more of the following areas — scalability, security, decentralization and operational efficiency – which in reality results in repeated outages and very high transaction fees especially once TPS is throttled during peak demand.

This is where layer 2 solutions such as Polygon, for example, find their value propositions. Remember, layer 1 networks are operating on open source blockchain which allows such layer 2 solutions, that are faster and less expensive, to be built on top of them. Layer 2s have quite successfully enabled various use-cases that fuels today’s Web3 ecosystem. All the major utilities that you see in the blockchain industry were originally envisioned by layer 1 networks like Ethereum but they are now largely administered, so to say, through alternative scaling solutions like L2s and sidechains. Layer 2’s are more flexible in their operations that allows for higher scalability, but are not necessarily heavy-weight on security and/or decentralization. This is why experts often advocate addressing the scalability trilemma at the foundational (L1) level, as it will directly benefit any products built upon it.

Blockchain’s Scalability Trilemma

Ethereum’s famed co-founder, Vitalik Buterin, coined the term, “scalability trilemma”. The scalability trilemma highlights the challenge faced by public blockchains, where achieving a balance among the three fundamental properties of security, scalability, and decentralization simultaneously is difficult. Vitalik maintains that, at best, only 2 out of the 3 properties can coexist at a given time sacrificing the 3rd one. Since security is a key attribute on public blockchain platforms, they often make a choice between scalability and decentralization.

Now, even if the scalability issue is mitigated with layer 2 solutions and applications, there have only been incremental improvements to meet the demands of the industry. While we aim to replace Web2 with Web3 and rightfully so, the former provides a largely seamless user experience we’re all accustomed to at global scale. In order to successfully replace Web2, Web3 needs to scale up in terms of throughput capacity for masses to realize and adopt the embedded values of permissionless blockchain technology – decentralization, security and data ownership. A case in point is, Visa – a global payment company – processes an average of 4000 TPS while public blockchains (including the more recent ones) processes a max. of 400 TPS.

There’s More to the Scalability Issue

Consider a decentralized social media application. Today, social media apps host billions of activities and users on their networks. As users, we create and publish text, audio and video contents with significant ease on such platforms. A single person can perform numerous activities such as ‘like’, ‘share’, ‘subscribe’, and add emojis in response to contents published. Consider the throughput required for such activities produced by almost 7 billion people! The fact is, decentralized blockchains are not even close to offering the throughput centralized competitors offer today, and definitely struggle to offer a UX up to par with what society is accustomed to. This excerpt from a recent article on CoinDesk, properly summarizes the situation.

“How many blockchain games have lore or world building? Where is the community-driven content and engagement on social media? It’s difficult to even find YouTube or Twitch gameplay of crypto games that supposedly have hundreds of thousands of users.”

Zach Hungate/Coley Hungate

Since throughput directly impacts your ability to scale, it is no longer sustainable to overlook the scalability aspect in blockchains. Without the ability for blockchain networks to scale, businesses and consumers will have to continue relying on centralized platforms such as Facebook (Meta), Google and Twitter, to handle our Personally Identifiable Information (or PII). Such entities are known to have repeatedly abused the power of hosting and controlling users’ data, which has been a catalyst for decentralization.

What is the Root Cause of Blockchain Trilemma?

While issues such as data storage and network finality play a major role, which we will cover in the upcoming paras and sections, we would like to isolate the primary root causes before taking on the other issues one by one.

Self-Imposed Block Limits

In the wake of the 2008 financial crisis, Bitcoin emerged as a means to mitigate the adverse effects of traditional centralized institutions, such as banks, repeatedly leveraging their power to inflict widespread losses on the general public. Earlier blockchain networks made their protocols focused on decentralization and security by essentially limiting their scalability potential.

Blockchain platforms either have a fixed or varied block size and block production time depending on the traffic in the network. Larger block size means higher storage/TPS capacity, at the expense of lower decentralization and security. Bitcoin, for instance, have a smaller block size and longer block confirmation time (that complements its resource intensive Proof-of-Work consensus mechanism) due to its rightful obsession with security. Further, Bitcoin does not have smart contract functionality as its community insists on keeping the oldest crypto no more than a peer-to-peer payment network and a store of value. 

Ethereum, on the other hand, revolutionized blockchain technology through the introduction of a decentralized Virtual Machine (EVM) in 2015, which empowers the execution of smart contracts capable of automating diverse operations across real-world industries, all without the need for intermediaries. Public blockchains were simply not prepared to suddenly handle an industry that topped $1 trillion in 2021! To draw a parallel, it took big tech companies such as Apple and Google at least 2 decades to reach that figure. And mind you, they didn’t have to concern themselves with self-imposed scaling limits or user data privacy.

Vertical Scalability & Sub-Optimal Architecture

Networks, especially, with a smart contract platform preferred a varied or hybrid block size and time. Together with lighter consensus mechanisms like Proof of Stake (PoS), they were able to process higher transactions per second while at the same time capping the size and block rate to prevent centralization and attack vectors. Remember, permissionless, open-source blockchains rely on public participants (nodes) to validate transactions. Therefore, consensus mechanism and self-limiting their architectures became indispensable.

Limiting TPS and finality eventually results in network congestion. In order to decongest, the networks would need individual nodes to be equipped with high hardware and software requirements. At that point, only limited people can afford to run a node. When network congestion results in higher costs to operate a node coupled with the high demand for limited block space, nodes prioritize transactions in order of highest fees paid instead of processing transactions in the order they were received inevitably driving up the average transaction fees for users. Note, public blockchains record transactions in public ledgers which are open to anyone giving rise to this unwelcome consequence of front-running and MEV.

To be clear, there is no one silver bullet to solve the scalability trilemma. A practical solution lies in the human’s ages-old endeavor to research and develop innovative protocols. Together with numerous innovations and some of the proven novel techniques used by traditional networks, Shardeum demonstrates it can finally solve the trilemma problem and promote mass adoption of the Web3 ecosystem by individuals and institutions alike through an optimal architecture. Let’s see how!

How Does Shardeum Solve the Scalability Trilemma?

Sharding is not a new concept to the blockchain industry. On the contrary, it is a very well researched solution by top layer 1 blockchains such as Ethereum. In fact the tech industry have recognized sharding centralized databases as a prime solution for scalability in various applications for decades now. How does sharding help with scaling centralized networks? Below you will find an image – it’s a simplified illustration of sharding. In simple words, sharding breaks the job of validating and confirming transactions into small and manageable bits, or shards. While sharding is ultimately the best way to tackle the scalability issue, applying it to blockchain-based networks is not nearly as easy as applying it to centralized databases.

Sharding example
Source : devopedia.org/Oracle Docs 2020b, fig. 49-1

The good news with Shardeum is that the consensus and processing are done at transaction level and not at the block level. And, through dynamic state sharding, the network will shard its state by evenly and dynamically distributing workload to all the nodes. This not only allows for parallel processing of transactions but also very low overhead for validator nodes as they will store only the state data of accounts they are handling.

And why are they important? Well, this is how Shardeum will get to maintain low transaction fees forever. Just to be clear, dynamic state sharding is the most advanced version of transaction, network, static and adaptive state sharding employed by the latest sharded chains which runs into both inter-related and standalone problems such as high latency, vertical scaling (as opposed to linear scaling), sybil attack, slow finality and lack of cross shard composability. That said, dynamic state sharding is also the most complex way to shard the state of a network because Shardeum essentially shards the historical state of the network (and not just the current state)!

What is Dynamic State Sharding in Shardeum?

To begin with, the network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. Dynamic state sharding will work hand in hand with Shardeum’s auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the current workload and historical data. This allows the system to pro-actively optimize performance and maintain high levels of scalability as it grows and evolves.

Static state sharding only enables a blockchain network to have static/pre-defined group of shards and transactions can only be processed sequentially after a minimum number of nodes join the network to create a new shard. There are two issues at play here. For one, this does not allow for growing dynamically in proportion to the varying demand in the network. Two, sequential processing slows down the network as a result of high latency arising from the time taken for new nodes to sync-up to the latest state of shards they’ve joined.

Dynamic state sharding allows validator nodes on Shardeum to be assigned dynamic address ranges across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a different address range, but there is significant overlap between the addresses covered by nodes in respective shards. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus. This not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing.

Dynamic State Sharding vs Adaptive State Sharding

Adaptive state sharding is used by networks like MultiversX (formerly Elrond). It is more advanced than static state sharding where the shards are adapted to the current state of the network. This is done by changing the number of shards or the size of the shards based on factors such as the number of nodes in the network and the amount of data being processed. For example, if there is a sudden increase in the number of transactions, the network deploying adaptive state sharding can automatically create more shards to handle the increased load. This helps to ensure that transactions are processed quickly and efficiently, even during periods of high traffic with parallel transaction execution. Essentially, this type of sharding aims to combine the benefits of state, network, and transactions sharding. Adaptive state sharding, as a concept, renders load balancing on a reactive basis (and not on a proactive basis like dynamic state sharding) because the distribution of load happens based on the current state of the network and not on the historical state of the network.

Dynamic state sharding takes adaptive state sharding to another level by helping a network to distribute the load more efficiently and on a pro-active basis. It shards the state, network and transactions evenly based on not just the current state of the network but also based on the historical states of the blockchain (which includes latest state, and transaction history). For example, a dynamic state sharding algorithm might track the average number of transactions processed by each shard over time. This information could then be used to predict which shards are likely to become overloaded in the future. The algorithm could then take steps to pre-emptively balance the load across the shards. Another example is, the algorithm might notice that certain types of transactions are typically processed by specific shards. This information could then be used to proactively route new transactions to the appropriate shards

Atomic Processing & Cross Shard Composability

Cross-shard communication allows for transactions to access and utilize data and state from different shards, enabling complex transactions and smart contracts to be executed in a sharded environment. Atomic composability is also important because it ensures that transactions are executed atomically, meaning that either all parts of the transaction are executed successfully or none of them are. Without atomic composability, transactions could potentially fail or leave the blockchain in an inconsistent state, leading to security risks and reduced reliability. Shardeum will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the integrity and consistency of the blockchain.

Linear Scalability on Shardeum

With this in place, every node added to the network will increase the transaction throughput instantly. So, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security and constant transaction fees irrespective of the demand in the network.

Side-note: Shardeum’s underlying protocol, Shardus, demonstrated linear scalability in a sharded environment in its Q3 2021 update event, where a network of 1000 nodes running on AWS t3.medium hardware was shown to reach 5000 TPS of signed coin transfer transactions across shard. Shardeum, being a public smart contract platform, is aiming for 1 TPS or more per node which will be a huge breakthrough for the Web3 ecosystem. Shardeum’s idea is to mobilize millions of nodes when needed, which can help process over 100,000 TPS to realistically host a ton of dapps providing products and services to billions of users and eliminate middlemen in the process.

Consensus Algorithm on Shardeum

Now, let’s take a look into the consensus algorithm deployed on Shardeum. Transactions are verified/updated through an innovative consensus mechanism called PoQ or Proof-of-Quorum. Nodes in PoQ validate the transactions individually as soon as they are received in FCFS basis. This is followed by gossiping the transactions to all the other nodes within a consensus group on the network instead of every node on the network. Every node will be made aware that all other nodes within that group have knowledge about a specific transaction. This enables a trustless collection of votes (or quorum) in the form of receipts. And when there are more than 50% of the receipts, each transaction is confirmed/updated on the network with instant finality. Individual transactions, as such, will be grouped together before they are passed on to the archive nodes.

Shardeum will operate by combining both PoQ and PoS (Proof of Stake) for consensus to increase network security. Nodes aiming to operate on the network will need to stake a minimum number of network coins so potential misbehaviors are slashed. The consensus algorithm will also play a key role in assigning a ‘node ID’ randomly to validator nodes before they join the network. On Shardeum, there will be a third set of nodes apart from validator and archive nodes, called ‘standby nodes’. These are nodes waiting for their turn to be validator nodes. Standby nodes use-case on Shardeum is not limited to just accommodating more capacity when the demand surges. With the help of node IDs, the network will constantly auto-rotate the validator and standby nodes to make it even more difficult for bad actors to take over at any given point in time.

Autoscaling & Anyone Can Operate a Node

Auto-scaling, essentially, enables the network to independently scale up or down its capacity proportional to the demand. Auto-scaling is crucial because when you build a network, it should ideally be able to self-govern the number of nodes it needs to properly incentivize. Shardeum’s protocol auto-detects the current capacity on the network and will work its way towards an ideal number of active validator nodes either by adding or removing from/to standby nodes pool and scaling its shard size accordingly.

Further, Shardeum will make it easy for average people to join the network and operate a node with minimal resources and compute requirements to maximize decentralization. As mentioned previously, the validator nodes on the network, who are responsible to process transactions sent by end users, would need to maintain only the current state within a shard (pertaining to those transactions they handle) while the historical data will be offloaded to archive nodes on the network. With reasonable staking amount and affordable hardware, you can also run a node on the network and keep it safe in return for network tokens. This helps the network to achieve another core objective — growing horizontally with its community.

In Conclusion

Shardeum’s guiding principle is OCC which is short for Open, Collaborative and Community Driven. Anyone anywhere in the world can join Shardeum and its movement to operate transparently and help reach its milestones. Further, the network is EVM-based which is developer friendly and as a developer, you will never have to worry about rising gas fees again to enrich UX of the dapps you build on Shardeum.

Shardeum co-founders make it clear that they not trying to compete with other L1 networks. Instead, Shardeum is keen to be the light at the end of tunnel for existing and future Web3 platforms by disrupting the way blockchain is utilized, or rather, under-utilized today. The project is hyper-focused to deliver a pivotal transformation in the Web3 movement and it recognizes today’s youth are willing to be part of an active generation and make steady progress towards a more equitable world.

Shardeum Foundation, a non profit organization behind the project, will also seek to learn from its peers, while inspiring them to decentralize industries at a global scale and empower dapps ecosystem to prepare for serving billions of users within this decade. With that, we will come closer to Web3’s larger agenda – when a trustless society becomes reality, where you don’t have to necessarily bank (no pun intended) on an intermediary to include you or exclude you. Shardeum will open-source its code and release its whitepaper in 2023. You can find the premise of the protocol and its technology documented in its litepaper.

Popular Searches

Layer 1 Crypto Projects | EVM Wallet Address | Ethereum That Are Compatible With The EVM | Types of DDos Attack | What is AMM in Crypto | Cryptocurrency Career Opportunities | Decentralized Identifiers | Advantages of Consortium Blockchain | What is Cloud Mining  |  Best Place to Mint NFT  |  What is Automated Market MakerHow to Mint CryptoSB Token  |  What is Web 3.0  |  Scalability Trilemma  |  NFT Use Cases | What is a Blockchain Node  |  What is a Blockchain Bridge  |  What are Peer to Peer Payments  |  Pros and Cons of Blockchain Technology | What is Typescript | What is Truffle in Blockchain | What is Node JS | What is Foundry | THNDR Games



5
The Shard

Sign up for The Shard community newsletter

Stay updated on major developments about Shardeum.

  • Share