A new EVM-based layer 1 and community driven blockchain is on its way! How will it position itself among other layer 1 blockchains like Bitcoin, Ethereum, Cardano, Solana? How will it solve the challenges other smart contract platforms haven’t, retaining scalability, security and decentralization? Although you will get a clue from the name of the blockchain itself — Shardeum, there’s more than meets the eye. With a ground breaking technology and architecture, Shardeum is aiming to enable the adoption of DApps at a truly global scale. In this article, we will unpack the interesting mechanics behind the project.
As many of us know by now, established layer 1 networks like Bitcoin and Ethereum are currently slow and expensive. More recent layer 1 blockchains claim thousands of TPS (Transactions Per Second) with very low gas fees. However they sadly fall short in one or more of the following areas — scalability, security, decentralization and operational efficiency – which in reality results in repeated outages and very high transaction fees especially once TPS is throttled during peak demand.
This is where layer 2 solutions such as Polygon, for example, find their value propositions. Remember, layer 1 networks are operating on open source blockchain which allows such layer 2 solutions, that are faster and less expensive, to be built on top of them. Layer 2s have quite successfully enabled various use-cases that fuels today’s Web3 ecosystem. Note, all the major utilities that you see in the blockchain industry were originally envisioned by layer 1 networks like Ethereum but they are now largely administered, so to say, through alternative scaling solutions like L2s as a result of L1s inability to overcome scalability trilemma.
Table of Contents
Blockchain’s Scalability Trilemma
Ethereum’s famed co-founder, Vitalik Buterin, coined the term, “scalability trilemma”. The scalability trilemma refers to blockchain’s inability to have all 3 of its fundamental properties — security, scalability and decentralization — to coexist at the same time. Vitalik maintains that, at best, only 2 out of the 3 properties can coexist at a given time sacrificing the 3rd one. Since security is a key attribute on public blockchain platforms, they often make a choice between scalability and decentralization.
Now, even if the scalability issue is mitigated with layer 2 solutions and applications, there have only been incremental improvements to meet the demands of the industry. While we aim to replace Web2 with Web3 and rightfully so, the former provides a largely seamless user experience we’re all accustomed to at global scale. In order to successfully replace Web2, Web3 needs to scale up in terms of throughput capacity to enable mass adoption and realization of the embedded values that comes with the design-build of blockchain – high security, privacy and decentralization.
There’s More to the Scalability Issue
Consider a decentralized social media application. Today, social media apps host billions of activities and users on their networks. As users, we create and publish text, audio and video contents with significant ease on such platforms. A single person can perform numerous activities such as ‘like’, ‘share’, ‘subscribe’, and add emojis in response to contents published. Consider the throughput required for such activities produced by almost 7 billion people! The fact is, even the more recent layer 1 or 2 networks are not close to offering the throughput centralized competitors offer today, and definitely struggle to offer a UX up to par with what society is accustomed to. This excerpt from a recent article on coindesk, properly summarizes the situation.
“How many blockchain games have lore or world building? Where is the community-driven content and engagement on social media? It’s difficult to even find YouTube or Twitch gameplay of crypto games that supposedly have hundreds of thousands of users.”
Zach Hungate/Coley Hungate
Without the ability for blockchains to scale, businesses and consumers will have to continue relying on centralized platforms such as Google, Twitter, or Facebook (Meta) to handle our Personally Identifiable Information (or PII). Centralized entities are known to have repeatedly abused the power of hosting and controlling users’ data, which has been a catalyst for decentralization.
What is the Root Cause?
Most of the current distributed ledger protocols have self-imposed scaling limits mainly because their design-build does not allow for optimal security when these scaling limits are breached. The protocols of, say, Bitcoin or Ethereum, or any other typical blockchain today, stipulate maximum size for their blocks and the rate at which blocks are produced thereby setting a ceiling on the rate at which their network can process transactions.
Limiting TPS eventually results in network congestion. In order to decongest, the networks would need individual nodes to be equipped with high hardware and software requirements. When network congestion results in higher costs to operate a node, nodes prioritize transactions in order of highest fees paid instead of processing transactions in the order they were received driving up the average transaction fees for users.
How Does Shardeum Solve the Trilemma?
Sharding is not a new concept to the blockchain industry. On the contrary, it is a very well researched solution by top layer 1 blockchains such as Ethereum. In fact the tech industry has recognized sharding centralized databases as a prime solution for scalability in various applications for decades now. How does sharding help with scaling centralized networks? Below you will find an image – it’s a simplified illustration of sharding. In simple words, sharding breaks the job of validating and confirming transactions into small and manageable bits, or shards. While sharding is ultimately the best way to tackle the scalability issue, applying it to blockchain-based networks is not nearly as easy as applying it to centralized databases.
The good news with Shardeum is that the consensus and processing are done at transaction level and not at the block level. And, through dynamic state sharding, the network will shard its state by evenly and dynamically distributing compute workload, storage, and bandwidth among all the nodes. This not only allows for parallel processing of transactions but also very low overhead for validator nodes as they will store only the state data of transactions they are involved in.
And why are they important? Well, this is how Shardeum will get to maintain low transaction fees for developers and end users perpetually. Just to be clear, dynamic state sharding is the most advanced version of state, transaction/network and static state sharding employed by more recent sharded chains which runs into both inter-related and standalone problems such as high latency, vertical scaling (as opposed to linear scaling), sybil attack, weak finality and lack of cross shard composability. That said, dynamic state sharding is also the most complex way to shard the state of a network.
What is Dynamic State Sharding in Shardeum?
To begin with, the network won’t have a static group of nodes as fixed shards. Nodes on Shardeum are free to move around and accommodate more data as dynamic shards. Dynamic state sharding will work hand in hand with Shardeum’s auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the current workload. This allows the system to optimize performance and maintain high levels of scalability as it grows and evolves.
Static state sharding only enables a blockchain network to have static/pre-defined group of shards and transactions can only be processed sequentially after a minimum number of nodes join the network to create a new shard. There are two issues at play here. For one, this does not allow for growing dynamically in proportion to the varying demand in the network. Two, sequential processing slows down the network as a result of high latency arising from the time taken for new nodes to sync-up to the latest state of shards they are involved in.
Important Considerations (more details of dynamic state sharding will follow in the paras below and it is not considered for this section yet)
- Atomic and cross shard composability: Transaction and Network sharding are reliable here. State and Static state sharding, on a standalone basis, does not guarantee them
- Linear scalability: None of the above sharding methods allow for true linear scalability and therefore, true decentralization. State and Static state sharding, though, is superior in utilizing network resources more efficiently when compared to transaction and network sharding. The former has an edge here with higher scalability and performance
- Throughput, Latency and Gas Fees: Due to the inherent characteristics of the above 4 sharding techniques, they typically result in limited scalability, high latency and rising gas fees during traffic spike
- Security: Security takes a hit for every shortfall in any of the above metrics
Dynamic state sharding allows validator nodes on Shardeum to be assigned dynamic address ranges across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a different address range, but there is significant overlap between the addresses covered by nodes in respective shards. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus. This not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing. Shardeum will further ensure cross shard composability.
Atomic Processing & Cross Shard Composability
Cross-shard communication allows for transactions to access and utilize data and state from different shards, enabling complex transactions and smart contracts to be executed in a sharded environment. Atomic composability is also important because it ensures that transactions are executed atomically, meaning that either all parts of the transaction are executed successfully or none of them are. Without atomic composability, transactions could potentially fail or leave the blockchain in an inconsistent state, leading to security risks and reduced reliability. Shardeum will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the integrity and consistency of the blockchain.
Linear Scalability on Shardeum
With this in place, every node added to the network will increase the transaction throughput instantly. So basically, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably including throughput, decentralization, security and constant transaction fees irrespective of the demand in the network.
That’s how Shardeum’s underlying protocol, Shardus, was able to demonstrate 500 TPS with 100 nodes over the last 3 years. Shardeum is aiming to enable up to 1 TPS or more per node which will be a huge breakthrough for the Web3 ecosystem. Even with 2k active nodes on existing blockchain networks, we can only see them process an average of 350 TPS at best when Web2 peers like PayPal and Visa processes an average of 5000 TPS every day. Shardeum’s idea is to mobilize millions of nodes eventually which can process over 1 million TPS to realistically host a ton of DApps providing products and services to billions of users and eliminating middlemen taking advantage of our data/privacy.
Consensus Algorithm on Shardeum
Now, let’s take a look into the consensus algorithm deployed on Shardeum. Transactions are verified/updated through an innovative consensus mechanism called PoQ or Proof-of-Quorum. As opposed to PoW and other consensus algorithms in use today, nodes in PoQ validate the transactions individually as soon as they are received in FCFS basis. This is followed by gossiping the transactions to all the other nodes within a consensus group on the network instead of every node on the network.
Every node, essentially, will know that every other node in the consensus group knows about a particular transaction. This enables a trustless collection of votes (or quorum) in the form of receipts. And when there are more than 50% of the receipts, each transaction is confirmed/updated on the network. Individual transactions, as such, will be grouped together before they are passed on to the archive nodes.
Shardeum will operate by combining both PoQ and PoS (Proof of Stake) for consensus to increase network security. Nodes aiming to operate on the network will need to stake a minimum number of network coins so potential misbehaviors are slashed.
The consensus algorithm will also play a key role in assigning a ‘node ID’ randomly to validator nodes before they join the network. On Shardeum, there will be a third set of nodes apart from validator and archive nodes, called ‘standby nodes’. These are nodes waiting for their turn to be validator nodes. Standby nodes use-case on Shardeum is not limited to just accommodating more capacity when the demand surges. With the help of node IDs, the network will constantly auto-rotate the validator and standby nodes to make it even more difficult for bad actors to take over at any given point in time.
Autoscaling & Anyone Can Operate a Node
Auto-scaling, essentially, enables the network to independently scale up or down its capacity proportional to the demand. Auto-scaling is crucial because when you build a network, it should ideally be able to self-govern the number of nodes it needs to properly incentivize. Shardeum’s protocol auto-detects the current capacity on the network and will work its way towards an ideal number of active validator nodes either by adding or removing from/to standby nodes pool and scaling its shard size accordingly.
Further, Shardeum will make it easy for average people to join the network and operate a node with minimal resources and compute requirements to maximize decentralization. As mentioned previously, the validator nodes on the network, who are responsible to process transactions sent by end users, would need to maintain only the current state within a shard (pertaining to those transactions they handle) while the historical data will be offloaded to archive nodes on the network. With reasonable staking amount and affordable hardware, you can also run a node on the network and keep it safe in return for network tokens. This helps the network to achieve another core objective — scaling horizontally while reinforcing its security quotient.
In Conclusion
Shardeum’s guiding principle is OCC which is short for Open, Collaborative and Community Driven. Anyone anywhere in the world can join Shardeum and its movement to operate transparently and help reach its milestones. You can find the roadmap of the project on the website here. Further, the network is EVM-based which is developer friendly and as a developer, you will never have to worry about rising gas fees again to enrich UX of the DApps you build on Shardeum.
Shardeum co-founders maintain that they not trying to compete with other L1 networks. Instead, Shardeum is keen to be the light at the end of tunnel for existing and future Web3 platforms by disrupting the way blockchain is utilized, or rather, under-utilized today. The project is hyper-focused to deliver a pivotal transformation in the Web3 movement and it recognizes today’s youth are willing to be part of an active generation and make steady progress towards a more equitable world.
Shardeum Foundation, a non profit organization behind the project, will also seek to learn from its peers, while inspiring them to decentralize industries at a global scale and empower DApps ecosystem to prepare for serving billions of users in this decade. With that, we will come closer to Web3’s larger agenda – when a trustless society becomes reality, where you don’t have to necessarily bank (no pun intended) on an intermediary to include you or exclude you. Shardeum will open-source its code and release its whitepaper in Q1 of 2023. You can find the premise of the protocol and its technology documented in its litepaper.
Popular Searches
Layer 1 Crypto Projects 2022 | EVM Wallet Address | Ethereum That Are Compatible With The EVM | Types of DDos Attack | What is AMM in Crypto | Cryptocurrency Career Opportunities | Decentralized Identifiers | Advantages of Consortium Blockchain | What is Cloud Mining | Best Place to Mint NFT | What is Automated Market Maker | How to Mint Crypto | SB Token | What is Web 3.0 | Scalability Trilemma | NFT Use Cases 2022 | What is a Blockchain Node | What is a Blockchain Bridge | What are Peer to Peer Payments | Pros and Cons of Blockchain Technology
Opinions expressed in this publication are those of the author(s). They do not necessarily purport to reflect the opinions or views of Shardeum Foundation.
