What is Dynamic State Sharding by Shardeum?
Shardeum uses dynamic state sharding to scale its L1 network linearly and maintain operational efficiency ensuring very low fees for end...
Shardeum uses dynamic state sharding to scale its L1 network linearly and maintain operational efficiency ensuring very low fees for end...
|
Getting your Trinity Audio player ready...
|
According to a recent report by crypto.com, the total number of global cryptocurrency owners may grow by 3x in 2022. We are really looking at billion+ users sooner rather than later. When we look at existing networks and their capabilities though, we can see they are all falling agonizingly short of mass adoption. The current layer one (L1) blockchains are simply not designed to serve at such a large scale yet. That said, there have been no dearth of L1 blockchains attempting to scale up in the recent past. Even those who are successful at it today, end up compromising decentralization and/or the security tenets of blockchain technology. Other measures like increasing the block sizes or having layer two (L2) solutions come with their own problems.
We are also realizing that vertical scaling with more hardware resources is not sufficient to take blockchain towards mass adoption. It is safe to say that permissionless blockchains have rendered Moore’s law unpredictable in the world of distributed computing networks. Moore’s law observes that the processing power of computers doubles approximately every two years, leading to exponentially greater capacity yet costs reduce by half. This lays the ground for our next question. How can sharding be the panacea to the overarching problem in blockchain technology known as “scalability trilemma” – where it retains all the three main properties – security, decentralization and scalability – simultaneously at any given time? While at the same time ensuring costs and network congestion remains negligible even when the ecosystem grows exponentially!
Before we get into the details of dynamic state sharding, let’s answer the basic question – what is sharding?
Sharding is a method used to scale networks by dividing the overall workload across multiple servers or nodes. Each shard, or partition, handles a fraction of the network’s total computational and storage requirements, thereby distributing the transaction load. This division allows each node to manage and process transactions specific to its shard, enhancing efficiency and speed. By breaking down tasks into more manageable components, sharding helps achieve faster transaction processing and consensus within smaller subsets of the entire network. This technique is particularly valuable in distributed systems like blockchains, where it can significantly increase throughput and reduce latency.
To maintain clarity and brevity, we will take a quick glance on various types of sharding employed in the blockchain industry so far without getting into details. However if you’re keen to delve deeper into each type, we recommend exploring this detailed blog for a comprehensive understanding.
Shardeum is not the first blockchain network to leverage sharding. A good example is Zilliqa, which implemented network-level sharding, specifically concentrating on transaction sharding since 2019. This approach enables parallel transaction processing across various shards, substantially boosting throughput and minimizing latency. Another example is NEAR protocol that has implemented the most advanced type of sharding (until recently) called state sharding. It utilizes a sharding mechanism designed to split the network’s state into multiple shards, allowing each shard to process transactions and maintain a portion of the state independently. This approach enhances scalability and transaction speed by distributing the workload across multiple nodes, making it more manageable and efficient.
But what makes sharding on Shardeum different? Dynamic state sharding! What is it? Let’s find out 🙂
Dynamic state sharding is designed for flexibility and adaptability, seamlessly adjusting to evolving conditions within the blockchain ecosystem. It is the most advanced and complex way to shard a distributed network because it shards the State, Network, and Transactions and that too dynamically, as opposed to, in a pre-determined or pre-scheduled way.
Dynamic state sharding is engineered to accommodate seamless addition or removal of shards both, in response to new developments in the network and anticipating load changes based on historical state of the network. It facilitates optimal allocation and routing of resources, enhancing overall efficiency. The key advantage lies in its ability to scale the network efficiently without disrupting ongoing operations or diminishing performance.
In static state sharding, a blockchain network is divided into a fixed, pre-defined number of shards (also called fixed account spaces). When demand increases, a new shard is created only after a minimum number of nodes join the network. Consensus and processing typically occur at the block level, with transactions batched into blocks and processed together. While state sharding is more efficient than transaction and network sharding, it has limitations.
First, it doesn’t dynamically scale with demand, which leads to inefficient resource utilization, as shards may be either underloaded or overloaded, reducing the overall network throughput due to bottlenecks in processing capacity. Second, the network experiences high latency due to the time required for new nodes to synchronize with the latest state of the shards they join disrupting the current operations. And ultimately, with the block-level consensus, achieving parallel transaction execution becomes challenging.
Dynamic state sharding by Shardeum enable nodes to be assigned dynamic address range (or account spaces) across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a unique address range, although with a significant overlap between the addresses covered for sufficient redundancy. And since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block-level consensus. This not only reduces the time to process the transaction even if it affects multiple shards but also ensures atomic processing.
Through dynamic state sharding, the network will shard the state, network and transactions evenly based on not just the current state of the network but also based on the historical states of the blockchain (which includes latest state and transaction history). For example, a dynamic state sharding algorithm might track the average number of transactions processed by each shard over time. This information could then be used to predict which shards are likely to become overloaded in the future. The algorithm could then take steps to pre-emptively balance the load across the shards. Another example is, the algorithm might notice that certain types of transactions are typically processed by specific shards. This information could then be used to proactively route new transactions to the appropriate shards.
Shardeum’s dynamic state sharding, further, will work hand in hand with its other innovative feature called auto-scaling, enabling the network to automatically adjust the number and size of shards based on the current workload. This allows the system to optimize performance and maintain high levels of scalability as it grows and evolves.
Through dynamic state sharding, every node added to the network will increase the transaction throughput instantly. By simply adding more nodes from the network’s ‘standby’ validator pool during increasing demand, the TPS will increase proportionally making Shardeum the first blockchain network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably including scalability, decentralization, security, and low transaction fees irrespective of the demand in the network.
Shardeum pioneers a new approach with sharding. With dynamic state sharding on top of its EVM compatibility, Shardeum will be truly capable of handling a multitude of dapps and layer 2 solutions across industries and consumers with optimum performance. The network will indeed maneuver in unchartered territories in the blockchain world as it aims to better even Web2 networks in terms of throughput capacity while retaining high security and decentralization that Web3 networks are known for. These are really exciting times ahead for the Web3 world. So let’s stay tuned 🙂
Adaptive state sharding adjusts the size of its shards based on the current state of the network. This is done by changing the number of shards based on factors such as the number of nodes in the network and the amount of data being processed. For example, if there is a sudden increase in the number of transactions, the network deploying adaptive state sharding can automatically create more shards to handle the increased load.
There are five main types of sharding in blockchain technology: Network Sharding, Transaction Sharding, State Sharding, Adaptive State Sharding, and Dynamic State Sharding. Each type aims to enhance the scalability and efficiency of blockchain networks by dividing and processing data in different ways.
Dynamic state sharding enhances adaptive state sharding by distributing network load proactively and efficiently. It balances state, network, and transactions using both current and historical blockchain data. By analyzing transaction trends over time, it can anticipate and manage shard overloads and route transactions to optimal shards. Adaptive state sharding adjusts the size of its shards on a reactive basis taking into account the current state of the network.
Layer 1 Crypto Projects | What is a DDoS Attack | DAO Guide | Cloud Mining Platform | How to Mint NFTs | Web3.0 Games | How Proof of Work Works | How to Buy Land in Metaverse | NFT Use Cases | How Altcoins Work | Enterprise Grade Blockchain | Main Features of Blockchain | How to Avoid Crypto Scams | Basics of DeFi | Best Dapp Browsers | What is Miner Extractable Value | What is White Paper in Cryptocurrency | Best Crypto Community | How to Make NFTs to Sell | Web 3 Learning | What Is intrinsic Value | Defi Hacks | Blockchain Node Providers | Difference Between Cryptocurrency and Blockchain
Opinions expressed in this publication are those of the author(s). They do not necessarily purport to reflect the opinions or views of Shardeum foundation.
The contributing author is a Web 3 and blockchain enthusiast based out of Gurgaon, India. He provides strategic research-based inputs, analysis, and content support for entities in this space. You can follow and contact him on Twitter
Learn more about different types of node on Shardeum, what role archive nodes play on the network and how to run an archive...
Shardeum’s transaction level consensus, dynamic state sharding and linear scalability will ensure the network not only achieves parallel transaction execution but also atomic and cross shard...
Shardeum, with its pioneering protocol, will achieve immediate finality and low latency through transaction level consensus & parallel execution of...
