How Shardeum Achieves Low Latency Enabling Dapps & Users With a Smooth UX

Introduction

Latency, in general, is the time taken for data to travel from its source to its destination. Network latency can be described as the delay or lag between the time data is sent from one point and is received at its destination. Long distance, multiple device hops, poor infrastructure among other factors result in higher-than desired latency. 

What is Low Latency and Why is it Important?

Low latency refers to a system’s capability to minimize the delay between data transmission and reception as much as possible, which can make all the difference in user experience. Reducing latency is an essential aspect of modern-day technology and is being widely optimized for various industries including blockchain networks. Whether it’s gaming, financial trading, or telecommunications, low latency is the crucial ingredient real-time applications/systems require for immediate response times. 

For instance, low latency is essential for high-frequency and algorithmic trading in the financial industry, where even milliseconds can significantly affect a trader’s position. Online gaming is another area where low latency is crucial, allowing players to react quickly and effectively in fast-paced games. This guide will explore how low latency benefits smart contract platforms, and how Shardeum, as an L1 network, will maintain very low latency, which has historically been a challenge for the blockchain industry.

Latency, Finality & Scalability in Blockchain Networks 

Latency, in the context of blockchains, is the total turnaround time between submitting a valid transaction to a blockchain network and for the network to process/confirm the transaction. Keep in mind that latency and finality work very closely in blockchains but they are not the same. Finality refers to the irreversibility of a confirmed transaction added to the blockchain network. So finality time is the total turnaround time between submitting a valid transaction to a blockchain network and for the network to finalize the transaction. At that point, the transaction is deemed final and can no longer be changed. Therefore, very low latency and finality is crucial to not only scale up the network but also process a very high number of transactions per second (TPS).

Likewise, latency and scalability are inter-related but they are not the same. Low latency focuses on minimizing the time it takes to process individual transactions, while high throughput or scalability (typically measured in TPS) emphasizes the ability of the blockchain network to handle a large volume of transactions simultaneously.

Latency Time – Web2 vs Web3 Networks

Centralized networks (that include private blockchains) typically have lower latency times compared to decentralized blockchain networks. The fundamental reason for this latency difference is the architecture and processing mechanism used by these two different systems. Centralized servers are controlled by a single entity or organization, which means they can process and validate transactions quickly as there is no need for multiple parties to agree on the validity of each transaction.

Latency in centralized systems is mainly dependent on factors like the speed and capacity of the server’s hardware, the efficiency of the software, and the network conditions. Along with other techniques like load balancing, and network maintenance, such entities solve the lag times by scaling vertically i.e. by increasing the processing power of their system hardware.

How is Higher Latency Impacting the Blockchain Industry Today?

1. Solving Scalability Trilemma Becomes Challenging

Decentralized networks distribute workload to numerous individual nodes/servers globally and then achieve consensus on transaction validity followed by recording transactions in the network’s distributed ledger/blockchain. The consensus algorithm and other self imposed scaling limits adopted are crucial for public networks because they are fundamental to the blockchain’s core USPs – security and decentralization. 

Blockchain platforms vary their block sizes depending on network traffic, and craft a suitable consensus algorithm accordingly. Larger blocks offer higher TPS but risk centralization and attacks. Bitcoin, with its fixed smaller blocks, prioritizes security over smart contracts. Ethereum, on the other hand, expanded blockchain’s potential by enabling smart contracts and more commercial use cases. Networks as such prefer a varied or hybrid block size allowing themselves to process marginally more transactions per second while still capping the size limit to prevent centralization and attack vectors.

As a drawback, this inherently introduces latency and slow speed that has rendered decentralized blockchains less suitable for real time applications and utilities like online gaming, supply chains, telecommunication, and healthcare so far. That’s why there is a race among the latest networks to solve the blockchain trilemma – where they will be capable of maintaining security, decentralization and scalability at all times.

2. Latency & Finality is Inversely Proportional to Throughput

A key aspect here is that the relationship between latency and throughput is inversely proportional. This means that as latency and finality time increases, throughput decreases curtailing a network’s ability to process a high amount of transactions per second (TPS). This has been the case with traditional systems and is indeed applicable for blockchain-based networks as well, at least at a more foundational level. 

Blockchain network latency, though, has more moving parts such as smart contracts and consensus mechanisms in its effort to provide real world solutions without an intermediary. They rely on a very high level of automation through smart contract coding. That said, low latency plays a more significant and active role than say, bandwidth requirements in determining the scalability and throughput of a blockchain network. In fact, a blockchain network can do well with a lower data bandwidth provided it has an optimal architecture built into it that ensures network finality is as low as possible. 

3. High and Unpredictable Transaction Costs

The ultimate result of vertical scalability, low TPS and the inability to solve scalability trilemma is high transaction costs/gas fees especially when the traffic on a network increases. Higher latency further makes it that much more expensive and slower for users and dapp developers to interact with a smart contract platform on account of increased computation and communication costs.

Despite the promise of decentralization, many of the tools and services used to enable decentralized ecosystems today still function in a largely centralized way. To prevent network congestion, outages, and poor user experience, public blockchains often end up scaling vertically themselves by increasing the computing power of their nodes. However, this approach can make it financially impractical for everyday users to operate nodes on these networks, ultimately increasing transaction costs for both users and developers creating dapps.

4. MEV and Front-Running Risks

Public blockchains record transactions transparently. And on top of self-imposed scaling limits, validators found themselves incentivized to prioritize high-fee transactions as an unintentional consequence to limit network congestion and discourage high traffic. However, validators rarely deviate from network protocols enough to manipulate the entire market due to the prohibitive cost (like slashing) associated with such actions and the robust consensus mechanism itself. 

Things get out of hand when the same front-running is done by malicious actors where they can see a transaction (large ones typically) before it is confirmed and then place their own transaction ahead of it to profit from the price difference. Here, they not only seek to gain an unfair advantage over other users but also essentially manipulate the larger market causing losses to the general public. Studies by Flashbots and University of California, suggests the cost paid by average Ethereum users due to front-running by malicious actors has amounted to more than $2 billion in 2023 alone. The collateral damage done to the crypto market could be a lot larger.

Low latency, and ultimately, immediate finality, becomes all the more important here by making it more difficult for bad actors to see transactions before they are confirmed on blockchains. This is because low latency blockchains can process transactions more quickly, allowing less time for attackers to see and exploit transactions. When the time to confirm transactions fluctuates based on the demand in the network, getting the most out of even a timestamp based autonomous ordering protocol will be impractical.

5. Bad User Experience (UX)

As you can imagine, achieving mass adoption for blockchains and the dapps built on them hinges on a positive user experience (UX) and minimal transaction costs. And we have discussed how the current ecosystem lacks them today on account of  blockchain’s design-build. And as indicated above, centralized systems realized this a long time back and have since introduced various latency improving techniques including CDNs, edge computing, constant deployment of powerful data centers across the world and load balancing.

The objective is simple – to bring compute data and storage closer to the end user making the response and load time instant in a world increasingly craving for real time data processing. Because that is the only way they can fulfill the requirements of today’s industries ranging from AI, autonomous vehicles, to algorithmic trading.

High latency and slow finality, for instance, makes the decentralized crypto exchanges (DEXs) operating through smart contracts less efficient than centralized exchanges (CEXs) operating through centralized protocols. As mentioned, high latency means it can take longer for transactions to be processed and confirmed while slow or weak finality means the confirmed transactions have a chance to be reversed later. This can be a problem for DEX traders who need to make quick trades with instant settlement. Further, it introduces frequent slippages leading to bad user experiences. And bad UX, in turn, leads to systemically lower liquidity in DEX inducing high volatility in cryptocurrency prices. Doesn’t this sound like the current state of the crypto economy?

6. Security Attacks

L1 blockchains prioritize high security and decentralization, making them resilient to sophisticated attacks we see today in Web3. However, the same level of security isn’t always guaranteed for solutions and dapps built on L1 blockchains, despite leveraging their underlying security. This is because these solutions are designed with end-users and UX in mind, often compromising decentralization and the robust security of L1 blockchains.

Security threats come in the form of phishing scams, malware attacks, and DoS attacks among many others. DoS and DDoS attacks, in particular, have become a thorn even for newer L1 chains focused on scalability and low gas fees. These attacks involve flooding a network with many small transaction requests inexpensively, typically using bots, causing it to become slow or entirely unreachable for legitimate users. Low latency can help mitigate this risk by making it harder for attackers to overwhelm the network, as they would need to send more transactions.

Let’s consider this analogy here. Think of it like trying to flood a fast-draining sink. If the sink (the network) drains water (requests) quickly (low latency), you’d need to pour a lot more water at once to make it overflow. On the other hand, if the sink drains slowly (high latency), even a steady trickle could cause it to overflow over time. In essence, low latency can significantly reduce the potential for various sophisticated attack vectors while high latency increases the vulnerability many fold.

The Good News – Bigger Picture

The good news for public blockchain networks and the Web3 ecosystem at large is that it inherently seeks the participation of common people whether it be in the form of validators, or smart contract developers or as storage resources. This is something its Web2 peers cannot claim where the power is largely with companies owning the data centers and cloud servers. More recent blockchains protocols are aiming for high scalability from the ground up, optimal consensus mechanism, and other techniques like auto-scaling to solve the scalability trilemma.

How Does Shardeum Achieve Low Latency?

Let’s start by how we ended the last para – by solving the scalability trilemma 👨‍💻

Shardeum is an EVM-based layer 1 smart contract platform which is currently in its betanet stage of the project. On Shardeum, consensus will be done at the transaction level instead of the block level and transactions will be processed parallelly through dynamic state sharding with a latency of very few seconds. Well that’s a mouthful, isn’t it? Ok let’s break it down in simple words. And we’ll engage you with a chronological elaboration from the perspective of blockchain technology so you can see the complete picture.

1. High Fairness & Transaction-Level Consensus

To start with, validators on Shardeum will validate and process transactions on a first come first serve basis to virtually render MEV and front-running transactions impractical on the network. Note, this elusive practice also known as high fairness, removes the first bottleneck to achieve low latency. Now validation, which is followed by consensus and processing, will be done individually for each transaction. In conventional blockchain networks, a batch of validated transactions based on the block size is collected before being added to blocks. Consensus and processing occur for each block containing all the validated transactions. This naturally results in increased latency, as it requires time for all nodes, or even a subset of nodes, to process and finalize blocks (which more often happens sequentially in practice).

2. Immediate Finality

On Shardeum, each individual transaction confirmed will have immediate finality which sets it apart from other blockchains that offer probabilistic or absolute finality. It is further a breakthrough in blockchain technology, as it provides finality without the need to wait for multiple blocks to be confirmed. The network, later, batches such transactions together and moves them to the archive nodes. This further allows the validator nodes on the network to only store the current state of the accounts within the involved shards. Let’s come back to this part again shortly. Staying on course, Omar Syed, co-founder of Shardeum summarizes that “once you’ve done the consensus for the transactions (individually), the data structure (referring to blocks) you use to store them does not matter. It matters for other networks because the data structure is tightly coupled with the consensus.”

3. Dynamic State Sharding & Parallel Transaction Execution

Shardeum’s dynamic state sharding will work hand in hand with its auto-scaling feature allowing the network to automatically adjust the number and size of shards based on the prevailing workload. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously instead of sequentially. This not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing with a network latency of just a few seconds. 

4. Equal Distribution of Workload

Asides from enabling secure verification of transactions, Shardeum’s leaderless consensus mechanism, reinforces network security and decentralization by rotating the validator nodes regularly. Ultimately Shardeum ensures equal distribution of workload to network resources while maintaining operational efficiency as it grows and evolves in a sharded environment.

5. Linear Scalability & High Performance

Dynamic state sharding enables the most important X factor for Shardeum – linear or horizontal scalability as opposed to vertical scalability. Every node/shard added to the network from its ‘standby nodes’ pool will proportionally increase its TPS capacity during periods of high traffic. Shardeum’s ability to scale linearly and operate efficiently makes its network overheads such as latency, finality, and gas fees, highly predictable even for complex smart contract transactions.

Conclusion

In a nutshell, Shardeum will allow average people/computers to run a node on the network resulting in very low and constant fees permanently empowering end users with a UX we’ve all been accustomed to in traditional Web2 networks. Shardeum’s linear scalability will extend to dapps built on its network, allowing them to process transactions quickly and seamlessly, even as the network grows.

Frequently Asked Questions (FAQs)

1. Why is Low Latency Important?

The importance of low latency lies in it helps reduce the delay in data transfer, resulting in a faster and more efficient online experience. This is especially crucial for online gaming, high-frequency trading, and healthcare applications, where getting data in real-time can mean the difference between life and death.

2. Is Low Latency Good?

Of course. Low latency is considered good because it results in a faster and more efficient data transfer, allowing for a smoother online experience. However, the level of “goodness” will depend on the specific use case, as different applications will have different latency requirements.

3. Does Low Latency mean Faster?

Low latency means faster data transfer, resulting in a faster online experience. This is because low latency reduces the delay in data transfer, allowing for more real-time interaction between applications and users.

---