Time-Based Transaction Ordering: How Shardeum Enables Chronological Consistency and Fairness
This blog will take you through time based transaction ordering in blockchain and how Shardeum will process transactions in a FCFS manner to achieve high...
This blog will take you through time based transaction ordering in blockchain and how Shardeum will process transactions in a FCFS manner to achieve high...
Fairness in blockchain refers to the principle that all participants in a blockchain-based system should have an equal opportunity to participate in the transaction processing, consensus process, governance, resource allocation, and receiving rewards for their contributions. It ensures the network is secure and resistant to attacks, promoting credibility and legitimacy of the blockchain system, which is crucial for its adoption and success. We will take a detailed look at the topic that is often ignored by many – time-based transaction ordering process – which is the entry point for issues like MEV and unfairness in the blockchain industry at large.
Achieving fairness in a blockchain is not a straightforward task. Challenges such as limited scalability, high latency, weak consensus algorithms, sybil attacks, and economic incentives can create power imbalances and give some participants an unfair advantage. At the same time, we need to get the context right here because front running, for instance, was an unintended consequence of blockchain’s transparency and self-imposed limits to keep a public blockchain network secure and decentralized simultaneously.
Compare this to the enforcement of car seat belt rules and ticketing for non-compliance when driving. While the majority of law enforcement officials issue tickets to individuals simply for not following the rules, a few officials may view it as an opportunity to benefit from the system designed to credit them for enforcing the law. While their motivation may not always be ideal, it is undeniable that this provides an incentive to protect the lives of drivers and co-passengers and promote adoption of the rules. A new technology can have challenges on its way to wider usage and till it evolves through research and development, certain trade offs must be considered and implemented.
It must be emphasized that blockchain networks, which were introduced in the aftermath of the 2008 financial crisis, were never conceptualized to scale up to the various needs of every person on the planet. Bitcoin is rightly considered the best peer to peer payment network with maximum security, transparency and decentralization while retaining privacy of users. It later shaped the industry’s efficiency in cross border payments making it much cheaper and faster than the Web2 peers. However, if you see, Bitcoin can process only up to 10 TPS when the Web2 peers like Paypal and Twitter can process more than 10,000 TPS. 10 TPS was sufficient when barely anyone used public blockchains before things took a pivotal turn in the middle of last decade.
World adoption is only going to happen when there are multiple use cases for a technology. If AI is used only for language translations, it would have been limited to the scope of translation services with limited adoption. But today, AI powers your Siri, chatbots, voice recognition, cybersecurity, IoT devices, exchange order books, fraud detection, and many more. Although blockchain is still a young industry, one of the earliest signs of its potential to become massively popular was seen with the introduction of smart contracts by Ethereum and lighter consensus algorithms. Other than just being a payment network, smart contracts have enabled a multitude of use cases for blockchains across various industries.
Blockchains can now connect the data between its networks and the outside world and produce products and services without an intermediary. Public blockchains, further, have demonstrated it can pretty much do anything you can imagine in a Web2 world, albeit, transparently. Similar to how an app is created on top of Android or iOS, anyone can create dapps (decentralized applications) atop layer 1 blockchains like Shardeum. And, unlike a Web2 entity, every transaction is recorded in a public network’s digital ledger transparently. This gave an additional impetus for industries such as finance, supply chain, retail and consumer goods, manufacturing, etc. to adopt blockchains to varying degrees. DappRadar, a global app store for dapps, reports that there are a total of 12k+ dapps as of May 2023 with a TVL of approximately $55 billion notwithstanding multiple downturns and negative perceptions of the industry.
Most of the current distributed ledger protocols have self-imposed scaling limits as they stipulate maximum (fixed or varied) size for their blocks and the rate at which blocks are produced thereby setting a ceiling on the rate at which their network can process transactions. Remember, consensus and self imposed block rates are crucial to the security and trustless nature of public decentralized networks. Limiting TPS and the consequent slower finality results in network congestion. In order to decongest, the networks would need individual nodes to be equipped with high hardware and software requirements. At that point, only limited people can afford to run a node which makes the network susceptible to 51% and DoS attacks.
The other unwelcome outcome of blockchain’s efficiency lies in its transparency as mentioned above. Public blockchains record every transaction cryptographically on a public ledger validated by unrelated network participants (or nodes/validators) distributed across the world. While user information is encrypted for security, block explorers, similar to search engines, can be utilized to track and identify the status of a transaction from its initiation by a user until the network confirms and processes the transaction at each stage. Basically, it is open to anyone including users, data analytics firms, law enforcement and of course, the validators.
Most PoS networks follow a more or less similar process to validate and process transactions. Once a user submits a transaction, it will land in the network’s memory pool. The mempool serves as a repository for unconfirmed transactions within blockchain networks, awaiting validation by node validators. Once selected, individual validators verify these transactions before they are added to a block. As the block approaches its maximum capacity, it is appended to the network chain. The validated transactions within the block are then broadcasted to other validators on the network to achieve a majority consensus, a crucial step in ensuring the security of public networks. Once a majority consensus is reached for a block, the transactions within it are confirmed and the block is permanently added to the network chain, preserving its integrity and immutability. Note, a consensus mechanism can also plays a key role in transaction ordering process in blockchain networks. We will find more about that in the paras below.
As you can see here, validators play a major role in the operations of a public blockchain network. You may expect the transactions are selected and processed on a FCFS basis ideally. But, as a result of the transparent nature of blockchains and the scalability issues discussed in the paras above, transactions in the mempool compete for limited block space, and validators prioritize them based on factors like transaction fees, transaction size, and network congestion. Typically, transactions that offer higher fees have a greater chance of being included in the next block by validators, incentivizing users to attach higher fees to their transactions to expedite confirmation.
Here, the validators prioritize higher-value or more lucrative transactions while including lower-value or smaller transactions to meet the block space limit and achieve consensus and block confirmation swiftly. The ordering of transactions is done, albeit manually, instead of an autonomous or independent process to maintain high fairness removed from any manipulation and bias.
Significant or inconsistent network latency can introduce discrepancies in the perceived order of transactions, even when the network employs a clock synchronization protocol. Even the more recent sharded blockchains struggle with inconsistent latency issues especially during peak demand because they perform consensus at the block level which more often makes it difficult to parallelize transactions.
When there is a demand spike, these blockchains typically make use of a static/pre-defined group of shards in the network. Transactions on such sharded platforms are either processed sequentially after a minimum number of nodes join the network to create a new shard and/or processed after a wait time for the new shard validators to sync up to the latest state of the network. This increases the network latency which directly impacts its finality rendering it unable to process a high number of TPS.
When the time to confirm transactions and ensure their irreversibility (finality) fluctuates based on the demand in the network, getting the most out of a timestamp based ordering protocol will be impractical. The high latency can also potentially allow bad actors, in the meantime, to create hard forks resulting in several failed or delayed transactions. Such transactions would have to be re-submitted by users disrupting the fairness in the blockchain system.
Some transactions may have dependencies on other transactions like in the case of smart contract interactions, token transfers, chain reorganizations in the form of hard forks which is a side effect of deterministic finality taking the network too long to confirm transactions as discussed above. Blockchains are increasingly finding it challenging to achieve atomic and cross shard composability, which will in and itself introduce latency on the network.
Without atomic composability, transactions could potentially fail or leave the blockchain in an inconsistent state, leading to security risks and reduced reliability. Without cross-shard communication, transactions will not be able to access and utilize data and state from different shards, curtailing complex transactions and smart contracts to be executed in a sharded/partitioned environment. In a nutshell, it will not be possible to maintain a correct time-based ordering on the network without ensuring dependencies are properly synchronized across shards.
Primary node election in consensus mechanisms, such as Proof of Stake (PoS) or Delegated Proof of Stake (DPoS) used by several blockchain networks today, is also vulnerable to MEV crisis, front running and sandwich attacks. In a primary node election-based consensus mechanism, where a single or a small group of nodes are chosen as block validators or leaders, there is a risk of these nodes having an advantage in extracting MEV. They may have the ability to prioritize their own transactions or engage in other manipulative practices, leading to unfair economic advantages. And worse, they will be vulnerable to bot attacks and takeover attempts by malicious actors causing dysfunction to the chronology of processing transactions.
In addition to the risks associated with MEV, if a consensus mechanism relies on the primary node’s authority to validate transactions and establish a consistent ordering, the particular node’s efficiency and speed in reaching consensus can impact the accuracy of time-based ordering. Primary node election can also impact the stability of the network. If the primary node is not elected or replaced in a timely manner, it can lead to network disruptions or delays in transaction processing distorting the accuracy of time based ordering.
Although consensus algorithms (or mechanisms) play a larger role in reaching consensus for validated transactions among validators, they can also play a considerable part in the way transactions are ordered on a blockchain network. For instance, performing consensus at the block level introduces complexities in getting the actual timestamp of transactions. If you see, many blockchains have a predefined block size limit to ensure network scalability and performance. When the number of transactions exceeds the block size limit, the selection and inclusion of transactions in a block become a challenge. They are often influenced by the various prioritization mechanisms used by networks other than chronological ordering to prevent network congestion and outages.
Further, consensus done at block level often results in sequential processing of transactions unless blockchain networks scale vertically. Both sequential processing and vertical scalability, however, leads to slower processing speeds and/or stability issues that hinder the chronological ordering of transactions in a block.
Layer 2 blockchains are introduced as a solution on top of L1 platforms to address scalability issues to a certain degree. L2 solutions, including rollups, aim to alleviate the burden on the main blockchain by processing a significant portion of transactions off-chain. These solutions typically aggregate multiple transactions into a single batch and submit the batch’s outcome to the main chain. It is important to note that final ordering on the main chain is influenced by the roll ups or layer 2 solution’s submission timestamp, more often than not!
While transactions within the layer 2 solution may achieve faster confirmation or settlement, they might require some time to be included in the main chain affecting the precise time-based ordering of transactions on the main chain, especially if there are conflicts or reordering during the submission process. We also must take into account the communication between the layer 2 solution and the main chain can introduce delays influencing the time-based ordering unless the network accomplishes atomic composability.
Bitcoin introduced decentralization. Ethereum scaled up the decentralized economy. Together with other recently launched blockchains and utilities, the industry was worth more than $2 trillion at its ATH (almost equal to the largest public Web2 company – Apple – in terms of market cap).
Shardeum is aiming to take Web3 and decentralization mainstream by solving the scalability trilemma and retaining low and constant gas fees forever. The inherent nature of the network incorporates breakthrough mechanisms to address both unfairness and the MEV crisis prevalent in blockchain networks.
Shardeum will use time based ordering of transactions before network validators validate and reach consensus on each of them. The L1 smart contract platform will scale linearly through dynamic state sharding. It will shard its state by evenly distributing compute workload, storage, and bandwidth among all the nodes while at the same time auto-scale the number and size of shards based on the prevalent workload dynamically.
It is further important to note that, on Shardeum, processing and consensus are applied at transaction level and not at the block level enabling parallelized processing of transactions across shards while retaining atomic and cross shard composability. Shardeum will ensure complex transactions and smart contracts are executed effectively in a sharded environment while maintaining the consistency of the blockchain. Once the transactions are validated and processed individually across shards, they will be grouped together without any constraint on the size and limit of such groups/partitions. The grouped partitions will be passed onto archive nodes on Shardeum which is responsible for storing transaction history.
Every node added to the network especially during demand spikes will instantly form shards increasing its throughput proportionally. If 1 node can process 1 TPS, 1000 nodes can process 1000 TPS and so on. And because Shardeum scales linearly, transaction fees on the network will always be very low and predictable. As a result, the network finality will be instant with a network latency of just a few seconds across the network with 0% chance of reversibility once a transaction is finalized. The immediate finality is crucial for a network to not only remain scalable but also process a high number of transactions per second.
Along with linear scalability, Shardeum’s consistency in terms of composability and instant finality will remove high-latency and MEV based challenges towards a time based ordering protocol. This further removes any necessity to build layer 2 solutions exclusively for scalability purposes while also keeping the network’s time based ordering intact and independent from dapps building on it.
Consensus algorithms used in Shardeum will go a long way in helping the network achieve high fairness. Shardeum employs Proof of Quorum (PoQ) and Proof of Stake (PoS) consensus algorithms to process transactions on the network. PoQ enables trustless and leaderless collection of votes after validations and subsequent consensus are reached on individual transactions. PoS will ensure validators stake a minimum amount of network coin to participate in the consensus process. Misbehaviors will be slashed. Importantly, the consensus algorithm will enable the auto-rotation of validator and standby nodes randomly to maximize security irrespective of the load on the network.
The efficiency and results achieved through dynamic state sharding, auto-scaling and consensus mechanism on the network will enable Shardeum to maintain an independent and autonomous clock synchronization. This in turn will facilitate the network to process transactions on a FCFS basis. No transaction will be given preferential treatment or prioritized over others based on factors such as the transaction sender’s identity, wealth, or any other discriminatory criteria. Every transaction is treated equally, and they are processed in the order they enter the transaction pool. As you can imagine, time based ordering will only be effective when a network auto-scales linearly with a solid consensus algorithm and atomic composability.
Edits: Latency/Finality time is updated for clarity and accuracy on 16th Sept 2023.
Maximal Extractable Value within a blockchain are transactions executed for unfair financial gain. Read more to know about what is...
This blog visits how Shardeum launched its project with the help of its community in an open and collaborative...
Do you know what is the beauty of the Internet? The fact is that it is owned by everyone, and available for everyone. This is where the concept of OCC comes in. Shardeum is being built on the guiding...
