Inside Shardeum’s Archiver Verification of Validator Data

This blog post continues our series, offering an insider’s look at how our engineering team is working hard to pioneer the development of the world’s first autoscalable blockchain that solves the scalability trilemma. In this blog, we will uncover why and how Shardeum’s archive nodes verify the data from validator nodes. This discussion highlights our ongoing commitment to not just address the scalability challenges of blockchain technology but also enable the robustness and dissemination of our unique network protocols. Let’s now understand the role and function of archive nodes in verifying data from validator nodes.

Purpose of Verifying Validator Data by Archivers

In essence, archiver verification of validator data is a method to double-check the data flowing from our validators to archivers. Due to Shardeum’s sharded network design using dynamic state sharding, validators store only the partial state of the network. Validators stream and send their partial state to archivers. For one, by offloading the responsibility of maintaining the complete ledger to archivers, validators can operate more efficiently and focus on processing transactions. This separation of duties is fundamental to Shardeum’s scalability as it enables the network to manage increased transaction volumes with agility, while allowing archivers to maintain a comprehensive record of its history.

Second, by verifying the data they receive from validators, archivers ensure that this data has not been tampered with or corrupted during transmission. Moreover, the archiver verification protocol provides an additional layer of security. It acts as a second check, confirming that the data upheld by validators meets the network’s strict cryptographic standards. This redundancy is key in preventing single points of failure and in protecting against potential security breaches within a shard.

If archivers do not verify validator data, the blockchain could face increased risks of errors and fraudulent transactions being permanently recorded, leading to compromised data integrity and weakened network security. This lack of rigorous verification will ultimately reduce trust among users and stakeholders, affecting the network’s adoption and reliability.

How Does This Work?

Here is a more technical, yet an interesting view into the process flow of how this works fundamentally.

1. External Receipt Validation

Archivers connect to validators and receive streamed transactions and receipts. This is in order to consolidate and store data from across the sharded network as all archivers need to store the entirety of the network state.

2. Signature Verification

Archivers validate receipts by checking signatures and confirming the signatories’ legitimacy and relevance to the transaction shard. Although a key method of verification, this method can be compute-intensive and might not scale well with increased traffic and transaction throughput. Due to these potential challenge vectors, the alternative method of “robust query” has also been introduced to complement signature verification.

3. Robust Query

As well as using signature verification to validate receipts, an alternative method is using robust query to validate receipts. Here, the archivers request confirmation of transaction details from multiple validators. This approach reduces the computational load on the archivers themselves by minimizing the need for detailed individual receipt verification and enhances robustness against rogue validators.

That being said, this method shifts more responsibility to the validators. At high transaction rates or during peak network activities, this can significantly increase the computational burden on validators as they respond to multiple queries from different archivers. When needed, our next strategy “Archiver Gossip of Receipts” comes into play to maintain efficiency without compromising the network’s responsiveness and validator performance.

4. Archiver Gossip of Receipts

After verification, archivers share received receipt information, reducing reliance solely on validators. Gossiping or distributing the validated receipt information throughout the network not only lightens the computational load on validators by sharing data storage and dissemination responsibilities, but also boosts the detection of rogue validators further.

5. Grouping Receipts into Blocks

Even though validators verify and reach consensus on transactions individually on First Come First Serve basis on Shardeum, they will create blocks of receipts to be validated collectively rather than individually by archivers. This not only creates a more efficient validation process, but also ultimately makes transaction storage a much more streamlined affair.

6. Validation With or Without Nodelist

Downstream nodes on Shardeum like relayers have the option to validate blocks without maintaining a comprehensive list of all active validators (or nodelist) in the network. In other words, downstream nodes have the option to simply trust that the blocks validated by archivers are correct and they do not perform their own independent validation, instead, relying on the validation already completed by archivers.

If downstream nodes possess a list of active validators, they can choose to validate blocks independently. This means they verify the blocks themselves, using the list of validators to ensure the blocks were validated according to the network’s rules. This solidifies the verification process by giving a trust-based option to relayers, and also a verification based option to ascertain the validated blocks.

7. Edge Cases and Exception Handling

Like all good protocols, the archiver verification of validator data protocol has ways to handle edge cases. Therefore, the archiver verification of validator data has procedures for handling mismatches in Merkle roots and for missing receipts. Validator nodes may retain certain data for a period of time to support these processes.

8. Handling Bad Receipts

Archivers can query validators for receipts of specific transactions and verify signatures in case of discrepancies. Although this would only occur in rare circumstances, it would be essential for handling exceptional situations. This allows archivers to tackle the other major edge case of querying and verifying bad receipts, making Shardeum a verification-first network built on cryptographic guarantees. These methods are designed to maintain the integrity and reliability of the data stored by archivers, ensuring that the network remains secure and scalable as it grows and handles more transactions.

Archiver verification on Shardeum utilizes a comprehensive approach that incorporates all of the methods described above dynamically. This multifaceted strategy is designed to ensure the accuracy, integrity, and reliability of the data within the network.

Benefits of Archiver Verification of Validator Data

Let’s now summarize the multi-benefits of this protocol –

• Enhanced Network Security: By rigorously validating the authenticity of transactions and receipts through hygienic routines like signature verification and robust querying, Shardeum significantly minimizes the risk of fraudulent activities and enhances the overall security of the network.
• Data Integrity and Accuracy: The process of external receipt validation (outside of validators) and the handling of bad receipts ensure that only accurate and verified data is stored, maintaining the integrity of the network.
• Scalability and Efficiency: Grouping receipts into blocks and optimizing bandwidth through archiver gossip mechanisms contribute to the network’s scalability. These processes allow the network to handle increasing volumes of transactions without compromising on speed or efficiency.
• Reduced Computational Load: By implementing strategies such as robust querying, which verifies the uniformity of receipts across multiple validators, Shardeum significantly reduces the computational overhead, making the validation process more efficient.
• Decentralized Trust: The archiver gossip of receipts fosters a decentralized form of trust, as the validation process does not solely rely on a single node or validator. This approach enhances the credibility and trustworthiness of the network.
• Resilience Against Attacks: The increased resilience against rogue elements, as documented in the prior section, protects the network against various types of attacks and malicious behaviors.
• Flexibility in Data Validation: Providing options for downstream nodes to trust validated blocks or perform independent validations offers flexibility and caters to varying levels of trust requirements within the network.
• Comprehensive Network Resilience: Preparing for and managing edge cases ensures that the network remains robust and functional even under atypical scenarios, thus enhancing its overall resilience.
• Simplified Validation Process for Downstream Nodes: The trust-based simplification for downstream nodes, such as relayers, reduces their computational and operational burden, facilitating a more streamlined data distribution and validation process.
• Data Consolidation and Standardization: Connecting archiver nodes to multiple validators for data aggregation fosters a cohesive network structure, facilitating easier management and access to data.

Conclusion

In conclusion, Shardeum’s implementation of archiver verification marks a significant advancement. By employing a range of strategies—including external receipt validation, signature verification, robust querying and gossiping—Shardeum enhances data integrity and network resilience. This multifaceted approach not only mitigates risks of fraud but also supports scalability by optimizing computational load and fostering decentralized trust. Through these innovations, Shardeum solidifies its infrastructure, ensuring a reliable and scalable network ready to meet future demands.

---