Validation
Validation of Hyperledger Fabric Simulations
Dec 09 2020

Introduction

Blockchains are complex systems. They display emergent behaviour, which is furthermore sensitive to small design changes. At the same time, the design space is large and measuring real deployments expensive. The combination of these factors renders simulation attractive. Simulation has proven its aptitude for handling complex systems in a variety of disciplines, while usually being orders of magnitude more affordable than real-world experiments. However, the major challenge with leveraging simulation is faithfully modelling and representing real-world systems.

We have developed a detailed model and simulation environment for Hyperledger Fabric, allowing us to predict the behaviour of a large range of deployments. Our simulation are event-based, building on OMNEST [1] and using the INET library [2] for communication protocols. Here, we will compare our simulation environment to actual deployments of Hyperledger Fabric on AWS, demonstrating its capabilities.

System, Deployment and Results

We investigate a Hyperledger Fabric system with one channel and four organisations. Every organisation runs seven AWS m5.2xlarge instances in a single datacenter. Every instance is dedicated to a particular function: one orderer, two peers and four clients. All transactions are update transactions with a 40 byte payload and they require two endorsements, one from their own organisation and an external one.

We vary their geographic distributed: within a single datacenter, across Europe (left figure), world-wide (right figure). Every configuration is measured with 10 and with 100 transactions per second (tps). The transactions originate uniformly from the four organisations. Finally, system configurations are deployed ten distinct times, 60s per run (30s warm-up and 30s measurement).

As can be seen in the figure below, the simulated latencies compare well to the real-world deployments. Both the European and the world-wide simulation data match the deployment data within a few percent. For the single datacenter scenario percentage differences are slightly higher, however, absolute differences are similar.

The above figure compares real-world deployment vs. simulation. Specifically client-side update transaction latencies for different geographical distributions of organisations and 10/100 tps. Error bars are standard deviations across transactions (over all measurements); differences stem mainly from waiting times to be included in the next block.

Conclusions

The validation results indicate that our models capture the workflow of Hyperledger Fabric and its behaviour on AWS accurately. It is therefore capable to reliably predict the complex behaviour of real-world deployments to well within 10%. This allows for highly competitive design space exploration and risk scenario analyses, as our simulations are multiple orders of magnitude less expensive compared to the actual deployments.

References

Back
Contact us

    Name
    Email
    Message

    Send message
    Contact us

      Name
      Email
      Message






      Add Application

      Send message