Askemos Accuracy Test

About the Test

Here we compute some statistics on the accuracy of the Askemos network. Accuracy is approximately the equivalent of "uptime" for physical computers. Within the network we have two types of availability:

locally available / read available

Data can be read, and modified on a single machine, but modification with respect to replica synchrony is not checked.

replica synchrony / write available

A replicated transaction was performed under non-repudiation protocol.

Since we run from the same basic hardware and - for the most part - software setup as most web services (PC's loaded with Linux or FreeBSD) average read availability is a fixed value of about 99.999% (less than 5 minutes a year of downtime).

Test Setup

To measure replica synchrony, each node in the network is configured to send a local, public write message with it's idea of current time to it's replica of the clock over there. The nodes try to find an agreement on that common time and if that can be found within 5 seconds, keep it and notify this agent here.

Here in turn we run the agreement protocol again to count how many minutes passed since the last agreed-on time tick. Furthermore we count the missing minutes as errors.

Interpretation

The whole test amounts to two runs of the protocol, which both fail with a certain probability for a variety of reasons like network latency, messages drops, delays from data base access, software updates, broken hardware etc. The time out of 5 seconds is actually sometimes hard to meet for the WAN test setup, though on average we have less than 0.3 seconds per agreement over SSL. The logfile confirms, that messages arrive as late as 20 seconds past the full minute.

The reason for the low availability measured here is that we measure the test network, which has downtime due to bugs.

The average accuracy is with a_total total number of agreements (failed and successful) and a_failure number of failures roughly^[1]:

Results

Source

You may visit this place in debug mode (read only) or read the source code of the agent.

^[1] Only geometric mean at this time. There is a need for more spot test data to gather enough information to evaluate a sufficient guess for the distribution. We can not take a normal distribution as given.