One Ring to Prove them All

As we near the final moments leading up to the full launch and unlock of mainnet, the topic of prover rings has become of deeper interest. While there are various places in which it has been discussed, it is a topic that has been in need of a deeper dive because it is a bit more unique compared to other networks, despite its concise description: "early provers have priority on declaring ranges of shards to prove over".

If you've stuck around for most of the streams or discussions, you may have a good enough idea and are looking for answers to the calculation of seniority or how to handle having changed keys over time, scroll down or Ctrl+F to "How are seniority and penalties calculated?" and "I changed peer keys and configurations over time, can I combine them for seniority?"

Indeed, some folks picked up on it early, and, naturally, as the internet does, made a meme of it:

The design of Quilibrium makes use of a proof system that demonstrates possession of data over a consistent window of time. This is crucial to ensure that provers cannot "cheat" – that is, they cannot hold only a partial quantity of the data and/or summon those components via the protocol itself on demand to fulfill the proof basis. This part has been previously described in many different forms including well before the project became Quilibrium itself, so I will omit this portion of the protocol in the explainer. But one aspect of that proof system's function is actually becoming a prover for those given shards of data – enter the prover rings.

Data replication is important in any distributed system – block chain approaches generally take on full replication of state – either from the very genesis, or cleaved history by terms of state rent or simply data that no longer matters, such as Alice sending Bob all of her coin. The task of holding onto all of that data is a problem for this structure, as even in the state rent or garbage-collected model, the inevitable outcome of a protocol that grows in use is that it will indeed, continue to grow in size. Bitcoin at this time is roughly at about 600GB for full history, and Ethereum has exceeded 1TB. Solana, full history, is hundreds of terabytes. Notably, there is a correlation one could draw between the speed of these protocols as well with respect to their overall size. In some respects, this is a consequence of the Jevons Paradox in action.

But nevertheless, to run a true "full node" that verifies from genesis to present under a blockchain, you are taking on the task of holding onto all data indefinitely. This consolidating pressure for higher throughput chains results in extremely expensive, powerful servers, and as few of them as possible while enough they can claim to be "decentralized". In these higher throughput models, the leader of a given next block in sequence is chosen via a deterministic approach, either economically disincentivised to not misbehave (PoS/slashing) or algorithmically all provers are simultaneously doing the same work slightly differently such that they achieve a "magic' number that meets a low enough value that theirs wins (PoW). This collective is something we at Quilibrium generically refer to as a type of prover ring.

Quilibrium is not a block chain – while someone may squint and say the sequencing of frames greatly resembles a block, and at the highest level of the network they do indeed form a linear sequence, these "blocks" are missing something quite crucial – the underlying data. In distributed (trusted) database design, a higher level shard coordinator is strictly algorithmic: a function (typically a hash) takes an input of a key, and that determines in some way which shard the key and value will reside. The shards themselves initially are "logical" (i.e. a single server may contain many shards and respond on behalf of all of the ones which it covers the range for), and as the limits of that server are reached, the logical shards are split further out into smaller subsets with distinct servers. For those who are more familiar with NoSQL style architectures, this pattern is extremely well understood.

But this works strictly in a trusted design, and there exists a need to ensure that one shard cannot in some way collude to impart a disadvantage towards another, and thus we have a higher order problem that trusted designs do not: how do you ensure the shards cannot become out of step with one another, while simultaneously not having to retain all shard data in one place? We break things down into three logical tiers: at the highest level, we have a global shard. For the near future, we expect servers will likely assert an ability to prove over all 256 of the global shards. Given the initial data set is small, this is certainly reasonable. Of the 256 global shards with applications settling into three of any of the 256 shards, the combinatorics of 2,763,520 distinct global shard proof combinations are a simple thing to track – these proofs are able to be aggregated to the 256 collective proofs very cheaply.

The second layer is when the multiplicative combination of the above global shard proof slices amplify, such that application data settles under a combination of 24 of 65536 distinct core shard proofs per set of 3 global shards. This results in a global keyspace capable of addressing more possible shards than atoms in the universe, Under those tiers, the data shard becomes the final layer, able to hold up to 1GB.

To prove these shards individually would be effectively impossible, and so provers on the network must declare a range of shard key values they wish to prove over, at the global layer, the core layer, and the data layer. The multiproofs, previously described, allow efficient compaction of proving over those larger data sets, and are what enable this to not grow unwieldily while the network is small. A prover ring, in Q parlance, is simply a collection of provers that have declared to prove over that same range. Under Proof of Meaningful Work, the reward of a multiproof is scaled relative to the span of data included in a single proof such that while it the reward of a multiproof is less than the raw sum of rewards for individual proofs for each data shard, the network is too small in resources at this time to attain that level of granularity, ergo the most economically advantageous strategy is to do the multiproof. The key here is that while the network grows, the more easily prover rings can narrow in range, the larger the portion of reward earned for the data becomes, encouraging deep distributed replication and more efficiently addressable access.

Replication is valuable, but overindexing on replicating the same data has diminishing returns, and so the construction of prover rings is intended to have an inverse correlation of reward the farther out a node is in declaring (and fulfilling) intent to replicate. This scaling factor was previously detailed in Proof of Meaningful Work. Node operators who have maintained nodes well, for longer durations, accrue "seniority" such that when attempting to resolve a conflict between two equivalent prover ring join requests for a given range, the disambiguation first defers to the prover with longest duration as a functioning prover. But that still leaves an almost MEV-like situation where the sequencing of those requests clearly can create an unfair advantage when there does not exist a disambiguation between equivalent seniority. What resolves this problem is our use of the mixnet outputs. It not only ensures all traffic is defended against timing analysis, or that traffic cannot be blocked for a given requester, but in the case of disambiguating prover slots, it provides a deterministic and effectively random disambiguation for otherwise equal requests.

Seniority is a function of time. Prior to 1.4.19, the retroactive rewards had explicit units of real world time correlated to the recipients, and for the .19/.20/.21 series, the increments themselves serve as a proxy to time. In 2.0 where VDF difficulty is adjusted to align to 10 second increments on average, we once again become bounded more closely to real world time. These segments of time accumulate as a simple "point" based system. During the 2.0 mainnet stasis period, the network will perform the clock iterations and negotiations that set the initial baseline VDF difficulty for ten second increments. This sets the evaluative bar (the determined difficulty factor) for determining how the increments from the .19/.20/.21 series translate to these seniority points. In short: the total seniority is derived from the ten second intervals of peer uptime prior to .19, the total difficulty accumulated during .19/.20/.21 for the increments a prover generated divided by the determined difficulty factor, and thereafter, the proofs generated by a peer for their prover rings.

The second factor is punishing bad behavior in the form of penalties. Inactivity when not part of a prover ring does not penalize beyond simply not accruing further seniority, but if part of an active ring under 2.0, missing proof intervals, issuing invalid proofs, or other misbehaviors will have penalties to seniority that can cause a peer for the prover range to be "evicted" by protocol rules and the next lower earning ring slot member will be upgraded to stand in place.

Many node operators experimented with different deployment strategies in order to maximize rewards as the network evolved. To ensure that fairness is possible for operators who have changed peer keys and configurations, the prover slot join request is going to include the option of essentially "combining" multiple prover signatures, such that their seniority is additive, but not overlapping (albeit, overlapping time intervals if both peer keys were active do not count as "extra"), so for someone who ran a node during the Dawn era, generated a new key set in the lead up to 1.4.19, and oncemore did so during 1.4.19, they can combine these keys via a configuration step such that their initial join request can have greater continuity of seniority for their prover range claim. Instructions on how to perform this step will accompany the 2.0 release guide, and we are working with guide writers to ensure their own instructions will include these steps in conjunction with their upgrade info.

We have a discourse! https://discourse.quilibrium.com is a great place to ask these questions, and feedback that helps further clarify will be added here if need be.