Hey there,
This article is set to go originally live on our Substack. Users that follow us on Paragraph, will be able to mint this as an NFT.
Today, we are joined by Jose Sanchez from Spartan Capital. We have been collaborating with him on research around Oracle Extractable Value. Today’s piece is the result of our interactions. Follow Jose on Twitter for insights on all things crypto.
Crypto is a multiplayer game, and so is research. In that spirit, we are inviting thematic experts to collaborate on stories. If you have been diving deeper into a theme or have an idea to share, reach out to us using the form below. Come jam with us - Pitch Stories
And as always, if you are a founder building cool things, use the form to drop your decks - Deck Drop
Back to the day’s story…
It is often hard to lay a bull case for an industry with few users and almost no revenue. Ethereum is one of the biggest exceptions to this rule. While most chains and applications incentivise validators and users with token emissions, Ethereum has reduced its supply by ~450k ETH since the merge. You would expect miners or validators to no longer be commercially aligned with Ethereum when rewards no longer exist. But that isn’t the case.
Why? Because there were other sources of revenue for validators in the form of fees or value paid by users without users being aware. This is what we refer to in our industry as maximum extractable value, or MEV. Today’s piece explores the concept of oracle extractable value and order flow auctions.
Before I rush into explaining how OEV works, I want to draw parallels between daily economic interactions and MEV itself for our readers. One way to think of a market is as a machine that constantly analyses and prices things using new information. A storm in Indonesia could run the price of coffee up. Or an issue at the Suez Canal could push back the delivery time for your shoes.
Seemingly random things can move prices, and markets are an attempt at constantly accounting for this information and repricing. This is the basis of the efficient market hypothesis. The idea is that the price of an asset accounts for all available information about the asset.
Picture a restaurant. The items on the menu have no fixed pricing. The owners have embraced capitalism and believe the free market should determine what their items cost. There are no caps on the number of customers the restaurant can serve. Since the amount of goods, chefs and service workers at the restaurant is limited, our hypothetical restaurant will see a rise in prices depending on demand for the dishes.
Initially, most items could be priced similarly, but as 6:00 PM arrives, people leave their offices and flock to the restaurant. The prices for individual items start increasing till the dishes get so costly that hungry customers recognise that it is cheaper to go lower the stack and buy items that are less popular, i.e., priced lower. This happens until demand for the lower-priced items brings those prices up.
Eventually, one of two things can happen. The restaurant runs out of dishes to serve due to running out of meat and vegetables. This scenario is closer to the situation we see with Bitcoin ETFs now because there’s a supply cap on Bitcoin. Or, the prices of individual goods rise so high that the average person can no longer afford them. This is what we are seeing with housing prices in many large cities.
Why does any of this matter? In this hypothetical restaurant, there is a supply chain for information. The guy holding the door open sees the influx of guests. The person taking the orders sees what items are being sold the most. The chef, while clueless about guest count – understands item inventory at the restaurant. In a free market, where the business is not predicated on serving food but speculating on the pricing of the items themselves, all of them can benefit from the information arbitrage that they possess.
The doorman can choose to take a fee from some of the customers in exchange for letting them in. The person taking the orders can choose to relay which orders are sent to the chef first. They can also take bribes from patrons to take their orders to the chef on priority. The chef can afford to report the lack of items to sell to a guest in hopes of selling at a higher price.
Our hypothetical restaurant may not last long in the real world, but many of our markets have variations of this constantly happening. To understand how this translates to the blockchain ecosystem, it helps to draw parallels between the supply chain of value on-chain and that of markets.
(Sidenote: Anthony Bourdain’s Kitchen Confidential is a beautiful book if you want to read about the shenanigans of a real restaurant.)
The Supply Chain
The value generated in the crypto ecosystem is generally split among users, developers, and validators. Ecosystems that distribute value in proportion to these players’ contributions are likely to thrive. Even when Ethereum was a proof-of-work-based chain, miners extracted MEV by exerting control over reordering transactions.
Joel’s Note: I originally thought that Ethereum ordered transactions in proportion to gas fees paid. However, the protocol gives miners enormous flexibility regarding how each transaction is ordered. This means a miner can choose to prioritise the transactions that go through first. For instance, they can skip all the transactions routed to a DEX with too much slippage, as they are bound to fail, or they can prioritise a transfer from a particular account.
This flexibility is what has given birth to what we now know as MEV. A version of this exists with Bitcoin’s blockchain too. This note from the University of Illinois is a good breakdown if you want to go further down the rabbit hole.
Once it was clear that MEV would become a significant source of value capture, a special class of actors called searchers emerged. In contrast to miners or validators, who reorder transactions, searchers scan the mempool and submit transactions to miners to extract profit. Of course, searchers must pay extra tips to miners or validators to ensure that the transactions get executed in their preferred order.
Searchers often end up paying high fees to validators. This is because if a searcher’s bundle (a specific sequence of transactions) is not included in the next block, someone else would execute it, or the opportunity might vanish as a person conducting a transaction (on a decentralised exchange [DEX], for instance) expects it to go through within a few blocks.
Between December 2019 and September 2022 (till the merge), $675 million worth of MEV was captured on Ethereum, out of which $241 million (~36%) was directed towards miners.
MEV typically originates via transactions that users generate, and participants who can extract it do so. Let’s clarify this with an example: When you want to swap ETH for USDC, most DEXs tell you the minimum amount of USDC you will receive. Say the DEX shows that you will receive a minimum of USDC 4000 for ETH 1. But what if someone (a searcher) can sell your ETH for USDC 4050? They keep the extra USDC 50 and give you USDC 4000.
Because the searcher must execute this on priority, they may share anywhere between 0 to 50 USDC with a validator. The USDC 50 in the example is the MEV.
There’s nothing good or bad about MEV. It will exist as long as users interact with a chain. For example, generating a swap transaction with a 5% slippage on Birdeye becomes a target for a searcher as they can earn up to 5% on that transaction. From the 5%, they will be willing to share something with a validator who prioritises their transaction. The point is about how the extracted value gets distributed.
Going back to the analogy of my hyper-capitalistic restaurant, a searcher is closer to the waiter taking your transaction in a particular order to the chef. The validator is the chef who determines whether to accept the order. Sometimes, the searcher pays a high bribe to the validator for executing a transaction. This is closer to a waiter paying the chef a bribe for getting a dish out faster because a customer is willing to pay a high premium (slippage) to do a transaction.
Why would a customer pay a high premium? He’s likely aware of some information that makes him think the price of the purchased asset will rise higher than the slippage fee paid.
Note: Slippage is how high or low an asset’s price tends to go when you buy or sell it because there isn’t enough supply or demand for the asset at the price you are selling it. It can be up to a few percentage points depending on the liquidity of the asset.
Out of the three stakeholders mentioned above, most of the MEV flows to validators, which sit at the end of the supply chain. This phenomenon largely results from intense competition among searchers, who consistently outbid each other in pursuit of MEV opportunities.
This competitive bidding escalates to the point where the bids nearly match the total value available for extraction, funnelling the bulk of MEV to the validators strategically placed to capitalise on these dynamics.
The chart below shows what portion of value captured through MEV goes through back running. Over time, these profits are tending towards 0 as most of the profits have to be shared with validators. So, assuming that 6% goes to the searcher, they capture 3 USDC out of the total 50 USDC MEV, and the remaining 47 USDC goes to the validator.
If MEV is a fact of life, then mechanisms that ensure equitable distribution are necessary. Products like Flashbots and CoW Swap are about democratising and redistributing the MEV back to the value originators. But whether that’s enough is unknown. Moving a step further towards evening out the distribution of the MEV involves oracles. Typically, oracles are data vehicles that bring information that exists outside the purview of a blockchain or application.
For example, when the price of ETHUSDC changes on Binance or other centralised exchanges (CEXs), oracles bring this information on-chain.
Oracles typically post this information in a mempool, where it gets picked up by a searcher. Information related to price updates is sometimes critical for DeFi applications, such as lending markets. Aave, for example, is a lending platform that allows users to borrow and lend crypto assets.
Users can deposit an asset as collateral and borrow another one up to a certain amount, which is usually a percentage of the value of the collateral deposited. Say you have $150 in ETH tokens: you may be able to take up to $75 in loans.
Leaking Value
Loans on Aave are almost always over-collateralised. That is, the value of the tokens you leave on Aave’s smart contracts is always greater than the value of the loan you took. When the loan value increases to become higher than that of the collateral provided, the smart contracts on Aave determine that your collateral should be sold to recoup the loan that was given to you. This process takes place through smart contracts, which live on a ledger.
As such, they require an external mechanism that will always know the value of the collateral and borrowed assets. The protocol knows how much the collateral and loan are worth through oracles.
(Think of oracles as the data feed that constantly updates the pricing of items on the menu in our hypothetical restaurant.)
Aave is willing to sell its collateral at a discount because it cares about bad debt. It can’t afford bad debt generated by the collateral value falling below the debt that the borrower owes. Thus, it applies a discount to economically incentivise the liquidator to perform the liquidation.
The discount or liquidation bonus offered is a fixed parameter set by governance, and it usually ranges between 5% and 10% depending on how ‘risky’ or volatile the collateral asset to be liquidated is. This range is inefficient and tends to be higher than it needs to be. This bonus or discount is where lending protocols ‘lose’ value. The ‘loss’ here is the amount protocols (like Aave) lose to pricing inefficiencies from selling the bad collateral.
Note: An off-chain version of the collateral sale happened when Curve’s founder took out a loan. The collateral – as in the case mentioned above – was sold at a discount. We wrote more about it here.
As liquidators are incentivised to take on bad debt, they constantly track oracle feeds for price changes. To ensure that positions are sold on priority, liquidators bid for blockspace with high fees. This way, the value flows from protocols to liquidators to validators. This value leakage is the oracle extractable value or OEV, a subset of the MEV.
To make it easier to understand – protocols like Aave and Compound lose value because they offer liquidation bonuses. A solution to this problem can be that oracles update the price feed and include liquidation transactions at the same time. But they don’t do it because liquidations get centralised if they do so.
The problem here is that in case oracles fail, there will be no liquidations. And lending protocols can’t have that. OEV solutions are essentially about reducing the value leakage without oracles taking control of liquidations. They can pass on the rights to use price feeds (essentially prioritising who sees the price first before everyone does) via an open market for things like collateral liquidations on lending markets. This way, liquidations are not centralised, and a significant chunk of the value is returned to dApps.
According to EigenPhi, liquidation of bad collateral accounts for about 2.9% of all MEV value capture.
It might not seem big on its own, but notice the two charts below. The calculation methodology (from @tumilet) is as follows —
All the liquidations on Aave were aggregated (across chains and v2 + v3) every month. It was assumed that 7.5% of the liquidation bonus went to searchers. (This was based on the searcher profit margin chart above. So 92.5% was left.
Half of this, or 46.25%, went to Aave, and 46.25% went to Chainlink and UMA). We allocated only 50% to Aave because this is what Uma originally proposed.
This number can be higher. Note that these numbers are based on conservative assumptions, and the Oval proposal for Aave mentions that Aave can capture 90% of the liquidation bonus.
If a protocol such as Aave had been able to capture all the value it should have during liquidations, it might have seen a revenue rise of $42 million. This is nearly 50% of its current revenue, which is significant. As the total value locked (TVL) on DeFi platforms increases, we will see increasing slippage.
So, the industry is at a juncture at which it is aware that a significant portion of revenue is going to third parties. Imagine if Robinhood were losing half of its revenue to a third party. That would be … interesting, right?
Something quite similar is happening. But before we tread there, it would help to understand how order flow auctions (OFA) work.
Auction It All
Think of a regular auction. The auctioneer unveils an item and starts collecting bids for it. Auction participants openly convey their bids. After the highest bid is received, the item is sold. DEX aggregators are the first step towards letting traders find the best prices. They look for optimal prices across DEXs and get the best execution price for traders.
The ‘price’ you see on a platform like Uniswap or CoW Swap includes value you may lose to slippage and MEV. A class of products has been built around protecting users from MEV so they get better prices. These products engage in MEV (much like anyone else) but bundle the bulk of the profit (usually 90%) back to the user and keep a small cut for themselves.
This makes MEV a bit of a race to the bottom as everyone will compete for the lowest share in MEV profits for themselves, as users (or wallets) will send their transactions through routes that offer them the best price.
In my restaurant example, order flow auctions are a bit like inquiring with multiple waiters about how much it would cost to get your favourite dish. A smart enough waiter might quote you a high price but offer you the bulk of the premium he’s charging you back as a rebate to compete with his peers. This will eventually lead you to place orders only through that particular waiter.
While the waiter might make lower profit margins on each order, the recurring number of orders (transaction frequency) would help him make hefty profits while pushing others out of competition.
Remember the bit where I said menu items in this restaurant keep updating? What if the waiters and the source for that data could work together to give customers at the restaurant better pricing? It would lead to more customers getting better pricing for their favourite dishes and make them come back. A few start-ups have been doing similar functions on-chain.
Currently, over 10% of all Ethereum transactions (graph above) go through private RPCs or OFAs. It is expected to increase over the next few months as wallets start integrating with OFAs to monetise some of their order flow. Every time you make a transaction through a DEX, there’s a complex machinery of pricing that transaction that happens in the back-end. A few infrastructure players are now focused on helping make the pricing of those transactions far more efficient.
One of them is Oval.
Presently, protocols like Aave or Compound receive their price feeds from Chainlink. Whenever prices change, searchers create bundles that back-run these price updates to determine their profitability. (Back-running means ensuring that your transaction is right after a particular transaction; in this case, it is the price update transaction.)
For instance, the price of ETH has gone from $2550 to $2555, and there was a liquidation due at $2554 because someone’s loan is now under-collateralised. The protocol (Aave) does not know about any of these parameters.
Usually, Chainlink feeds that data and searchers bundle the transaction that does the liquidation. In a pre-Oval world, searchers were forced to give up most of the slippage (or inefficiencies) that came in this transaction. Oval offers a mechanism for part of this value to return to the protocol.
How is this done? Each time there are liquidations, Oval protocols can have searchers bundle transactions so that much of the profit goes back to the protocol (and users). What Oval is doing is inserting an auction mechanism and letting the market decide what a particular price update is worth. Essentially, the auction is to delay the public price update and sell it to the highest bidder a few moments before it is released.
The winner unlocks the price by paying the bid. This fixes the inefficiencies associated with having a governance pre-decided fixed percentage for liquidators, where most of the time, the value goes to validators.
Oval enables Aave to internalise the oversold value and return it to the protocol. The idea is that Oval turns these fixed liquidation bonuses into something dynamic. Oval shares a portion of bids received from searchers with protocols like Compound, opening up a new revenue stream for them. The chart above shows that the potential revenue for Aave through Oval could have been $42 million.
Another solution to turn the fixed liquidation bonus into a dynamic market-driven variable through competition is API3's OEV Network. This solution is an extension of API3 Oracles that introduces an auction system built into them, which allows liquidators to compete for valuable oracle updates.
Like Chainlink, they operate deviation and heartbeat-based oracles but delay them by 15 seconds. Through the OEV Network, real-time data is being auctioned off, and liquidators can win the exclusive right to perform the liquidations that result based on such updates. In effect, API3's model combines traditional push-based oracles such as the one Aave is using today through Chainlink with an auction gated-pull model (akin to Pyth Network) built on top.
What sounds complicated is quite easily explained if you think about it this way –
There is a suitcase with a million dollars 5km away, for which you and nine others are competing by foot. At some point, one of you will get there and win it all by being in better shape or taking the better route (e.g., a more efficient searcher).
But what if there was a guy with a motorcycle there, willing to take one person to the suitcase? What if he only took the person offering him the most money? The highest bidder is guaranteed to reach the suitcase first and make whatever is left after the 'bribe'.
In API3's model, 90-100% of such a 'bribe' is returned to Aave, with protocol fees for API3 being immutably hard-capped at 0-10%. This means that instead of $42 million (at 50%), Aave could have made ~$75 million (at 90%).
The difference between Oval and API3’s solution is that Oval has to wait for Chainlink updates; API3, being the first-party oracle, doesn’t have to wait. This allows API3 to have continuous auctions instead of conducting an auction after every Chainlink update. API3’s base feeds have a 15 seconds delay, which is sufficient for a buyer of a ‘real-time’ feed to have an advantage. The challenge is relative centralisation.
For instance, Oval does its auctions off-chain, presumably for the confirmation time it takes. This can lead to a situation where we recreate the dark pools from Wall Street—a walled-off ecosystem with little transparency. API3 is now trying to bring this process on-chain. They have implemented an app-specific rollup using the Polygon CDK framework, which provides an immutable and transparent record of previous auctions and end-to-end auditability.
All About the Flow
Regulators in the traditional financial ecosystem have differing views on order flow value captured by market-makers or other middlemen. In 2021, Robinhood’s stock took a hit when Gary Gensler hinted that payments for order flow could be banned. That year, market-makers paid ~$3 billion in the US alone for retail brokers’ order flow.
There is no real mechanism to ban payments for order flow because markets constitute multiple parties, and each will devise new mechanisms to extract value.
For a sense of scale, Robinhood earned $650 million from payments for order flow in 2020. According to a report from Galaxy, the size of MEV (TAM) is around $450 million across all avenues. These are still early days. What differentiates the blockchain ecosystem is that there are mechanisms to ensure more transparency and fairer prices for participants.
In traditional finance, market makers currently make all of the money from order flow. In crypto, there is a slight chance that it can be shared with users. It is part of the promise made by products like MEVShare. So, is this approach more democratised? Is it efficient? We don't quite know.
We are still in the early days, but here's what's evident: The tooling for all crypto markets—whether decentralised exchanges, lending, or in-game asset swaps—is rapidly evolving, bringing with it a new subset of opportunities.
In the hypothetical restaurant that blockchains are, sandwich (attacks) are a hot-selling item. Until next time,
Acknowledgement - Ugur Mersinlioglu, from API3 helped with incredible feedback on this piece.
