Recovering Safe Wallet Signatures

I recently presented some analysis at the Ethereum Engineering Group Meetup (Youtube and Slides) on the WazirX hack. The analysis answered a recurring question that kept surfacing in my mind: Was it more cost-effective (time/complexity) for the WazirX attacker (Lazarus) to find a vulnerability or bypass in Liminal Custody, or to compromise a 4th WazirX signatory? Answering this question is a challenge and requires you to calculate hashes from specific states and then recover the signers.

To further my understanding, I set a learning objective: recover the signing addresses for the WazirX multisig from on-chain data. This investigation allowed me to explore tools like Alloy and Anvil while connecting various pieces of knowledge I had about Ethereum and the EVM.

If you want to catch up on the WazirX hack you can read my previous post.

onchain is a Rust CLI application that deploys a custom signature decoder contract and iterates through events emitted by Gnosis Safe Wallets in chunks of 10,000 blocks to find occurrences of the event ExecutionSuccess.

The source code for the project is here.

Upon detecting an event, the application constructs a Safe Transaction Hash and a Safe Data Hash using the nonce and threshold from the block prior to the transaction. Tools like Alloy support contract calls at specific block numbers using .block(). Querying the nonce, threshold, and transaction input (to decode the calldata) can be done via a standard Ethereum RPC, so forking the main-net state isn't necessary.

A custom Safe Decoder contract is deployed to a local Anvil chain by the onchain program. This decoder contract is used to extract signers when provided with the Safe Transaction Hash, Safe Data Hash, threshold, and signatures parameters.

Putting it all together;

1. Find all past transactions for the WazirX Safe Wallet that emit an event for ExecutionSuccess. 

2. Decode the transaction input or calldata with respect to the ABI for the executeTransaction function giving you all the Safe Wallet transaction parameters including the operation type and the signers.

3. Query the nonce and the threshold from the block before the transaction to calculate the Safe Transaction Hash and the Safe Data Hash.

4. Pass the transaction hash, data hash, signatures and threshold to SafeDecoder and return a list of the signers.

Safe Global provides an API that returns signers if you know the safe transaction hash. For example, you can use;

https://safe-transaction-mainnet.safe.global/api/v1/multisig-transactions/0x4e82121a3bc2fb62c0b06ab5fff5ca965ceab4f51cc949c6e50d85ed63e6aa70/confirmations/

By passing in the transaction Safe Transaction Hash (not the Ethereum transaction hash), the API will return the list of signers and their signatures. Here's an example of the response:

{
  "count": 4,
  "next": null,
  "previous": null,
  "results": [
    {
      "owner": "0x9AF78003CecC2383d9D576A49c0C6b17fc34Ae34",
      "submissionDate": "2024-07-18T06:17:47Z",
      "transactionHash": null,
      "signature": "0x3da7c6bd7c130430cf662de3d9af067c4a0629f849e29003c40ad3979cd670720c3ceb3a61e38fb7166353e3262eb6554c3f6571807cf22f9bdc4a83a56441661f",
      "signatureType": "ETH_SIGN"
    },
    {
      "owner": "0xD83b89E261D02B0f2f9E384B44907f8d380E9AF0",
      "submissionDate": "2024-07-18T06:17:47Z",
      "transactionHash": null,
      "signature": "0x8e64a89386af2f223b8433a1df65db8f0ff60544b2f02c56ec02b640d6fb15a11f7dffda5b99b0292b5e02d0cc44508d7d8f994515358e9da3016d8098b2258820",
      "signatureType": "ETH_SIGN"
    },
    {
      "owner": "0xd967113224C354600B3151E27Aaba53e3034f372",
      "submissionDate": "2024-07-18T06:17:47Z",
      "transactionHash": null,
      "signature": "0x000000000000000000000000d967113224c354600b3151e27aaba53e3034f372000000000000000000000000000000000000000000000000000000000000000001",
      "signatureType": "APPROVED_HASH"
    },
    {
      "owner": "0xfA54B4085811aef6ACf47D51B05FdA188DEAe28b",
      "submissionDate": "2024-07-18T06:17:47Z",
      "transactionHash": null,
      "signature": "0x40082eba0f71627a3451d47132b8f4c266bc9be2bebe4424848757d10f13e76963af0b543a2357765d9f290410e919301ad065f0f2efa1233eb8634c93111f0e20",
      "signatureType": "ETH_SIGN"
    }
  ]
}

I didn’t have the Safe Transaction Hashes for the WazirX Safe Wallet until I calculated them from on-chain data. Once I had them, I found it easier to use my own decoder contract. However, I could have also submitted them to Safe’s API.

When you examine the logs of execTransaction() on a Safe Wallet, you'll notice that it emits an ExecutionSuccess event. To find the corresponding event signature, you take the name of the event along with its parameters and compute a Keccak-256 hash to generate the topic hash.

For example in Chisel;

➜ keccak256("ExecutionSuccess(bytes32,uint256)")
Type: bytes32
└ Data: 0x442e715f626346e8c54381002da614f62bee8d27386535b2521ec8540898556e

Note: Be sure to remove any whitespace and parameter names, leaving only the types when calculating the hash.

In the onchain application, the event_signature is defined, and the RPC is used to process 10,000 blocks at a time, searching for transactions that match the event signature:

// keccak256("ExecutionSuccess(bytes32,uint256)")
let execution_event = b256!("442e715f626346e8c54381002da614f62bee8d27386535b2521ec8540898556e");

let filter = Filter::new().address(safe_wallet_address).event_signature(execution_event).from_block(from_block).to_block(to_block);

To improve efficiency, it's crucial to filter by the safe_wallet_address and search for all events emitted by that contract. This is much faster than looking for events across all contracts. Once the logs are iterated over, the transaction_hash can be used to query the RPC provider for the calldata (input) associated with the transaction. This calldata is then decoded using the ABI for execTransaction().

Alloy makes it really easy to incorporate an ABI for a function using the solc! Macro. For example;

sol!(
    #[allow(missing_docs)]
    #[derive(Debug)]
    enum Operation {
        Call, 
        DelegateCall,
    }

    #[allow(missing_docs)]
    #[derive(Debug)]
    function execTransaction(
        address to,
        uint256 value,
        bytes calldata data,
        Operation operation,
        uint256 safeTxGas,
        uint256 baseGas,
        uint256 gasPrice,
        address gasToken,
        address payable refundReceiver,
        bytes memory signatures
    ) public payable virtual returns (bool success);
);

Then I can use these types to decode the transaction input;

let decoded = execTransactionCall::abi_decode(&tx.unwrap().input, false);

execTransactionCall is generated for you without you needing to do anything else except define the solc! macro. Notice I was also able to define the enum Operation in the same declaration.

It begs the question as to why I want these decoded parameters to the original execTranscation(). The parameters are required to calculate the Safe Transaction Hash by calling getTransactionHash() and the Safe Data hash by calling encodeTransactionData().

For the nonce and threshold I call the contract and get the values for the block before the transaction that emitted the event. This is a little naive as there could be multiple transactions in a block but for the investigation of WazirX it was fine.

I knew the signers were returned by the ecrecover function in the Gnosis Safe Wallet checkNSignatures() and I could see it when I ran the following cast command;

cast run 0x48164d3adbab78c2cb9876f6e17f88e321097fcd14cadd57556866e4ef3e185d --verbose --rpc-url=$ETH_RPC_URL --no-rate-limit --decode-internal

However there was no way to programmatically access them. I toyed with the idea of porting checkNSignatures() to Rust however I wanted to do just enough to solve the problem and no more.

What I ultimately wanted was a function like checkNSigatures() but have it store and return an array of addresses for all the signers. So I copied the bulk of checkNSignatures() and required libraries across to SafeDecoder’s returnSigners().

I removed all the require statements and appended each address to an array of addresses then returned them at the end of the function call. This snippet is a good example of the approach;

After returnSigners() iterates through all the signature data, and recovers all the signers it returns them as an array at the end of the function.

I could even test using forge test, see the tests here;

onchain deploys the Safe Decoder Contract and then calls returnSigners() to return an array of signing addresses.

To be able to deploy SafeDecoder you need to execute forge build and then navigate to the out/ directory and find the JSON build artifact for SafeDecoder. See the figure below;

Within this JSON file is the bytecode for the SafeDecoder contract that will be deployed on the local Anvil chain. In the SafeDecoder.json file there are two bytecode sections - bytecode and deployed_bytecode. We want to use the bytecode.

The bytecode needs to be copied to the bytecode value in the solc! declaration for the SafeDecoder contract. I’ve truncated it here with … at the end.

// This is the decode contract in bytes. You need to build using forge build then navigate to the build `out` directory, 
// find the SafeDedoder.sol json and copy out the bytecode (not the deployed bytecode) and copy here.
sol! {
    #[allow(missing_docs)]
    // Needs to bytecode not deployed_bytecode.
    #[sol(rpc, bytecode="0x6080604052348015610010576000...")]
    contract SafeDecoder {
        address[] public addresses;
    
        function returnSigners(
            bytes32 dataHash,
            bytes memory data,
            bytes memory signatures,
            uint256 requiredSignatures
        ) public returns (address[] memory){}
    }
}

With the SafeDecoder ABI and contract defined it can be deployed and then passed the parameters it needs to return the signers - Safe Transaction Hash, Safe Data Hash, signatures and threshold.

let signers = safe_decoder
    .returnSigners(
        tx_hash.clone(),
        data_hash.clone(),
        decoded.signatures,
        threshhold._0
    )
    .call()
    .await;

onchain will start and print comma-separated values to stdout. The WazirX safe wallet contract address, start and end blocks are set as default values. 

#[derive(Parser, Debug)]
#[command(author, version, about, long_about = None)]
struct Args {
    #[arg(long, default_value = "0x27fD43BABfbe83a81d14665b1a6fB8030A60C9b4")]
    safe_wallet_address: String,

    #[arg(long, default_value_t = 16132578)]
    start_block: u64,

    #[arg(long, default_value_t = 20331566)]
    end_block: u64,

    #[arg(long, default_value_t = 10000)]
    chunk_size: u64,
}

If you run onchain and pipe to a file you end up with something like transactions.csv. It details every WazirX Safe Wallet transaction and what addresses signed it. If you execute the wazir-signers.ipynb python notebook you’ll see it creates a histogram of signers. 0xd967113224c354600b3151e27aaba53e3034f372 is the Liminal Co-Signer HSM. 

0xd83b89e261d02b0f2f9e384b44907f8d380e9af0    1713
0xd967113224c354600b3151e27aaba53e3034f372    1709
0x9af78003cecc2383d9d576a49c0c6b17fc34ae34    1699
0xfa54b4085811aef6acf47d51b05fda188deae28b    1666
0xae648f68823bc164ca3ad1f5f5dc0057d9d515ad      51
0x10f16cde93f1bc9c38a9e31c8db0eeb89a744824      29
0x711d01b75529aa700e821dc330a0addd2d869b0b       1

The histogram demonstrates there were 4 dominant signers and although the signature was a 4/6 there were three signers that were not active. Some of them for months and through to years.

• 0xae648f68823bc164ca3ad1f5f5dc0057d9d515ad not seen since Apr-13-2024.

• 0x10f16cde93f1bc9c38a9e31c8db0eeb89a744824 not seen since May-28-2024.

• 0x711d01b75529aa700e821dc330a0addd2d869b0b not seen since Dec-09-2022.

The attack happened on July 18th 2024 and if the adversary analyzed the signatures they would likely conclude that it was going to be easier to find some bypass in Liminal than having to compromise another WazirX Signer. Like alot of high value multisig wallets the quorum was optimised for availability rather than increasing the cost to any potential adversary.

As I noted in the talk a 4 of 6 multisig where the same signers always sign means an adversary has to compromise the least number of signers. If any of the signers can sign any transaction then the adversary must compromise more of the quorum. Scaling trust to more signers and introducing randomness to the signing process increases the cost to the adversary.