Fuzzing Complex Projects With Echidna: Sushi's BentoBox

December 21, 2021 by patrickd

During Secureum Bootcamp's second phase (opens in a new tab) "CARE" (Comprehensive Audit Readiness Evaluation), participants were asked to review their assigned Project for typical Pitfalls and Best Practices (opens in a new tab) to help prepare the project for an actual Audit. Since we were free to attempt going further than that, I wanted to try my hand at fuzzing with Echidna (opens in a new tab) and quickly realized that it wouldn't be easy to do that in this case.

The problem is that Echidna expects a rather simple environment: To run it, you need to specify a Solidity source file, a single Contract to fuzz, and the constructor must not require any parameters. The Project we were assigned to, on the other hand, Sushi's BentoBox-Strategies contracts (opens in a new tab) (Solidity 0.8.7), requires the BentoBox contract (opens in a new tab) to be deployed (Solidity 0.6.12), and both have 3rd party dependencies such as WETH9 (opens in a new tab) and Aave (opens in a new tab).

The BentoBox

BentoBox (opens in a new tab) is a Platform you can transfer your Tokens into, to make use of various Applications (currently only Kashi Lending (opens in a new tab), but the AMM Trident (opens in a new tab) and other things have been announced). Additionally, Sushi's Operations Team is able to decide on Investment Strategies for a percentage of the Tokens that are currently unused within the BentoBox.

To make things easier to grasp, we'll concentrate on a simplified use-case: A User deposits Ether into the BentoBox, without using any particular Application for now. BentoBox wraps them into WETH Tokens and at some point, they are rebalanced into the investment Strategy that the Sushi Ops team has set for WETH. This particular Strategy uses Aave to earn interest on the provided liquidity. For investing the WETH Tokens, the Strategy contract gets aWETH Tokens that increase over time and can be exchanged for the same amount of WETH when withdrawn from Aave. Sometime later the Ops Team decides to switch to another, more promising Strategy and the current one is exited, expecting the full amount of WETH (+ earned interest) to be returned into the BentoBox.

Using fuzzing we now want to find out whether there's any case where the Strategy might fail to correctly send back all funds when this happens.

Initialization via Remix

For contracts with complex initialization, the official Tutorial (opens in a new tab) recommends using Etheno and the Project's Testsuite (opens in a new tab) to basically record the transactions made when running a single test and replay them when initializing Echidna. The tests of the AaveStrategy however, make use of a chain-fork from an archive node - that might be a bit much for a simple initialization recording.

But why use tests if we can make use of Etheno (opens in a new tab)'s recording functionality and take care of the initialization by hand just once?

ubuntu@eth:~$ etheno --ganache --ganache-args "--deterministic --gasLimit 10000000"  -x init.json
INFO     [12-18|16:48:40][Ganache@8546] Ganache CLI v6.12.2 (ganache-core: 2.13.2)
INFO     [12-18|16:48:40][Ganache@8546]
INFO     [12-18|16:48:40][Ganache@8546] Available Accounts
INFO     [12-18|16:48:40][Ganache@8546] ==================
INFO     [12-18|16:48:40][Ganache@8546] (0) 0x5409ED021D9299bf6814279A6A1411A7e866A631 (100 ETH)
...
INFO     [12-18|16:48:40][Ganache@8546] Private Keys
INFO     [12-18|16:48:40][Ganache@8546] ==================
INFO     [12-18|16:48:40][Ganache@8546] (0) 0xf2f48ee19680706196e2e339e5da3491186e0c4c5030670656b0e0164837257d
...
INFO     [12-18|16:48:40][Ganache@8546]
INFO     [12-18|16:48:40][Ganache@8546] Listening on 127.0.0.1:8546
INFO     [12-18|16:48:40][Ganache@8546] eth_accounts
Etheno v0.2.4
 * Serving Flask app 'etheno.etheno' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8545/ (Press CTRL+C to quit)
 

If you are running this on the same host as your Browser, you can simply open your Metamask (opens in a new tab) extension and switch to the Network "Localhost 8545". Then look for the "Import Account" option and paste in the private key of the Account (0) from the Ganache logs. Within Remix (opens in a new tab), open the "Deploy & Run Transactions" sidebar and select the Environment "Injected Web3". All of the contract deployments and interactions you now make within Remix will be written into the init.json file, which can be imported into Echidna.

So, what should be recorded? Let's trace the dependencies the AaveStrategy (opens in a new tab) has:

contract AaveStrategy is BaseStrategy {
    ...
    constructor(
        ILendingPool _aaveLendingPool,                  // Aave LendingPool contract
        IAaveIncentivesController _incentiveController, // Aave IncentivesController contract
        BaseStrategy.ConstructorParams memory params
    ) BaseStrategy(params) {
    ...

abstract contract BaseStrategy is IStrategy, Ownable {
    ...
    struct ConstructorParams {
        IERC20 strategyToken;       // Token to invest (eg. WETH)
        IBentoBoxMinimal bentoBox;  // BentoBox instance
        ...
    }
    ...
    constructor(ConstructorParams memory params) {
    ...

contract BentoBox is MasterContractManager, BoringBatchable {
    ...
    constructor(IERC20 wethToken_) public { // specifically WETH (which has no dependencies)
    ...

While we could deploy the real Aave contracts since they're open source, that would likely make things a lot more complicated than necessary. Instead, it would be easier to assume Aave works as expected and only mock out the functions that the Strategy is actually interacting with. Aside from that, we also need an instance of BentoBox and a "Strategy Token" which for this use-case will be the wrapped ether contract "WETH9", which BentoBox has a dependency on anyway.

The fact that AaveStrategy's constructor needs some parameters, can be taken care of via the constructor of a "proxy" contract that does not, by using constants. That proxy can also take care of initializing the Aave mock contracts. BentoBox and WETH9 on the other hand use incompatible, older Solidity versions and it makes sense to record their deployment.

So first we'll copy WETH9's source code (opens in a new tab) into Remix, compile it with 0.4.18 and deploy it. After that, we'll copy a flattened version of BentoBox (opens in a new tab) that has already been conveniently adjusted to allow switching between Strategies without the usual 2 week waiting period, compile it with 0.6.12 and deploy it too. Finally, we want to directly transfer ownership of the BentoBox to where our "proxy" contract will be deployed to by Echidna, which we can find out by checking the default configuration (opens in a new tab), and calling transferOwnership(0x00a329c0648769a73afac7f9381e08fb43dbea72, true, false).

After stopping the Etheno command, you'll find all of the above transactions recorded in the init.json file. To avoid Echidna complaining about parsing errors when reading it, adjust the values of the gas_price and value fields from numbers to strings.

Initialization via "Proxy" constructor

Having taken care of dependencies with incompatible Solidity versions, the leftover initializations can be done by making use of the "Proxy Pattern" (opens in a new tab) (not to be confused with the delegate-call-proxy patterns):

pragma solidity 0.8.7;
 
import "../strategies/AaveStrategy.sol";
import "@openzeppelin/contracts/token/ERC20/ERC20.sol";
 
contract EchidnaProxy {
 
    // Deployed via replay of transactions from init.json:
    IBentoBox public constant bentobox = IBentoBox(0xd2a5bC10698FD955D1Fe6cb468a17809A08fd005);
    ERC20 public constant weth9 = ERC20(0x635B3a867D360C1ECf382A8d695D3092c1e15237);
 
    AaveStrategy public aaveStrategy;
 
    constructor() {
        // Initialize Strategy.
        AaveStrategy aaveStrategy_ = aaveStrategy = new AaveStrategy(
            ILendingPool(/* TODO */),
            IAaveIncentivesController(/* TODO */),
            BaseStrategy.ConstructorParams(
                weth9,             // strategyToken
                bentobox,          // bentobox instance
                address(this),
                address(0x0),
                new address[](0)
            )
        );
 
        // Activate the Strategy for the WETH token in BentoBox.
        bentobox.setStrategy(weth9, aaveStrategy);       // Propose.
        bentobox.setStrategy(weth9, aaveStrategy);       // Activate (2 week grace period bypassed).
        bentobox.setStrategyTargetPercentage(weth9, 50); // 50% of WETH balance should be invested.
    }
 
}

This Proxy contract will be our fuzzing target for Echidna. The AaveStrategy is (not quite yet) initialized to invest the WETH token and then activated within BentoBox (which the Proxy is able to do since ownership has been transferred to it when init.json is replayed).

If we had chosen to also deploy the real Aave contracts when creating that recording, we could simply specify those addresses here for ILendingPool and IAaveIncentivesController. But instead, we'll write some mock contracts simulating them in the most basic way:

// Contract that will be "aWETH" while holding "WETH".
contract AaveTokenMock is ERC20 {
    constructor(ERC20 strategyToken_, string memory symbol_) ERC20(symbol_, symbol_) {
        strategyToken_.approve(msg.sender, type(uint256).max);
    }
 
    // No authentication since this contract is for testing only and won't be fuzzed.
    function give(address to, uint256 amount) external {
        _mint(to, amount);
    }
 
    function take(address from, uint256 amount) external {
        _burn(from, amount);
    }
}
 
contract LendingPoolMock is ILendingPool {
 
    ERC20 public immutable strategyToken;
    AaveTokenMock public immutable aToken;
 
    constructor(ERC20 strategyToken_, string memory aTokenSymbol_) {
        strategyToken = strategyToken_;
        aToken = new AaveTokenMock(strategyToken_, aTokenSymbol_);
    }
 
    // Called by AaveStrategy when new balance was added for investment.
    function deposit(address asset, uint256 amount, address, uint16) external override {
        aToken.give(msg.sender, amount);
        ERC20(asset).transferFrom(msg.sender, address(aToken), amount);
    }
 
    // Called by AaveStrategy when profits are taken or when the Strategy is exited.
    function withdraw(address asset, uint256 amount, address) external override returns (uint256) {
        aToken.take(msg.sender, amount);
        ERC20(asset).transferFrom(address(aToken), msg.sender, amount);
    }
 
    ...
 
}
 
contract AaveIncentivesControllerMock is IAaveIncentivesController { ... }

With this kind of mocking, the AaveStrategy should be able to exchange between the strategy token (WETH) and the aToken (aWETH) without issues - but also without interest. Later, the mock contracts can be extended with functions that give profits, rewards and even simulate losses. We can make Echidna explore these scenarios by adding fuzzable util functions like aave_giveReward(uint256 amount) that will call them.

The fuzzing campaign can also be made more powerful by adding the option of using multiple strategy and aTokens instead of just using WETH. But for now let's keep things simple and just get the fuzzer running with some guardrails on, after it turns out that it's unable to find anything this way, we can always allow for more complexity.

Fuzz-targets

As things stand, there's still nothing Echidna can actually fuzz yet, so let's add some functions to the Proxy that should be its targets:

contract EchidnaProxy {
    ...
 
    function bentobox_deposit(uint256 amount, uint256 share) payable external {
        bentobox.deposit{value: msg.value}(address(0x0), address(this), address(this), amount, share);
    }
 
    function bentobox_withdraw(uint256 amount, uint256 share) external {
        bentobox.withdraw(address(0x0), address(this), address(this), amount, share);
    }
 
    function bentobox_setStrategyTargetPercentage(uint64 targetPercentage) external {
        bentobox.setStrategyTargetPercentage(IERC20(weth9), targetPercentage);
    }
 
    function bentobox_exitStrategy() external {
        bentobox.setStrategy(weth9, IStrategy(address(0x0)));
        bentobox.setStrategy(weth9, IStrategy(address(0x0)));
    }
 
    function strategy_safeHarvest(
        uint256 maxBalance,
        bool rebalance,
        uint256 maxChangeAmount,
        bool harvestRewards
    ) external {
        aaveStrategy.safeHarvest(maxBalance, rebalance, maxChangeAmount, harvestRewards);
    }
 
    ...
}

Here again, complexity was reduced by hardcoding function parameters, which can be exposed to Echidna later if necessary. In general, you'll need to find a balance here between too much exposure, where Echidna will find itself looking for a needle in the haystack, and too many assumptions, where an issue can't be discovered anymore since the guardrails prevent Echidna from doing so.

The Testcase

Finally, we have to give Echidna a way to check whether something has gone wrong while it poked at the BentoBox:

contract EchidnaProxy {
    ...
 
    // An exited AaveStrategy never has funds left in it.
    function echidna_exitedStrategyNeverHasFunds() public view returns (bool) {
        bool strategyWasExited = aaveStrategy.exited();
        uint256 aTokenBalance = lendingPool.aToken().balanceOf(address(aaveStrategy));
        uint256 strategyTokenBalance = weth9.balanceOf(address(aaveStrategy));
        // Property returns true (== state is valid) as long as either
        // - the Strategy was not exited yet
        // - the Strategy was exited and no funds are left
        return (!strategyWasExited || (aTokenBalance == 0 && strategyTokenBalance == 0));
    }
 
    ...
}

For the fuzzer to pick these functions up as testcases, they must be part of the abi (external/public), return a boolean value, and be prefixed with echidna_. Aim to write as many of these properties as you can come up with early on, since Echidna can check them "all at once" it would be a waste to have to rerun it for new testcases later.

Now before starting up the fuzzer I recommend doing a dry run by playing through this as the fuzzer would. Switch Remix back to its "Javascript VM" and deploy all of the contracts and their dependencies, just like we did when creating the init.json. Call each of the fuzzing target functions and check whether they had the expected effects. Between each call, check the testcase functions and make sure they all return true when you expect them to. Once confident that the environment you've built for Echidna works properly, it's time to start fuzzing.

Finally, Fuzzing!

To tell Echidna to initialize using the transaction log in init.json, we need a simple configuration file:

initialize: contracts/echidna/init.json # the transactions to replay for initialization
contractAddr: "0x00a329c0648769a73afac7f9381e08fb43dbea72" # where EchidnaProxy should be deployed to

After that, we only have to tell Echidna where to find the Proxy contract, its name, the config file, and make sure it knows that the OpenZeppelin contract dependencies can be found within the node_modules folder:

ubuntu@eth:~/bentobox-strategies$ echidna-test contracts/echidna/EchidnaProxy.sol --contract EchidnaProxy --crytic-args "--solc-remaps @openzeppelin/=$(pwd)/node_modules/@openzeppelin/" --config contracts/echidna/config.yaml
 
Loaded total of 0 transactions from corpus/coverage
Analyzing contract: /home/ubuntu/bentobox-strategies/contracts/echidna/EchidnaProxy.sol:EchidnaProxy
 
echidna_exitedStrategyNeverHasFunds: passed! 🎉
 
Corpus size: 1
Seed: -6400691107094289626

And now it's time to iterate: Tweak the fuzzing exposure, open up for more search space or narrow down to make it focus on a specific part of the code. For example, at the moment all of the function calls are made via the EchidnaProxy contract, which has Owner and Executor access roles, it would make sense to allow the fuzzer to make unauthenticated calls too. Things like that should be considered when adjusting the environment while waiting for the previous campaign to finish.

Tweak, fuzz, repeat