In part one we introduced LayerZero, its anatomy and key components, and showed the critical flaw of ULNv1. In part 2 we dived into Stargate and found two long-term DoS exploits, which were found just before us. In part 3, we'll persevere with our DoS efforts and finally pick up a bounty in a LayerZero asset.
Back to LayerZero
We've talked a lot about the blocking mechanism, where each (src chain, src address, dst chain, dst address) packet must arrive in order. LZ offers an easy way to opt-out of it, by composing with NonBlockingLzApp from the LZ SDK. It works as follows:
The Base LzApp implements the lzReceive() official entry point, performs basic validation and calls _blockingLzReceive():
NonBlockingLzApp overrides the blocking function:
The logic first tries to call nonBlockingLzReceive(). If it fails, it stores the failed message in a mapping. Later on, anyone can call retryMessage() to redeliver the failed message.
Apps only need to override _nonblockingLzReceive() to be fully-functioning non-blocking apps. The key point is - lzReceive() never reverts in non-blocking mode, so from the LZ Endpoint perspective, the payload was delivered successfully. The architecture is very clean, but what about the implementation?
A theoretical weak spot
Let's take a closer look at the _blockingLzReceive() implementation:
The first statement passes up to gasLeft() amount of gas to the local nonblockingLzReceive() function and reads up to 150 bytes of returndata (using the execessivelySafeCall() utility). Note that it can't actually pass the entire remaining gas due to the 63/64 rule, which forces 1/64 of the gas remaining at the moment of external call to remain in the callee context. But, if nonblockingLzReceive() ends up spending its entire allocation, only 1/64 of the gasLimit specified for delivery of the LayerZero message would remain to store the failed message. We recall from part 2 that Stargate set that limit to 175000 + user's optional amount. In other words, if Stargate used NonBlockingLzApp and there was a way to waste the entire allocated gas (for example, if sgReceive() did not limit the amount of gas sent to user's callback), there would only be 175000 / 64 = ~3000 gas to store the failed message. But that's impossible, because zero to non-zero SSTOREs cost 22.1k gas. So, the revert will bubble up into the Endpoint try/catch handling.
That is the portion which stores the payload and blocks the channel. Since gas is capped to gasLimit here, there's no risk of not leaving enough gas to store the failure in storedPayload, the Relayer is just expected to provide a small extra buffer for running the logic before and after lzReceive().
To unblock the queue, someone would have to re-submit the payload with enough gas to store at the NonBlockingLzApp level, which is a lot! Estimating 25k for storeFailedMessage(), it comes to 1,600,000 total, which is ~$400 as of the gas price and ETH price at time of writing. If the user pays for ~200,000 gas like in Stargate, that would go very little in compensating the Relayer for the losses.
Alright, so there's a weak spot but it can only be triggered when attacker can force delivery to waste the entire gas bank. It's time to look for concrete instances of the issue.
Meet ONFTs
LayerZero introduces two cross-chain token implementations - ONFT (ERC721) and OFTs (ERC20). They're implemented in the SDK. The token contracts are the LZ-facing application and operate by locking the tokens on one side, and minting or releasing them on the other. Let's take a look at how ONFTs are sent and received:
The debitFrom() function transfers tokens from the user's wallet to the ONFT. In checkGasLimit() it verifies the gas amount specified in adapterParams is enough. The minimum amount is the sum of:
A base of minDstGas, determined per chain (usually 260k from the docs)
An extra dstChainIdToTransferGas for each tokenID (100k each from the docs).
Finally _lzSend() calls Endpoint.send() to initiate bridging. On the receiving side, ONFT is a NonBlockingLzApp, so it implements the required function.
It implements basic fragmentation logic - creditTill() will transfer tokens as long as the remaining gas is safe. Afterwards, it stores the next index in storedCredits and exits.
In creditTo() the ONFT finally transfers ownership of the asset. If it exists, which means it was locked on this side before, it uses the ERC721 transfer(). Otherwise, it uses _safeMint(). This is taken directly from OpenZeppelin's code, which respects the ERC721 standard and calls onERC721Received() on the recipient to make sure they accept the transfer. This is where the issue lies - it never caps the amount of gas used by the callback! A malicious receiver could spent the entire 63/64 gas amount it received and revert. It will bubble up to nonblockingLzReceive(), which will have 1/64 of the gasLimit to report the failure and return nicely. Since we've seen the sending side will set gasLimit to 360k for a single token transfer, the behavior we discussed previously will happen, and the payload will be blocked at the Endpoint level.
To freeze the ONFT, a user would bridge the lowest-valued token and pass a malicious contract as the receiver contract. It would then be frozen until one of:
The ONFT owner calls forceResumeReceive() to permanently remove the payload from the Endpoint. That's a pain because:
It may permanently remove a non-malicious asset (What if the receiver legitimately didn't want to receive the asset at this point, and assumes it will be available for a re-submit?
Truly decentralized contracts renounce the owner role, so that option would not be available.
A victim will be willing to lose around 1.5M gas to resubmit the payload
Regardless, until the dust settles and the exploit root and recovery steps are clear, the ONFT would be frozen in each channel that's exploited. The attack is also repeatable.
Disclosure
The issue was disclosed to LayerZero through Immunefi in June 2023. They were quick to respond and confirmed the issue. We argued for high-severity due to the impact being repeatable freezing of funds. We were disappointed to find out that LayerZero specifically caps severity to LOW for OFT and ONFT contracts.
From an expectation of 5-6 figure payout, we were now looking at the $1-$10k range. LayerZero decided to award $5k for the finding, which was very surprising considering it was already a massive drop from the $25k-$250k range for High impact. Since it is within their rights to award anywhere on the range, we didn't dispute it.
The Fix
To be perfectly honest, we're not sure how LZ addressed the issue. We don't see any code changes and aren't going to check the exploit on a live testnet / mainnet. Also simulating the flow on two chains is complex. We've learnt from before that LZ could be hiding their fixes even in unverified contracts, so that demotivated further research.
The straightforward fix is to override _safeMint() to pass the remaining gas for the current transfer iteration, rather than 63/64 of the gas bank.
More broadly, the NonBlockingLzApp soft-spot should absolutely be fixed - it needs to leave ~25k of gas to store the failed payload. This would have addressed not just this particular issue, but for any non-blocking application that has a gas spending surface.
Bug theory
Once again, we combined soft-spot heuristics like callbacks/gas-calculations and a targeted specific impact - reverting in the lzReceive() call.
Series Summary
It felt good to finally pick up a bounty on LayerZero! It wasn't the 8-figures we were dreaming of, but it was still great to identify 3 high-severity issues missed by an incredible amount of auditors and internal reviews.
We're now more confident that LayerZero is a highly secure bridging system, but it needs to step up their game when it comes to communicating issues and how they were resolved. Security by obscurity is never a good idea, and even more so in Web3 where we take pride in full transparency of what code is running to safeguard our assets.
About Trust Security
At TrustSec we combine the depth of bounty hunting and breadth of auditing to provide our clients with the most comprehensive security reviews in the ecosystem. If you're building highly-complex EVM applications, do not hesitate to reach out and discuss how we may be of service.