Basically we should do something along the lines of what https://github.com/ethereum/solidity/pull/11949 tried.