clear_prefix iterates only over backend ignoring the overlay

[pepyakin]

While deciding which keys to remove from the storage, clear_prefix iterates only over the backend and ignores the changes in the overlay.

Example:

# 1. At the `Externalities` creation time we have the following state: 
backend = ["dog" => 1, "cat" => 2], overlay = []

# 2. Next `set_storage("doggy" => 3)` is called
backend = ["dog" => 1, "cat" => 2], overlay = ["doggy" => 3]

# 3. Next `clear_prefix("dog")` is called (and we iterate only over backend keys)
backend = ["dog" => 1, "cat" => 2], overlay = ["dog" => None, "doggy" => 3]

# 4. Transaction commits
backend = ["cat" => 2, "doggy" => 3]

[snd]

seems to me that the only (default) impl of StorageDoubleMap::remove_prefix calls runtime_support::storage::unhashed::kill_prefix which calls runtime_io::clear_prefix which calls ext.clear_prefix or ext_clear_prefix...

this here could be the culprit https://github.com/paritytech/polkadot/blob/master/substrate/state-machine/src/ext.rs#L124

[snd]

to me this looks less like a problem with remove_prefix than with the transaction commit where the overlay is ignored

[pepyakin]

Oh sorry @snd I think I've just expressed myself poorly. There is no problem that values from overlay is ignored.

I've meant that remove_prefix (or more specifically Ext::clear_prefix) iterates over keys with the given prefix only from backend. But there also could be some values with the given prefix accumulated in the overlay (see #2 in the example above), which will not be marked as deleted in the overlay.

I believe that #436 should fix this properly removing these accumulated values!