StorageIterator does not work

[RavenXce]

Please see neo-project/neo-devpack-dotnet#19 (comment) for example reproduction.

The issue is that when doing the prefix building in Storage.Find, the contract script hash and storage prefix are simply concatenated.

neo/neo/SmartContract/StateReader.cs

Line 951 in f49f266

prefix = context.ScriptHash.ToArray().Concat(prefix).ToArray();

However, for Storage.Put/Get, I believe that serialization uses WriteVarBytes for the key portion:

neo/neo/Core/StorageKey.cs

Line 46 in 07dd1f5

writer.WriteVarBytes(Key);

Which prefixes the length of the key:

neo/neo/IO/Helper.cs

Line 190 in 115a402

writer.WriteVarInt(value.Length);

I think we can safely construct and then serialize a StorageKey instead to ensure they match?

Edit: Actually, the length prefix messes up the whole implementation - one way to make it work is by specifying key length, so the keys iterated on are constrained to a single length and must be known.

[ixje]

For what it's worth, in neo-python the ScriptHash + prefix concatenation is workable, we just had to fix Storage Put/Get/Delete calls because they were creating a key that was not prefix searchable (e.g. ScriptHash + murmur_hash(key) -> convert to bigint then get bytes as key). The same might apply in C#, not sure

[RavenXce]

Since the keys in the db change does that mean that nodes need to re-build storage on upgrade?

The other solution here is to change how Put/Get/Delete works as well, but I thought that would be more complicated.

[ixje]

Yes for neo-python the user will have to rebuild storage. There was no way around it because previously the following 2 Put calls in a smart contract

    Put(ctx, 'my_prefixA', 1)
    Put(ctx, 'my_prefixB', 2)

would generate the following DB keys:

b'^\xbe\xe4\xf4\x00'
b'\xaa.\xd5t'

It was deterministic befor and would save space, but as can be seen that would never reasonable be prefix searchable.

[RavenXce]

I guess thats a possible solution here too.

I've tested and can confirm the issue is due to the key prefix as mentioned in OP. Manually prefixing an additional key length in addition to the desired prefix allows keys with that length and prefix to be returned.