A way to retrieve a specific value without ctor overhead

[peppy]

In realm-dotnet, there is a GC/alloc overhead associated with traversal. An example usage which can be quite expensive is as such:

var cats = realm.All<Cat>();
var list = new List<string>();

foreach (var c in cats)
    list.Add(c.NameTag.Name);

Even though we are only interested in the cat's name, this will construct every Cat and every NameTag, incurring an alloc/GC overhead, but also a realm handle overhead.

Is there any way to do this kind of query more efficiently? I've tried to find a method using the dynamic API but it doesn't seem readily possible.

[fealebenpae]

There isn't a way to do this more efficiently, I'm afraid. I have had the idea of allowing realm models to be value types, which should help reduce GC overhead, but there would still be the matter of performing the P/Invoke calls to get the Cat and NameTag handles.

I think in this case supporting something like realm.All<Cat>().Select(c => c.NameTag.Name).ToList() would be ideal, and it'd be worth looking into what the Swift SDK did for the Projections feature they released recently.

[peppy]

I think we can actually incorporate that immediately for the said case, great to know!

[peppy]

Interestingly, adding the extra query to the realm filter has quite a large performance overhead:

for (int i = 0; i < 5; i++)
{
    var results = realmFactory.Context
                              .All<BeatmapSetInfo>()
                              .Filter("DeletePending == false && Protected == false && ALL Beatmaps.Hidden == false")
                              .ToList();
    Console.WriteLine($"result 1: {timer.ElapsedMilliseconds}");

    timer.Restart();
    results = realmFactory.Context
                          .All<BeatmapSetInfo>()
                          .Filter("DeletePending == false && Protected == false")
                          .ToList();
    Console.WriteLine($"result 2: {timer.ElapsedMilliseconds}");

    timer.Restart();
    results = realmFactory.Context
                          .All<BeatmapSetInfo>()
                          .Where(s => !s.DeletePending && !s.Protected)
                          .ToList();
    Console.WriteLine($"result 3: {timer.ElapsedMilliseconds}");
}

result 1: 112
result 2: 51
result 3: 26
result 1: 129
result 2: 34
result 3: 27
result 1: 130
result 2: 30
result 3: 43
result 1: 124
result 2: 40
result 3: 45
result 1: 136
result 2: 26
result 3: 45

Because we are already detaching and the case of "all beatmaps being hiddden" is very rare, it would make more sense to do the check post-detach in this specific case, although one could argue that since we are running the realm query asynchronously if done via the Filter it's much-of-a-muchness.

[nirinchev]

I think you may have an error in the query there, which is likely impacting performance. In the first query, you're applying a filter of ALL Beatmaps.Hidden == false, which means you only want BeatmapSetInfos where all beatmaps are visible (i.e. none are hidden). This means that the query engine needs to evaluate every object in the list. If the lists are large, this will easily dominate the evaluation time. I believe (and your original code seems to support it), you actually want to get all sets that have at least one non-hidden beatmap and if the majority of your sets meet that condition and the unhidden beatmap is one of the first elements in the list, the query should complete much faster.

To be clear, collection queries are not free, so I still expect some overhead compared to querying on a bunch of properties, but they should still be faster than doing the filtering in managed code due to the elimination of managed->native transitions and temporary object allocations.

[peppy]

Good catch, double-negatives got the best of me.

Using .Filter("DeletePending == false && Protected == false && ANY Beatmaps.Hidden == false") now as proposed, doesn't seem to give different results though:

result 1: 129
result 2: 47
result 3: 43
result 1: 133
result 2: 35
result 3: 50
result 1: 140
result 2: 28
result 3: 50
result 1: 143
result 2: 53
result 3: 33
result 1: 123
result 2: 50
result 3: 42

[nirinchev]

Surprising - I'm guessing creating the iterator on the core side was more expensive than actually iterating the objects when querying. I'll try to create a simple benchmark reproducing this and bring it up with the team to see if there are obvious areas for improvement.