DNS Resolver Issue: Condition ' resolving == IP::RESOLVER_INVALID_ID ' is true. returned: ERR_BUG

[etalot]

Godot version:
3.x any

OS/device including version:
Windows 10

Issue description:
Porting game for social networks from Flash to GODOT.
Game on start loads icons of 190 friends. 190 icons, so it creates 190 HTTPRequests and gets:
Out of resolver queries
Condition ' resolving == IP::RESOLVER_INVALID_ID ' is true. returned: ERR_BUG

All icons images are located on one server! This means that now GODOT can't cache DNS resolve requests! or GODOT DNS resolver cacher works wrong.

With RESOLVER_MAX_QUERIES = 16384 all works great. But it's not correct solution.

Steps to reproduce:
Minimal reproduction project

Minimal reproduction project:
DNSResolver.zip

[akien-mga]

CC @Faless

[etalot]

UPDATE: Minimal reproduction project is added.
with "HTTPRequest.use_threads = true" no issue.
.use_threads by default is false
What is the difference for DNS resolver/cacher .use_threads true or false?

[etalot]

UPD:

with "HTTPRequest.use_threads = true" no issue.

This is true for Windows Editor F5 run project.

EXPORTS:

Android apk always flows logcat with "Condition ' resolving == IP::RESOLVER_INVALID_ID ' is true. returned: ERR_BUG" and make high CPU load. use_threads false or true make different bugs

HTML5
use_threads = true:
ERROR: HTTPClient blocking mode is not supported for the HTML5 platform

use_threads = false:
works ok

PS: I have own DNS server (in cacher mode and full logging enabled) and can help with checking Godot DNS resolve cacher.

[Faless]

What is the difference for DNS resolver/cacher .use_threads true or false?

Just timing, when using threads it polls more often so it likely do not get to the saturation point.

The problem is that a new ID is generated even if that hostname is already in queue. Returning the same ID is not going to be an option though, otherwise there might be conflicting erase...

ERROR: HTTPClient blocking mode is not supported for the HTML5 platform

Yes, threads are not supported in JS.

so it creates 190 HTTPRequests

That's not good, you should try to queue them, and do 4/5 at a time, definitely not 190 concurrent HTTPRequests to the same hostname.

[etalot]

190 concurrent HTTPRequests to the same hostname

Same like facebook.com is in reality not only one host. Now it's trend use load balance technique even with servers with statics data. =)
HTTPRequests make micro freezes, as result FPS falling. Or if use queue as result very long loading. So in some cases I prefer to do a lot of HTTPRequests while user will see screen like "Loading resources..." for few seconds. After show for him good animation with good FPS.
So I think it's better developer free to choice how to do in different situations.
I would like to to see this fix in 3.1 if possible.
Anyway RESOLVER_MAX_QUERIES = 16384 fix this issue.

[Faless]

Same like facebook.com is in reality not only one host.

I said hostname not host.
Probably DNS hostname would have been clearer.

HTTPRequests make micro freezes, as result FPS falling.

This shouldn't happen. I'll try to investigate.

So I think it's better developer free to choice how to do in different situations.

I agree, but again, 190 concurrent request is not good practise, even web browser have (much lower) default limits

Anyway, I agree this should be fixed.

Anyway RESOLVER_MAX_QUERIES = 16384 fix this issue.

I don't think that's the right fix, it's an hack which will waste user memory 99,99% of the time without really fixing the underlying issue (that a new resolver ID is generated even when the hostname is already in cache or in queue).
A proper fix needs rework in IP internals and most likely API. Which is why I set it to 3.2.

We could temporary raise RESOLVER_MAX_QUERIES as a workaround in 3.1 as you suggested (although 16Ki seems a bit too much).

[etalot]

This shouldn't happen. I'll try to investigate.

Truly saying I'm not sure about HTTPRequests. Maybe DNS resolver, or image processing/loading to texture in result, or loop processing result API response or even print debug console messages.

We could temporary raise RESOLVER_MAX_QUERIES as a workaround in 3.1 as you suggested (although 16Ki seems a bit too much).

This would be good. Thank you! Any value bigger than 32

I made queue like you advised me, with checking FPS while processing queue(if lower than threshold not load new images). Result looks very good. Thank you for advice!

[KoBeWi]

Seems still valid in 3.2.4 beta3

[martinpelli]

Hi I am having the same problem on 3.2.3 windows 64, could you explain me how to fix with RESOLVER_MAX_QUERIES = 16384?, I don´t know what this mean.

[Schroedi]

Still relevant in v3.4.beta2.official

@martinpelli I guess it means to change the maximum here:

godot/core/io/ip.h

Line 59 in 9990f28

RESOLVER_MAX_QUERIES = 32,

You have to compile the engine and export templates after the change.

[INdIEDeV016]

Reproducible even in v4.0.alpha.custom_build [49176bc0f] Godot 4.0 Alpha 4+

I don't know if it's relevant but in my case, the error spam occurs for a particular type of resource. If the resource is deleted from the server then the error spam is resolved.

[Mickeon]

I had this error occur when downloading several images at once from an URL. To address it, I manually limited the requests at once by counting them on a queue and firing them as soon as the queue has a free spot.

[Faless]

I had this error occur when downloading several images at once from an URL. To address it, I manually limited the requests at once by counting them on a queue and firing them as soon as the queue has a free spot.

We raised the limit now, but I suggest you keep such a system in place.

And I advise everyone to be mindful that doing too many concurrent requests will result in worse network performances than doing a limited number at a time.

Also note that some platforms (e.g. the HTML5 platform) limits the number of concurrent requests (e.g. most browsers limit it to 6, the original HTTP standard actually had a hard limit of 2 connections which was later removed from the standard).