Text editor lagging on script heavy projects

[MeiHouwang]

OS: Win7
PC specs: 4-core i5, 32Gb RAM
Godot version - 2.1.3 stable

On a script heavy project with a large scripts and many cross-references text editor starting to lag formidably on every syntax check/code competition. Increasing delays helps to type text but does not fix lags at all. Performance check shows a large spikes of CPU time consumption during text entry - up to 100% of one core usage. This really makes editor extremely uncomfortable to use in a large projects.

I still can't construct refined example project to show it seems(I'll try some more, it took, three months of coding to make my real project to lag this much).
My working project(that I still too embarrassed to show, sorry) have a heavy game logic and have 620 lines of code in the most lagging script, arount 40 functions in it, 7 imported scenes with heavy scripts and 8 scenes loaded from script also with scripts, and on top of that 4 autoload singleton scripts.

[Zylann]

I didn't notice such a lag in my game with Godot 2.1.3, where the biggest script was around 800 lines at some point before I refactor it down to 500. However I didn't test it on 2.1.4.
Do you have plugins enabled? Do you have a scene open at the same time? Does it lag with no scene open?

[volzhs]

I didn't notice that either, with 2.1.3 or 2.1.4
I'm using Ubuntu 16.04 with i5(4-core), 16GB ram.
And I have a script has 1100 lines with 80 functions, but I've not had any noticeable lagging.
There might be some specific condition for it.

[MeiHouwang]

No plugins. I have scene opened usually, but it lags even without open scene just the same.
I'm trying to make refined case but can't find reason yet, it's 1200 lines long and not lagging. Seems more like a large number of cross-references rather thatn script length.

[MeiHouwang]

Is there a way to profile editor btw? Or should I compile Godot from the source for this?

[MeiHouwang]

This issue is still bugging me very much. It is slowing development process and turns off from engine that I like much otherwise. Isn't there anything going to be done?
I believe this will be a real issue for anyone working on project large enough.

[Ranoller]

I notice this too in one of my autoload scripts, with a lot of signals and scene preloads. This happend in a script with 600 lines. But i have one script extending kinematic that have more than 1000 lines and this script doesn't lag. I think that is something releated with autoloads and autocomplete. Trying to assign new packedscene to a variable in preprocesor part is imposible. It freeze, window says that godot not responding and 6 o 7 seconds later editor unfreezes and cursor goes to the selected line. This only happend with the most weigthed autoload. Rest of scripts have good response.

[Zylann]

Any example project around?

[Ranoller]

Ok, It will take a bit. I start now.

[Ranoller]

I can confirm that the problem is preloading scenes. I delete all the preloading in this singleton and not lag, I´m adding this preloads to other singleton and start freeze. I will try to replicate in empty project.

[Ranoller]

....very difficult to emulate in a empty project. I think that the issue needs the combination of a lot of factors that only are in an avanced project (complex scenes, a lot of scripts, lot of images, sounds, animations, actions, lot of data, etc...). Putting random things i´m not accomplishing the behaviour.

Maybe someone can try to do an array of "scene preloads" in any of their autoloads of any of their avanced projects and see if text editor freezes. I will continue investigate this, but i suspect that next time i will see this problem will be in middle travel of next project...

[Ranoller]

More test:

Problem seems to be memory use.... I can´t replicate in new project because I´m not forcing godot editor to use up to 1GB of memory. Apparently this is the limit (at least in Win7 64 bit). I clarify that I have 16 Gb of RAM. Screen captures:

It seems that godot text editor do some kind of parse to the preloaded packed scenes that results in a huge amount of memory being allocated. If I delete all the "preloads", there is no freezes. But I think that Godot editor can freeze in other ways only by going up to 1GB of memory allocated. I continue testing., if i can replicate in a tiny empty project I put here.

Edit- More test: Seems that size of images-audios is not releated with the memory grow. Size of tscn files seems more releated. Tscn files with 6000 lines but less images open slowly and bloat more memory that tscn files with few lines but tons of images. So exported variables- modified transforms , etc... are worst to the "parse?".

[MeiHouwang]

I was really hoping this would get into 2.1.5
But it is still there, it is still lagging, it is still obstructing development. I don't think that I'll ever pick up Godot for any large-scale project again :(

[Ranoller]

There are some workarounds... first of all is important not have any "integrated" resource, all resources should be only referenced, no images inside TSCN files, not ultra-sized tscn files in general. In 3.1 problem seems to be less, but i haven´t jet a complex project like i has in 2.1... Other things that can be done are trying to not repeat node structures inside a scene: If you have 200 copied-pasted sprites or particles in the scenes you should have 1 packed-scene and later instantiated that 200 times... in this way the text file of the packed scene is 200 times small and reading time is 200 time less. There are other "good practices" that are "god practices" in godot to finish big project (Ej: every exported variable in editor is one line in the tscn file, it´s not sense to have 200 exported variables in a node only for "in case I ever use them"...) I don´t think that godot is worst that unity or unreal or game maker in developing, you would have the same problems in the others apis. Godot is best in edit-developing... maybe is worst in exporting-projects (¿consoles?, ¿stutter in some specific hardware configurations?), integrating APIS (steam, google, etc...). But men, in developing, the "doing the game" part, godot beats all of this programs by large, the more small, selfcontainted, portable, integrated "game maker program" in the market.... (¿market?... it´s free!!!!), and the fastest developed. So, if you think that you need a lot of exporting options for 3000 platforms and 2000 graphics apis... well, probably unity is the choice, but if the problem is the lag in 2.15 at the 1gb of memory in the editor... well, solution is not have 1gb allocated inside project editor, and you have in this post the solution...

[Ranoller]

But you are right in that "preload" memory allocate should be fixed... i will open an issue right now.

[akien-mga]

Is this still valid in the current master branch?

[Ranoller]

yes

[fracteed]

@Ranoller thanks for the heads up on this issue. I have been battling this for a while on 2 large scripts in my game (about 1000 lines each). They also have about 20 preloads of packed scenes. It seems that a combination of large scripts and preloads is what really brings the script editor to a crawl.

Now that I know what the issue is, I have been commenting out all my preloads while working on the script and then uncommenting before running. Not a great workflow, but better than script editor lag hell :)

[Calinou]

@fracteed Does the lag go away if you use a ResourcePreloader node to preload resources (then fetch them in the script from the node)?

[Ranoller]

Large scripts is not the problem at all:

Problem is preload.
It´s true that a lot of scripts and some of them large seems to broke in some ways autocomplete, but not contribute to lag.

[fracteed]

@Calinou Thanks for the suggestion. I had never even tried that node before, so I had a quick look at it. I don't think I really want to play around with that right now, as this is a very mature project and I have to be careful to not break anything. It probably would help, so I will keep it in mind for the future.

@Ranoller ok good to know. And wow that is a long gdscript!

[Ranoller]

@Ranoller ok good to know. And wow that is a long gdscript!

Yes, I do very long scripts ... I feel bad about it, I´m trying to correct that. :( :( :(

[Calinou]

Slow rendering should be resolved in Godot 3.2.2 thanks to the addition of batching when using the GLES2 renderer. This may also be added to the GLES3 renderer in a future version.

As for the startup time issue, this is being tracked in #39841.

[Ranoller]

Well, i don´t think rendering was the problem, is "preload"... If this helps, in my current project (very big, a lot of scripts) my workaround is this:

1. If i don´t need preload in startup: I´m using load
2. If i need to preload in game startup: I have a singleton with dictionaries with the names of the resources, I reference that singleton in the other scripts. This dictionaries has "loads" in _init(). It´s the same that preload, it runs before the first scene, but doesn´t overload the editor. It works, and the code is clean, with not "preload" duplicates, if you need one resource, you need to reference the singleton... plus, if you need to change, overwrite, move that resource, only have to do in 1 place, if you delete the dictionarie key, the resource is freed... I know that singleton seems to be an "anti-patern" but i think that in this case is the correct way to go.

[Calinou]

Closing in favor of #39841 and #42688.