Fix Gemma RMSNorm weight init

[albertz]

What does this PR do?

Gemma RMSNorm weight is used additively in ...*(1+weight),
thus it should be initialized with zero.

Fixes #40224

Who can review?

@ArthurZucker

[Rocketknight1]

Should this be here instead of in the Gemma modeling files? Leaving it to @ArthurZucker

[albertz]

Should this be here instead of in the Gemma modeling files? Leaving it to @ArthurZucker

I asked exactly about that in #40224 and @ArthurZucker suggested to do it that way.

[ArthurZucker]

Oups! Sorry @albertz ! The best way to do it:

1. you just update the _init_weight of gemma3 and gemma
2. normally this should set module._is_hf_initialized = True
   thus preventing doing it twice

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: gemma

[albertz]

I updated the code for Gemma, i.e. GemmaModel in .../gemma/modular_gemma.py.

I did not update the code in .../gemma/modular_gemma.py as this is automatically generated? How would I update this?

Similarly, I also did not update Gemma2Model, as this derives from GemmaModel, and should already be correct? Or not?

Similarly, I also did not update Gemma3TextModel.

As far as I understand the code, I don't need to set module._is_hf_initialized = True, as the _initialize_weights method already does this?

[albertz]

I just noticed that the changes I proposed have already been done in the meantime?

[albertz]

#40796 (b4ba4e1) by @vasqu.

I guess this PR here can be closed then.

[vasqu]

Sorry about that @albertz, didnt know that there was already a different PR for this :/ got to it due to a different issue I noticed

Thanks nonetheless for the PR 🤗