[Fuzzing X64] [EgorBo] Remove more unsafe code from IPAddress parsing

[MihuBot]

Job completed in 1 hour 34 minutes (remote runner delay: 1 minute 7 seconds).
dotnet/runtime#121225
Using arguments: fuzz IPAddress

Ran the fuzzer(s) successfully.

Artifacts:

• IPAddressFuzzer-inputs.zip (182 KB)

[MihuBot]

@MihaZupan