after an investigation, this is my plan to add native torch.Generator on pytorch/xla. Basically, follow the example of CUDAGeneratorImpl.cpp

Create a Custom XLA Generator Implementation: create a new C++ class that inherits from c10::GeneratorImpl. This class will be responsible for managing the state of the XLA RNG. This class should inherit from c10::GeneratorImpl and implement its virtual methods, including:

clone(): To clone the generator state.
current_seed(): To get the current seed.
manual_seed(uint64_t seed): To set the seed manually.
get_state() and set_state(const c10::TensorImpl& new_state): To get and set the generator's state.

Register the XLA Generator: make the custom generator known to the PyTorch dispatcher, register it. This will associate the custom generator with the XLA device type.
Integrate the Generator into XLA's Random Operations: Modify Random Functions like randn, bernoulli, normal to that they use the Generator to manage their random state and take an optional generator parameter for custom generator.
Expose to Python: Add the necessary Python bindings so users can create and use torch.Generator(device='xla').

@tengyifei can you nominate a person to review the plan above to see if it is feasible and potentially be the code reviewer for this issue? thanks!

Support torch.Generator on PyTorch/XLA #9159

Description