GenerationSession crashes if `stream` keyword argument is not `None`

[cody-moveworks]

TensorRT-LLM/tensorrt_llm/runtime/generation.py

Lines 311 to 313 in 11e1450

	if stream is None:
	self.stream = torch.cuda.Stream(self.device)
	torch.cuda.set_stream(self.stream)

If the stream keyword argument is specified when creating the GenerationSession, self.stream will not be initialized and the code will crash on L313.

[jdemouth-nvidia]

Can you share a way to reproduce the issue, please?

[cody-moveworks]

@jdemouth-nvidia Just create a GenerationSession and pass anything that is not None as the value for stream keyword argument. It will result in errorAttributeError: 'GenerationSession' object has no attribute 'stream'

[juney-nvidia]

@cody-moveworks we already have the fix for this bug internally, and it will get pushed to the main branch later.

June