refactor(streaming): introduce END_OF_STREAM sentinel and update handling

[Pouyanpi]

This PR resolves #1184 by introducing END_OF_STREAM sentinel and update streaming handling.

• Replaced inconsistent use of None and "" for stream termination
  in StreamingHandler with a dedicated END_OF_STREAM sentinel object.

• Modified push_chunk to convert None to END_OF_STREAM.

• Updated __anext__ to raise StopAsyncIteration only for END_OF_STREAM
  and to return empty strings or dicts with empty/None text as data.

• Adjusted _process to correctly handle END_OF_STREAM for buffering
  and queueing logic.

• Updated on_llm_end to use END_OF_STREAM.

• Revised tests in test_streaming_handler.py to reflect these changes,
  including how empty first tokens are handled and how __anext__ behaves
  with various inputs.

  requires:

• fix(tests): ensure proper asyncio task cleanup in test_streaming_handler.py #1182

• test(streaming): add extensive tests for StreamingHandler to enhance coverage #1183

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 98.27586% with 1 line in your changes missing coverage. Please review.

Project coverage is 68.62%. Comparing base (ca02cb6) to head (151f6b5).

Files with missing lines	Patch %	Lines
nemoguardrails/streaming.py	98.18%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1185      +/-   ##
===========================================
+ Coverage    68.57%   68.62%   +0.05%     
===========================================
  Files          161      161              
  Lines        15943    15966      +23     
===========================================
+ Hits         10933    10957      +24     
+ Misses        5010     5009       -1     

Flag	Coverage Δ
python	68.62% <98.27%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
nemoguardrails/rails/llm/llmrails.py	87.12% <100.00%> (ø)
nemoguardrails/streaming.py	98.59% <98.18%> (+0.69%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Pouyanpi]

requires:

• fix(tests): ensure proper asyncio task cleanup in test_streaming_handler.py #1182
• test(streaming): add extensive tests for StreamingHandler to enhance coverage #1183

[Pouyanpi]

Codecov Report

Attention: Patch coverage is 98.27586% with 1 line in your changes missing coverage. Please review.

Project coverage is 68.44%. Comparing base (b65cf0e) to head (fb86618).

Files with missing lines Patch % Lines
nemoguardrails/streaming.py 98.18% 1 Missing ⚠️
Additional details and impacted files
🚀 New features to boost your workflow:

test_anext_with_other_runtime_error covers that, we should get 100% for once!

[andompesta]

Adding include_generation_metadata option is great, but it generates problems with output rails buffer:

• the generate_chunk_str functions is not able to extract the text from the dictionary. Note that, without include_generation_metadata it works because chunks are just strings not dictionary
• is it possible to not put in the buffer chunks containing END_OF_STREAM object and the "" tokens ? this is because output-rails will be triggered with 2 or 3 empty tokens added at the end of the bot message. This might cause the message to be blocked involuntary

[tgasser-nv]

Looks good! I like using an object to mark the end of stream rather than other values like None or "". Looks a little weird with the is comparisons in the logic rather than == but we need to check memory locations not values. Thanks!

[Pouyanpi]

Looks good! I like using an object to mark the end of stream rather than other values like None or "". Looks a little weird with the is comparisons in the logic rather than == but we need to check memory locations not values. Thanks!
I prefer is for ref: https://python-patterns.guide/python/sentinel-object/#sentinel-objects