improve CODE_BLOCK_PATTERN for a more robust code match

[DearVa]

Why are these changes needed?

In the output of agent_human_feedback.ipynb, I notice that agent's code was not extracted correctly at line 69.

exitcode: 1 (execution failed)
Code output: 
  File "", line 2
    Step 2: Define the equations
         ^
SyntaxError: invalid syntax

That's because there is a space character before python.

Which means ``` python cannot be parsed.

What's more, redundant spaces after language name also cannot be parsed, ```python for example.

Related issue number

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[afourney]

Thank you for documenting your regular expression.

There are a few relevant issues worth linking. Notably, #414, and #399

I believe there are also cases where \r\n is output by the model and needs to be accommodated.

[DearVa]

@microsoft-github-policy-service agree

[DearVa]

Thank you for documenting your regular expression.

There are a few relevant issues worth linking. Notably, #414, and #399

I believe there are also cases where \r\n is output by the model and needs to be accommodated.

Sorry about my late reply. I've added a \r? to my regex to handle with the \r\n case.
By the way, I'm not sure if check 1 was necessary, so I left it unchecked.

[afourney]

Thanks. I will look into this ASAP.

Ideally we should be more flexible, and should adopt the code block definition specified in markdown here: https://spec.commonmark.org/0.30/#fenced-code-blocks

I'll look into this degree of compatibility tomorrow

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (143e49c) 29.81% compared to head (6ea84a5) 47.50%.

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #571       +/-   ##
===========================================
+ Coverage   29.81%   47.50%   +17.69%     
===========================================
  Files          27       27               
  Lines        3448     3448               
  Branches      780      822       +42     
===========================================
+ Hits         1028     1638      +610     
+ Misses       2346     1649      -697     
- Partials       74      161       +87     

Flag	Coverage Δ
unittests	47.27% <100.00%> (+17.51%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[afourney]

@victordibia @sonichi Please have a look at this PR. It has been sitting for a while, and I think it's good to go.