feat(bindings/python): Enhance Reader and Writer

[chitralverma]

Which issue does this PR close?

Closes #5943

What changes are included in this PR?

• Support all read and write options on sync/ async operator open(...)
• Support all read options on sync/ async operator read(...)
• Support all write options on sync/ async operator write(...)
• Add missing capabilities to py bindings
• Update documentation
• Update all capabilities
• Minor fixes linting, typing and typos

Are there any user-facing changes?

No breaking changes, but users can use the new features.

[chitralverma]

@Xuanwo @messense @Zheaoli please have a look whenever you get a chance

the 2 failing tests, seem to show a different service specific bug. the if_not_exist capability is there but the behaviour is not correct.

[chitralverma]

@Xuanwo @messense @Zheaoli any suggestions you have for this PR?
I think addition of reader/ writer options to AsyncOperator and the lazy streaming copy of data from 1 file to another can be quite useful features.

[chitralverma]

@messense @Xuanwo added some benchmarks and updated the scope of this PR.

See Comment

Please let me know if you have some review comments

[chitralverma]

@Xuanwo Since #6215 and #6216 have been merged, I think most of the options for reader_with/ writer_with are already covered there. Some one also suggested looking into write_from at a later time. so do you recommend closing this PR or is some here still valid?

[Xuanwo]

@Xuanwo Since #6215 and #6216 have been merged, I think most of the options for reader_with/ writer_with are already covered there. Some one also suggested looking into write_from at a later time. so do you recommend closing this PR or is some here still valid?

Hi, I think the additional options for WriteOptions and ReadOptions are still valid. We can go ahead and merge them. However, there are still some concerns regarding the new API related to write_from; perhaps we can leave them to another PR.

[erickguan]

Great feature and nice chore work!

I added a few suggestions.

[chitralverma]

added a temp fix for the breaking tests.
See discussion #6235

[erickguan]

Minor comments on test exceptions.

LGTM

[erickguan]

minor: blank around the keyword as.

Perhaps you can import ConditionNotMatch which aligns with other code. Then:

Suggested change

	with pytest.raises(Exception)as excinfo:
	async with await async_operator.open(filename, "wb", if_not_exists=True) as w:
	w.write(content)
	assert "ConditionNotMatch" in str(excinfo.value)
	with pytest.raises(ConditionNotMatch):
	async with await async_operator.open(filename, "wb", if_not_exists=True) as w:
	w.write(content)

[Xuanwo]

related discussion: #6235

[chitralverma]

@erickguan this is what i did before, but there was a different issue.
I'll address that in a separate PR.

[erickguan]

@chitralverma Thank you so much for bringing up the issue, investigation, and fix! You can add a TODO or a link to the GitHub issue. I will not make this comment in that case :)

[erickguan]

Suggested change

	with pytest.raises(Exception)as excinfo:
	with operator.open(filename, "wb", if_not_exists=True) as w:
	w.write(content)
	assert "ConditionNotMatch" in str(excinfo.value)
	with pytest.raises(ConditionNotMatch):
	with operator.open(filename, "wb", if_not_exists=True) as w:
	w.write(content)

[chitralverma]

@erickguan this is what i did before, but there was a different issue.
I'll address that in a separate PR.

related discussion: #6235

[chitralverma]

@Xuanwo @erickguan all your suggestions have been included now, and I've updated the description for this PR.
Please let me know if there's anything else pending before merging this.

[Xuanwo]

Thank you @chitralverma for working on this. Although we haven't addressed #6235 yet, I think it's fine to use a workaround in the tests for now.

Will merge after CI passed.