Extend pytest.approx to handle strings.

[Drew-Ack]

This issue is a developer and community discussion on how strings should be handled in pytest.approx()

Two of the easiest ways to do text matching in pytest are:

1. Regex Matching
2. assert "abc" in "abcdef"

I enjoy the benefits that using assert gives me because of its strength in pytest, but It's not as flexible as Regex could possibly be.

A handy use case I could see is checking to see if an expected string is close to a "template"

psudo-code###

template = "You have saved {%} record"
returned_string = "You have saved 10 records"
approx(returned_string, template) -> true

perhaps there is a bounds of some sort

template = "You have saved {%} record"
returned_string = "You have saved 7 records"
bounds = [0, 10]
approx(returned_string, template, bounds) -> true

bounds = [0,5]
approx(returned_string, template, bounds) -> false

It seems that a template/bounds combo seems to be a good way to extend string comparison beyond the substring in string model.

I am very eager to think about other ideas or use cases people have 😄

[pytestbot]

GitMate.io thinks possibly related issues are #3085 (Remove support for strings to pytest.main), #719 (pytest.mark.parametrize string-based parameter list doesn't handle single element tuples), #109 (Unable to extend reports when using pytest-xdist), #1411 (Pytest stops), and #544 (pytest 2.6 remove some strings in pathname).

[RonnyPfannschmidt]

i propose using a new name for comp arables that support string matching, as its strictly a different topic

[Drew-Ack]

If I understand correctly, we make a new class that is for comparing objects. Leaving approx alone so it can focus on numbers.

Should this new comparable class focus strictly on strings or be constructed in such a way that it can expand to other types?

[RonnyPfannschmidt]

this should be a plugin as a starting point as its not clear how to name things and how it should work

[nicoddemus]

I agree on using a different name than approx for this, besides the potential for confusion I see the "text matching" class using different configuration options on how to do the matching.

One pattern that I've used in the past is something like this:

assert text == pytest.text_diff(expected, diff_lines=[25, 26, 78])

Where diff_lines would specify lines where the texts could differ: this is common for generated text which includes check sums or date of generation.

OTOH I've not used the approach anymore because it is hard to maintain the text along the code (specially large texts) so lately I have been using an approach similar to pytest-regtest.

[RonnyPfannschmidt]

again - this should start outside of pytest, and have much more room for experimentation

[Drew-Ack]

Im fine with starting it as a plugin and going from there.
I believe this would be a good place continue discussing functionality of said plugin

[marcelotrevisani]

huumm... interesting, do you know if someone already started to create this plugin?

[RonnyPfannschmidt]

At least I am unaware

[marcelotrevisani]

I think I will try to develop a plugin like that. It does not seem to be that difficult, let's see
:p

[nicoddemus]

@marcelotrevisani cool, let me know if you have any questions.

Also please update this issue if you end up publishing the plugin. 👍

[nicoddemus]

I think we have a consensus that this should at least start as a plugin. Can close this?