Using fixtures in pytest.mark.parametrize

[pytestbot]

Originally reported by: Florian Rathgeber (BitBucket: frathgeber, GitHub: frathgeber)

---

I often have a use case like the following contrived example:

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.mark.parametrize('arg', [a, b])
def test_foo(arg):
    assert len(arg) == 1

This doesn't currently do what's intended i.e. arg inside the test function is not the fixture value but the fixture function.

I can work around it by introducing a "meta fixture", but that's rather ugly:

@pytest.fixture(params=['a', 'b'])
def arg(request, a, b):
    return {'a': a, 'b': b}[request.param]

def test_foo(arg):
    assert len(arg) == 1

It would be convenient if a syntax like in the first case was supported.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/349

[pytestbot]

Original comment by Matthias Geier (BitBucket: geier, GitHub: geier):

---

This would be a great feature!

I found another (I don't know if more or less ugly) work-around:

#!python

@pytest.mark.parametrize('arg', ['a', 'b'])
def test_foo(arg, request):
    val = request.getfuncargvalue(arg)
    assert len(val) == 1

This doesn't work, however, with parametrized fixtures.

BTW, it would also be great if fixtures were supported in the params argument of pytest.fixture.

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

Tentatively assigning this to me as I think I might be able to come up with a reasonable patch. It'll probably take me a long while though so don't let that discourage anyone else from working on this, assigning it more as a way of not forgetting about it.

[pytestbot]

Original comment by Praveen Shirali (BitBucket: praveenshirali, GitHub: praveenshirali):

---

The quoted examples work because functions a and b are part of the same module as test_foo, and within the scope of the example, the parametrization should work even if @pytest.fixture decorator isn't present around functions a and b. They are getting used as regular python functions and not as pytest fixtures. Note that fixtures can also be defined in external modules like conftest.py.

Another alternative to the above example is to directly call these functions in the list.

#!python

@pytest.mark.parametrize('arg', [a(), b()])
def test_foo(arg):
    assert len(arg) == 1

[pytestbot]

Original comment by BitBucket: dpwrussell, GitHub: dpwrussell:

---

This would be an awesome feature.

@praveenshirali I don't think your alternative is used as a fixture, it just calls the fixture function. So it would be run repeatedly. You would also have to specify the arguments to the fixture if there were any which could begin the cycle over again if they are also fixtures.

[mgeier]

Here's a related stackoverflow question: http://stackoverflow.com/questions/24340681/how-to-concatenate-several-parametrized-fixtures-into-a-new-fixture-in-py-test

[jlmenut]

Yes, I would like very much to have this feature also. Maybe a line in the doc explaining it's not possible for the moment would be useful also.

[kevincox]

It would also be killer if this supported parameterized fixtures generating the product of the fixtures. Although this might be a little much.

@pytest.fixture(params=["1", " ", 1, True, [None], {1:2}])
def truthy(request):
    return request.param

@pytest.fixture(params=[False, None, "", 0, [], {}])
def falsey(request):
    return request.param

@pytest.mark.parameterize("val,res", [
    (truthy, True),
    (falsey, False),
])
def test_bool(val, res)
    assert bool(val) is res

[SUNx2YCH]

+1 for this feature.
BTW, combining Florian Rathgeber and Matthias Geier solutions we can get a bit nicer "meta fixture":

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.fixture(params=['a', 'b'])
def arg(request):
    return request.getfuncargvalue(request.param)

def test_foo(arg):
    assert len(arg) == 1

[rabbbit]

+1 on this.

I'm currently writing tests that look like:

@pytest.fixture
def my_fixture():
      return 'something'

@pytest.fixture
def another_fixture(my_fixture):
      return {'my_key': my_fixture}

def yet_another_fixture():
     return {'my_key': None}

@pytest.mark.parametrize('arg1, arg2', [
    (5, another_fixture(my_fixture())),
    (5, yet_another_fixture()),
)
def my_test(arg1, arg2):
    assert function_under_test(arg2) == arg1

and that's rather ugly.

[RonnyPfannschmidt]

@rabbbit your example is structurally wrong and runs fixture code at test importation time

[rabbbit]

@RonnyPfannschmidt I know - and that's why I'd like to be able to use fixtures in parametrize? And that would be awesome.

My example is wrong, but it follows the guideline of "always use fixtures". Otherwise we'd end up with fixtures in normal tests, and workarounds in parametrized tests.

Or is there a way of achieving this already, outside of dropping parametrize and doing 'if/elses' in the test function?

[RonnyPfannschmidt]

There is a upcoming proposal wrt "merged" fixtures, there is no implementation yet

[nicoddemus]

For reference: #1660

[rabbbit]

ok, I don't understand python.

If the below works:

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

wouldn't the below be simpler? And an exact equivalent?

@pytest.fixture
def my_fixture
    return 1

@pytest.mark.parametrize('fixture', [my_fixture])
def test_me(fixture):
    assert 1 == my_fixture

Am I wrong to think that mark.parametrize could figure out whether an argument is a pytest.fixture or not?

[RonnyPfannschmidt]

atm parametrize cannot figure it, and it shouldnt figure it
there will be a new object to declare a parameter will request a fixture/fixture with parameters

some documentation for that is in the features branch

[rabbbit]

yeah, I read the proposal.

I'm just surprised you're going with pytest.fixture_request(' default_context')it feels very verbose?

after all,

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

could also turn to

@pytest.fixture
def my_fixture
    return 1

def test_me(pytest.fixture_request(my_fixture)):
    assert 1 == my_fixture

but that's not the plan, right?

On 5 July 2016 at 16:17, Ronny Pfannschmidt notifications@github.com
wrote:

atm parametrize cannot figure it, and it shouldnt figure it
there will be a new object to declare a parameter will request a
fixture/fixture with parameters

some documentation for that is in the features branch

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#349 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AARt0kUxVSL1GMeCtU-Kzy3Pg80mW7Ouks5qSmdmgaJpZM4FEMDj
.

[RonnyPfannschmidt]

thats not even valid python syntax

the fixture-request would be used as a parameter value to tell py.test "use the value of a fixture you create later on"

there are already parametzw using examples, and its not overly verbose

[kibernick]

It would be really convenient to have this functionality, perhaps along the lines of "LazyFixture" in pytest-factoryboy

[RonnyPfannschmidt]

@kibernick we already did put a outline of possible implementations into the documentation

we just need time or a person implementing it

[Brachi]

@RonnyPfannschmidt can you link to that part of the documentation you mention? Can't find it.
Edit: nevermind. http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html

[TvoroG]

@RonnyPfannschmidt, can you please check this out https://github.com/TvoroG/pytest-fixture-mark? Need some feedback

[Brachi]

@TvoroG good work.
Currently this seems to be failing. Is it possible to support it as well?

import pytest                                                                  


@pytest.fixture(params=[1, 2, 3])                                              
def one(request):                                                              
    return str(request.param)                                                  


@pytest.fixture                                                                
def two():                                                                     
    return 4                                                                   


@pytest.fixture(params=[                                                       
    pytest.mark.fixture('one'),                                                
    pytest.mark.fixture('two')                                                 
])                                                                             
def some(request):                                                             
    return request.param                                                       


def test_func(some):                                                           
    assert some in {'1', '2', '3', 4}

[TvoroG]

@Brachi, thanks for catching it! It works now, but more nested structures need some dependency sorting to instantiate fixtures in correct order. I'll update the plugin code when i'm done.

[TvoroG]

@Brachi, I fixed it. Let me know if there is more such cases when plugin is failing

[Brachi]

@TvoroG great, thanks for the quick reply. I tested it a little more and here's another contrived example that doesn't work, based on a real use case (where one actually returns an object)

import pytest


@pytest.fixture(params=[1, 2, 3])
def one(request):
    return request.param


@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_str(request):
    return str(request.getfixturevalue('one'))


@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_hex(request):
    return hex(request.getfixturevalue('one'))


def test_as_str(as_str):
    assert as_str in {'1', '2', '3'}


def test_as_hex(as_hex):
    assert as_hex in {'0x1', '0x2', '0x3'}


# fails at setup time, with ValueError: duplicate 'one'
def test_as_hex_vs_as_str(as_str, as_hex):
    assert int(as_hex, 16) == int(as_str)

[nicoddemus]

@TvoroG, pytest-fixture-mark seems very nice! I hope I get the chance to try it soon! 😄

Perhaps discussion specific to it should be moved to https://github.com/TvoroG/pytest-fixture-mark thought? 😉

[waynew]

For another concrete example, I'm currently working on a PR where I am trying to write a single test (suite) to run against several different backends. What I would like to be able to write is something like this:

@pytest.fixture(scope="module", params=get_mysql_versions())
def mysql_image(request):
    image = docker.pull(tag=request.param)
    yield image

@pytest.fixture(scope="module")
def mysql_container(mysql_image):
    # Maybe do some other stuff here
    container = docker.run(image=mysql_image) 
    yield container

@pytest.fixture
def mysql_cache(mysql_container):
    cache = MySqlCache(config=mysql_container.config)
    yield cache


# Do similar setup for other cache images, containers, and cache SUTs here

@pytest.mark.parametrize(
    "cache",
    [mysql_cache, consul_cache, etcd_cache, redis_cache, localfs_cache],
)
def test_cache(cache):
    result = cache.do_something('value')
    assert result == 'expected result'    

I'm currently working around it by using request.getfixturevalue, but that means that I'm gonna need a single fixture for every one of my versions 😞

Unless there's currently a better way to do this.

[RonnyPfannschmidt]

Pytest-lazy-fixtures is a external plugin to do part of that

[smarie]

And pytest-cases works too @waynew :) See #349 (comment) and doc reference https://smarie.github.io/python-pytest-cases/pytest_goodies/#parametrize

EDIT: simply replace @pytest.mark.parametrize with @parametrize and @pytest.fixture with @fixture in your example and it should work, with both parametrize and fixture imported from pytest_cases.

[chrisk314]

Just a quick note to say the pytest-lazy-fixture package mentioned as a workaround here appears to be dead, with many unaddressed open issues and it's not compatible with pytest>=8.0.0. This would be a nice quality of life feature to have natively in pytest.

[smarie]

@chrisk314 You can use pytest-cases as a workaround (https://smarie.github.io/python-pytest-cases), it is alive and kicking :) (and compliant with pytest 8 since version 3.8.3)
See #349 (comment)

Still, I agree with you that it would be better to refactor the internal engine to handle all of this in pytest. This would require massive rework unfortunately... which is why the team did not do it yet.

[MRGuziX]

Any news on that? That would be and great feature... and installing more and more libs to handle those kind of cases it is not the best answer :)

[The-Compiler]

If there were any news, it would be here in the issue. That's kind of the whole point of a public issue tracker.

[thetic]

https://github.com/dev-petrov/pytest-lazy-fixtures

[MRGuziX]

https://github.com/dev-petrov/pytest-lazy-fixtures

That cannot be the "only answer" to the problem...

If there were any news, it would be here in the issue. That's kind of the whole point of a public issue tracker.

TBH if there is an issue that is open for 12+years I feel like there is no hope for it and it will never be implemented due to beeing in some kind of "backlog hell". Last time there was some bump here it was 9 months ago, and having that would resolve huge amount of "edge" cases in testing.
If I am wrong, deep down it would be nice to be..., and there is a chance for introducing that keep it alive. If there is no option for that "because of reasons" it should be closed with proper comment.

[hpk42]

By chance am subscribed to this issue although i am not maintaining pytest for many years now (but using it still :). Not sure how much it's helpful, but here are my 2cents. Parametrizing a test function requires all values flowing into the test function to be known at test-collection time IIRC. Using `request.getfixturevalue()` already kind of breaks this constraint. Runtime-requested fixtures can not easily be parametrized themselves, unless something has changed in the code since i contributed it. Parametrizing a test function with fixtures (in whatever syntax) would probably need to add all parametrizations of those fixtures ... but in a kind of closure across all the different fixtures? Anyway, maybe the reason why this issue hasn't progressed is that it requires some careful thinking and design and equally careful implementation work. Just because a "rough example" is written down, does not imply that it's equally easy to actually implement it.

[RonnyPfannschmidt]

implementation is really tricky actually - external implementation currently break down the moment one of the fixtures we depend on is actually parameterized

parameterize would have to act like a fixpoint function to resolve that - and currently most of the internals don't support that

so right now are unable to internally express whats needed for correct resolution