How would you structure a pytest project with the following characteristics ?

[david-fliguer-ws]

Hi, we would like to dome some refactoring to our automation project.

We use python and pytest.

We would like to have a .py file for each test, like this

test_one.py
test_two.py

Each test should have its own setup and teardown.

Additionally I need to execute some actions only once before all the test run and those actions might define variables that I will need on the setup, run and teardown of the tests (This I was thinking to do on a separate conftest.py with a fixture with scope="session" but I'm not sure of how can then I use variables defined there in my setup, run and teardown of the tests)

Also we would like to be able to run in parallel using pytest-xdist

How would you recommend to organize a project with those characteristic (Regarding hierarchy, fixtures, scopes, etc)

If you need any more info I can try to be more specific, but I am open to ideas

Thanks !!!

[nicoddemus]

Hi @david-fliguer-ws,

Each test should have its own setup and teardown.

You can have a fixture in each test file:

import pytest

@pytest.fixture
def setup():
    # do some setup
    yield
    # do some teardown

def test(setup):
    ...

This I was thinking to do on a separate conftest.py with a fixture with scope="session" but I'm not sure of how can then I use variables defined there in my setup, run and teardown of the tests

Your have a session-scoped fixture in your conftest.py file that returns an object with your variables:

@pytest.fixture(scope="session")
def some_setup():
    return {"x": 10}

Then tests can access the object provided by the fixture:

def test(some_setup):
    assert some_setup["x"] == 10

Of course I'm using a dict here just for illustration purposes, it could be any object that makes sense in your application.

How would you recommend to organize a project with those characteristic

I think putting the tests in a tests directory in the root of the repository would suit you well.

Hope this helps and gives you some ideas to get started.

[david-fliguer-ws]

Hi, thanks for the fast answer

That seems similar to what I had in mind, but in order to use the variables that are defined on the session scoped fixture also in the setup and teardown, should I call to the fixture also on it ?

Something like this ?

import pytest
@pytest.fixture
def setup(some_setup):
    # do some setup
    print("Here I have access to " + some_setup['x'])
yield
    # do some teardown
    print("Here I have access to " + some_setup['x'])

def test_one(some_setup,setup):
    print("Here I have access to " + some_setup['x'])

And what if the setup and teardown of the test I would like to do them more in traditional xunit style ?

Is it possible to mix the two styles (I tried but I couldn't manage but if it is a bad idea then maybe just stick with something like this)

[nicoddemus]

should I call to the fixture also on it ?

Yep! 😁

Note that if your tests don't use the returned object from the fixture, you can use autouse=True to avoid having to name them in the test functions.

And what if the setup and teardown of the test I would like to do them more in traditional xunit style ? Is it possible to mix the two styles (I tried but I couldn't manage but if it is a bad idea then maybe just stick with something like this)

Currently due to #517 th xunit style setup/teardown won't play nicely with your scoped session fixture, so I would strongly recommend to stick to fixtures to avoid pitfalls. 😉

[david-fliguer-ws]

That sounds good, I think the challenge that remains is the following

I would like to run my tests in parallel using pytest-xdist.

I understand that there is an "issue" that when using pytest-xdist then the scope = "session" it is per thread instead of per all the threads, that means that in order to run code only once before all the tests I need to do some workaround on the fixture with scope session (Like checking the number of worker and do the code only for one of the workers) but does that mean that the rest of the workers won't get in return the object of the fixture and therefore I won't have it on the tests that run under that worker ?

Let me know if what I say it is clear or if maybe another workaround (Or even solution) is recommended to deal with scope = "session" and pytest-xdist

[nicoddemus]

I understand that there is an "issue" that when using pytest-xdist then the scope = "session" it is per thread instead of per all the threads, that means that in order to run code only once before all the tests I need to do some workaround on the fixture with scope session (Like checking the number of worker and do the code only for one of the workers) but does that mean that the rest of the workers won't get in return the object of the fixture and therefore I won't have it on the tests that run under that worker ?

Right, except that each worker runs in its own process, not in a separate thread.

What is the recommended approach really depends on what your session-scoped fixture does: if it initializes and returns an in-memory object that tests need to access, then it actually makes sense to let each worker create its own fixture. On the other hand, if it sets up an external resource (like a database) which only needs to be created once, then the approach of only setting up the resource for one of the workers and let the others wait using some synchronization mechanism (like a file-lock) might work for you (see pytest-dev/pytest-xdist#271 for a discussion on this).

[david-fliguer-ws]

Well the truth is in that session scoped fixture I need to do kind of a mix of both of them

Basically I need to do the following:

1. Delete a file from the PC and download a file from the internet

I need that to be "locked" in an atomic action that is done only once per all the processes because if one processes deletes and start to download and then another one goes and does same code It will try to delete the file that is being downloaded and start a new download and that is a problem.

2. Load a json file and set its values on a dictionary that I will use on the setup, run and teardown of the tests (Of all the tests that run on all the workers)

That is only reading so doesn't seem to be an issue from a parallel point of view but again, seems like a waste of time to make N (Number of workers) I/O operations that will load same values, ideally I would like this to happen once and be able to use the dictionary that I return in all the tests (No matter in what worker they run)

Hope this is clear, if you need more clarification let me know

Thanks a lot for the help !!!

[david-fliguer-ws]

I've been just reading a bit and I think I see two different ways to workaround about this

1. Separate the "setup" process in a different script or in some way that it doesn't run in the same run of the test

2. Deal with some kind of lock mechanism

I'd prefer not to do option 1 but I don't have too much experience with dealing with parallel and concurrency (Just some theoretical background and little practice from university) and definitively not in Python

Will a file lock be an option in this case ? Actually there is not a "shared resource" but rather some actions that I need to be done only once...

I am kind of lost with this but I would really like to run my tests in parallel since the test suite is starting to grow

[RonnyPfannschmidt]

if the library you use to download supports setting up file caches, perhaps just set up a cache folder in the pytest cache directory

[david-fliguer-ws]

if the library you use to download supports setting up file caches, perhaps just set up a cache folder in the pytest cache directory

I'm sorry but I don't understand what do you mean with this, each run I need to download a new file, I don't want to "cache" the file

[RonnyPfannschmidt]

@david-fliguer-ws im sorry, i misunderstood the intent then

[nicoddemus]

Will a file lock be an option in this case ? Actually there is not a "shared resource" but rather some actions that I need to be done only once...

I think a file lock might be a good solution then. Never used this, but the example shown in py-filelock seems like it could be what you need.