Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pytest_generate_tests based on params and marks from previous parametrize #13233

Open
Anton3 opened this issue Feb 17, 2025 · 6 comments · May be fixed by #13242
Open

pytest_generate_tests based on params and marks from previous parametrize #13233

Anton3 opened this issue Feb 17, 2025 · 6 comments · May be fixed by #13242

Comments

@Anton3
Copy link

Anton3 commented Feb 17, 2025

Issue

In userver, we add indirect params to all tests based on marks using pytest_generate_tests.
(See the full story in #13217.)

The issue arises when we want to apply our implicit param only to some of the test items spawned from an already parametrized test:

@pytest.mark.parametrize(
    "foo, bar",
    [
        ("a", "x"),
        pytest.param("b", "y", marks=pytest.mark.my_mark("hmm")),
        pytest.param("c", "z", marks=pytest.mark.my_mark("what")),
    ])
def my_test(foo, bar, my_fixture, ...):
    # ...


def pytest_parametrize_tests(metafunc):
    # metafunc represents a test function before the split into items.
    # How do we set indirect param "my_fixture" based on marks
    # from one of the cases?

The issue has been previously explored: #4050

Proposed syntax

What I want to discuss here is an implementation of that idea, starting with a possible syntax.

# pytest
@dataclasses.dataclass(frozen=True)
class ProtoItem(pytest.Node):
    # essentially CallSpec2 with a rich API from Node
    params: dict[str, object]
    own_markers: list[Mark]

# usage
def pytest_parametrize_tests(metafunc):
    def param_generator(proto_item: pytest.ProtoItem) -> ParameterSet | Sequence[object] | object:
        if (mark := proto_item.get_closest_marker("my_mark")) is not None:
            return (compute_param_value_from_mark(mark),)
        else:
            return (get_some_fallback_value(),)

    metafunc.parametrize(("my_fixture",), param_generator)
@RonnyPfannschmidt
Copy link
Member

The idea is something under discussion since 2014 as covariant parameters

It's necessary for any parameterisation where you want/need to choose parameter sets dependent on others

It's also a prerequisite for correct lazy fixtures as currently it's not possible to use a parameterized fixture via lazy fixtures

@Anton3
Copy link
Author

Anton3 commented Feb 17, 2025 •
edited
Loading

@RonnyPfannschmidt what about the proposed syntax? I need multi-parametrize that is not just a Cartesian product, and this would be enough for my use cases. Also looks unintrusive enough in terms of API.

@RonnyPfannschmidt
Copy link
Member

The meaning of the proposed object is very unclear to me, and I'd prefer to be able to work in terms of callspec

So more like a conditional parameterize that chooses based on individual callspecs instead of expanding all calls as product

@Anton3
Copy link
Author

Anton3 commented Feb 19, 2025

@RonnyPfannschmidt I've mostly implemented the feature, but now there's the issue that I want to use params and marks from pytest.mark.parametrize in my pytest_generate_tests. But I can't get my pytest_generate_tests to be executed after pytest.mark.parametrize, pytest.mark.parametrize always seems to be applied last. Any ideas how to change that?

@RonnyPfannschmidt
Copy link
Member

Either a trylast hook or a hookwrapper

@Anton3 Anton3 linked a pull request Feb 20, 2025 that will close this issue
Open
@Anton3
Copy link
Author

Anton3 commented Feb 20, 2025

@RonnyPfannschmidt Done, PR is mostly ready for review, but I've left a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow parametrize to depend on previous parametrize Anton3/pytest
2 participants
@RonnyPfannschmidt @Anton3