Support Poetry v2 dependency format

[rarkins]

Discussed in #33455

^{Originally posted by ryancausey January 7, 2025}

Tell us more.

Poetry v2 has updated their pyproject.toml specification to support the "project" section in the pyproject.toml specification.

This means that there are now more potential locations where dependencies can be defined within a pyproject.toml file:

1. project.dependencies
2. project.optional-dependencies
3. tool.poetry.dependencies
4. tool.poetry.group.< name >.dependencies

The other rub is that the project.dependencies list doesn't support all of the dependency specifiers that one of the dependency locations under too.poetry does: https://python-poetry.org/docs/dependency-specification/

It would be nice to have renovate support this new dependency location and formats under the project heading.

Minimal reproduction: https://github.com/ryancausey/poetry-v2-support-reproduction

[rarkins]

the project section is defined by PEP 621. So shouldn't it be already supported? docs.renovatebot.com/modules/manager/pep621

Renovate has separate "managers" for poetry and pep621. If Poetry v2's behavior is now fully aligned with pep621 then we can consider if it's possible to merge these managers and remove the poetry-specific one. Each different lock file variant of pep621 needs separate work, i.e. when Renovate docs say it supports pep621 it doesn't mean "and every possible implementation of lock file now and in the future" - work needs to be done for each different lock file.

[ryancausey]

The poetry docs still indicate there are scenarios where the dependencies under the tool.poetry section should be used, so I don't believe its behavior is fully aligned with pep621 in all cases. https://python-poetry.org/docs/dependency-specification/

In either case, the poetry.lock file is not listed as a supported lock file format under the pep621 manager.

[ryancausey]

It should also be noted, you can ignore the new pep621 syntax and continue to place all dependencies in the tool.poetry section by adding "dependencies" to the project.dynamic list.

[ryancausey]

Where would I start looking in the code base if I wanted to contribute a feature to support this? This issue is currently gumming up all of my renovate bot dependency updates.

[edgarrmondragon]

Where would I start looking in the code base if I wanted to contribute a feature to support this? This issue is currently gumming up all of my renovate bot dependency updates.

I would imagine https://github.com/renovatebot/renovate/tree/main/lib/modules/manager/poetry, but project.dependencies and project.optional-dependencies will probably have to follow python versioning.

[rarkins]

@ryancausey I've created 3 sub-issues above within this issue. It's my first time using sub-issues on GitHub and I assume everyone can see them. Is this a good starting point for you?

BTW if anyone who's commenting or +1'ing this is a paying Mend customer, please contact your account or customer success manager to alert them to your interest in this feature. For everyone else, please consider becoming one, so we have more resources in future to work on things like this, and your favorite issues can get priority.

[github-actions]

Hi there,

Please do not post comments like: "Any update on this?" or "When will this be fixed?".

If you're asking because you want to know if there's been some progress, then please know that we work in the open in this repo. If there was an update to this Discussion or Issue, you would see it already. So please don't ask "just in case".

If you're asking because you want to get our attention, then please don't do that either. It wastes maintainer time and pollutes the repo. You're welcome to fix issues yourself, hire someone to help you, or wait for someone else to do the work. You're not welcome to post the equivalent of "Hey, someone else should really do something about this".

If you are a paying Mend.io customer, contact your support or customer success representative, and tell them this issue is important to you.

We expect everyone to follow these rules, no matter if you use the free tiers, or are a paying Mend customer. Please read our Code of Conduct, how we prioritize work to learn more about how we prioritize what to work on.

Thanks, the Renovate team