feat: support poetry v2

[jkowalleck]

poetry v2 just got released: https://github.com/python-poetry/poetry/releases/tag/2.0.0

add support for it and it's new features, if any

• new lock file format?
• any new pytproject.toml declarations
  • support for PEP621 - metadata and dependencies
    Goal: not either/or, but simultaneously the "old" tool.poetry and the "new" project

    Add support for the project section in the pyproject.toml file according to PEP 621
    -- Add support for the project section (PEP 621) python-poetry/poetry#9135
    -- PEP 621: fix poetry source add if there is no tool.poetry section yet python-poetry/poetry#9917

  • investigate what else is new ... TBC ...
• ... TBC ...
• tests
  • since poetry v2 there is a new lock file format in town: 2.1. this means we can keep current test structures and add a new folder for each lockfile thing where needed.
  • some new cases may not be available for older poetry - like the PEP621 in all its variants... -

[⤴ this list will be updated continuously based on comments below, until the initial feature was provided eventually]

[jkowalleck]

Note

in case somebody wants to champion this feature, feel free to let us know and organize yourselves in the comments section 📣

[kwaegel]

I've started using cyclonedx-py with Poetry 2.0.1 in requirements mode, and the main issue I've run into so far is not reading metadata from the PEP 621 [project] section.

Initially I was getting a CRITICAL | CDX > 'name' fatal error, but adding a duplicate of the name field in the [tool.poetry] section fixed that (though now Poetry emits a warning about duplicate fields).

(This may be irrelevant if I switch over to using uv for packaging, though.)

[sam5827]

We recently upgraded to Poetry v2 and changed to using the [project] table in the pyproject.toml, but Cyclone DX doesn't seem to support the dependencies defined in that table and they are not included in the SBOM.xml output.

I can see in poetry.py there's only mention of the [tool.poetry] section, so it appears it will only be considering dependencies defined there and won't consider one's in [project].

If that's right, then I would like to see support for Poetry 2 and the PEP621 [project] standards.
Please let me know if I've misunderstood this though!

(I'd consider attempting to contribute if I had a bit of help as well)

[jkowalleck]

i did not look into all details of poetry2's docs.
could you point me to the docs, where they allow project instead of tool.poetry?

BTW: PEP621 is already implemented: https://github.com/CycloneDX/cyclonedx-python/blob/main/cyclonedx_py/_internal/utils/pep621.py
It is just not applied, since poetry went with tool.poetry, in the past.

https://python-poetry.org/docs/managing-dependencies/

Poetry supports specifying main dependencies in the project.dependencies section of your pyproject.toml according to PEP 621. For legacy reasons and to define additional information that are only used by Poetry the tool.poetry.dependencies sections can be used.

[m-erhardt]

@jkowalleck it's stated in the release notes for poetry 2.0.0

• Add support for the project section in the pyproject.toml file according to PEP 621 (#9135, #9917).
  [...]
• Deprecate several fields in the tool.poetry section in favor of the respective fields in the project section in the pyproject.toml file (#9135).

Since 2.0.0 poetry prints a warning if name, description, etc are specified in the tool.poetrysection of pyproject.toml

[armingerten]

We are also heavily awaiting the support for poetry 2 for cyclonedx-py. However, I have been thinking about the following work-around:

poetry export | cyclonedx-py requirements -

Doesn't this yield the same result as cyclonedx-py poetry (with poetry 2 support)?

[jkowalleck]

poetry export | cyclonedx-py requirements -

Doesn't this yield the same result as cyclonedx-py poetry (with poetry 2 support)?

not at all.
have you tried it?

[jkowalleck]

We are also heavily awaiting the support for poetry 2 for cyclonedx-py

Everyone is awaiting, nobody is contributing - yet.
Feel free to champion this feature, I will be there to assist you.

[armingerten]

Doesn't this yield the same result as cyclonedx-py poetry (with poetry 2 support)?

not at all.

Hmm, what's the difference? 🤔

We are also heavily awaiting the support for poetry 2 for cyclonedx-py

Everyone is awaiting, nobody is contributing - yet. Feel free to champion this feature, I will be there to assist you.

Yeah, I get that. I didn't intend to put pressure on anyone with that. I was just showing my interest in this issue 😉

[jkowalleck]

In the meantime, I suggest looking into cyclonedx-py environment - https://cyclonedx-bom-tool.readthedocs.io/en/latest/usage.html#for-python-virtual-environment
It is probably the most true and complete BOM you could get.

[armingerten]

Thanks for the hint to cyclonedx-py environment!

So

cyclonedx-py environment "$(poetry env info --executable)" --pyproject ./pyproject.toml

seems to be even superior to

cyclonedx-py poetry

I just noticed that specifying the --pyproject parameter would also yield an error (CRITICAL | CDX > 'name') when using the PEP621 style in your pyproject.toml.

[sealedtx]

@jkowalleck @armingerten

I just noticed that specifying the --pyproject parameter would also yield an error

Can we simply update priority, take first [project] if present instead of [tool.poetry] here

cyclonedx-python/cyclonedx_py/_internal/utils/pyproject.py

Lines 33 to 40 in 12cc59b

	def pyproject2component(data: Dict[str, Any], *,
	ctype: 'ComponentType', fpath: str) -> 'Component':
	tool = data.get('tool', {})
	if poetry := tool.get('poetry'):
	return poetry2component(poetry, ctype=ctype)
	if project := data.get('project'):
	return project2component(project, ctype=ctype, fpath=fpath)
	raise ValueError('Unable to build component from pyproject')

[jkowalleck]

Can we simply update priority, take first [project] if present instead of [tool.poetry] here

unfortunately not. please read this very ticket's (updated) description:

any new pytproject.toml declarations

support for PEP621 - metadata and dependencies
Goal: not either/or, but simultaneously the "old" tool.poetry and the "new" project

Add support for the project section in the pyproject.toml file according to PEP 621

[jkowalleck]

started looking into this.

will provide test setups (lockfiles) for poetry v2 for the existing cases, and see how this turns out.
from there on, i might adjust the ticket's description to reflect needed tasks.

Done:

• tests: add setups and snapshorts for poetry v2 #851

feature development may start, now