Schema validation failed due to duplicates

[martin-reich]

Running the following command fails due to duplicates in the dependency for all releases after v11.0.10.
cdxgen . --profile license-compliance --required-only --no-auto-compositions -o sbom_out.cdx.json

The error message when running it in a docker container with the ubuntu:latest image on arm64:

Schema validation failed for secobserve
[
  {
    instancePath: '/dependencies/40/dependsOn',
    schemaPath: '#/properties/dependsOn/uniqueItems',
    keyword: 'uniqueItems',
    params: { i: 31, j: 30 },
    message: 'must NOT have duplicate items (items ## 30 and 31 are identical)',
    schema: true,
    parentSchema: {
      type: 'array',
      uniqueItems: true,
      items: [Object],
      title: 'Depends On',
      description: 'The bom-ref identifiers of the components or services that are dependencies of this dependency object.'
    },
    data: [
      'pkg:npm/@eslint/[email protected]',
      'pkg:npm/@eslint/[email protected]',
      'pkg:npm/@eslint/[email protected]',
      'pkg:npm/@fortawesome/[email protected]',
      'pkg:npm/@fortawesome/[email protected]',
      'pkg:npm/@mui/[email protected]',
      'pkg:npm/@mui/[email protected]',
      'pkg:npm/@textea/[email protected]',
      'pkg:npm/@typescript-eslint/[email protected]',
      'pkg:npm/@typescript-eslint/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:npm/[email protected]',
      'pkg:pypi/secobserve@latest',
      'pkg:pypi/secobserve@latest'
    ]
  }
]

Full SBOM here: sbom_broken.cdx.json

The error can be pinpointed to cdxgen version v11.1.0. The previous version v11.0.10 does not throw a schema validation error.

Steps to reproduce:
Run cdxgen . --profile license-compliance --required-only --no-auto-compositions -o sbom_out.cdx.json for cdxgen in v11.1.0 or higher against the repository containing the source code of SecObserve.

Additionally, it seems odd that a npm package has a dependency on 'pkg:pypi/secobserve@latest' according to the generated SBOM.

[prabhu]

The project has many moving parts, so cdxgen must be invoked multiple times for each language type, and the results aggregated using the cyclonedx CLI. If you have some cycles, you can try investigating an approach to make cdxgen work with a single invocation, but it won't be easy.

In general, generating a proper SBOM is an involved process. Users must be willing to invest time configuring and fine-tuning settings for maximum precision. How can they expect a tool to work magic and operate out of the box with a single invocation for arbitrary real-world projects?

[mx1up]

i also have this validation error, but an sbom still seems to be generated. can it be ignored? 🤔

[martin-reich]

@mx1up I do not think the error can be ignored, since the dependencies are listed twice and validations of the SBOM will fail when using it with other tools afterwards, i.e. SBOM utility for evaluation purposes.

@prabhu since you know the codebase well as a maintainer and the error can be pinpointed to v11.1.0 as mentioned above (all previous versions work fine!). Could the changes introduced by #1567 lead to the duplication error? I did not mean to be harsh and expect the tool to work out of the box.

[prabhu]

Good find. @malice00, should we improve this logic to filter duplicates?

 let parentDependencies = dependencies.find(
       (d) => d["ref"] === parentComponent["bom-ref"])

I have another angry client who reported the same issue, so back at work at 9pm just before the Valentine's Day ;)

[prabhu]

#1633