Incompatibility with SBOM from @cyclonedx/cyclonedx-npm

[rkg-mm]

Current Behavior

We found some kind of incompatibility with some SBOMs generated by @cyclonedx/cyclonedx-npm. We could not reproduce this with SBOMs from the older NPM tool or other sbom tools.

When a project exists with a BOM from @cyclonedx/cyclonedx-npm, and you try to upload a very different BOM file or one generated by another tool, upload API will return 200, but the "Last BOM Upload" timestamp will not update, neither will the components reflect what the latest uploaded BOM contained.

The log shows an error:

2022-12-07 12:12:15,441 [] INFO [org.dependencytrack.tasks.BomUploadProcessingTask] Processing CycloneDX BOM uploaded to project: 6c3b8a36-7454-4a8a-934d-5e28a0c6258c
2022-12-07 12:12:17,067 [] INFO [org.dependencytrack.tasks.BomUploadProcessingTask] Identified 0 new components
2022-12-07 12:12:17,067 [] INFO [org.dependencytrack.tasks.BomUploadProcessingTask] Processing CycloneDX dependency graph for project: 6c3b8a36-7454-4a8a-934d-5e28a0c6258c
2022-12-07 12:12:21,220 [] ERROR [org.dependencytrack.tasks.BomUploadProcessingTask] Error while processing bom
javax.jdo.JDOObjectNotFoundException: Object with id "org.dependencytrack.model.Component:0" not found !
	at org.datanucleus.api.jdo.JDOAdapter.getJDOExceptionForNucleusException(JDOAdapter.java:634)
	at org.datanucleus.api.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManager.java:1726)
	at org.dependencytrack.persistence.ComponentQueryManager.recursivelyDelete(ComponentQueryManager.java:414)
	at org.dependencytrack.persistence.ComponentQueryManager.reconcileComponents(ComponentQueryManager.java:517)
	at org.dependencytrack.persistence.QueryManager.reconcileComponents(QueryManager.java:771)
	at org.dependencytrack.tasks.BomUploadProcessingTask.inform(BomUploadProcessingTask.java:138)
	at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:101)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.datanucleus.exceptions.NucleusObjectNotFoundException: Object with id "org.dependencytrack.model.Component:0" not found !
	at org.datanucleus.store.rdbms.request.FetchRequest.execute(FetchRequest.java:473)
	at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.fetchObject(RDBMSPersistenceHandler.java:354)
	at org.datanucleus.state.StateManagerImpl.loadFieldsFromDatastore(StateManagerImpl.java:1608)
	at org.datanucleus.state.StateManagerImpl.validate(StateManagerImpl.java:5570)
	at org.datanucleus.ExecutionContextImpl.findObject(ExecutionContextImpl.java:3446)
	at org.datanucleus.ExecutionContextImpl.findObject(ExecutionContextImpl.java:2928)
	at org.datanucleus.api.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManager.java:1721)
	... 8 common frames omitted

Steps to Reproduce

1. Upload a BOM file generated by @cyclonedx/cyclonedx-npm (not sure if special conditions apply here regarding the packages, we identified at least 2 projects BOMs showing this behaviour)
2. Upload a very different BOM file to update the Dependency-Track content. In our case we replaced the NPM tool in one project with the webpack-cyclonedx tool, which produced a completely different BOM (as expected). But it also reproduces when just import a BOM from any other tool or project
3. When trying to upload another bom again and again to the same project you will notice more and more of the old components are being deleted, doing it often enough will result eventually in a success at some point when all old components are deleted.

Expected Behavior

1. BOM processing should not fail, result after processing should reflect the new BOM, no matter what was in the old BOM
2. If anything fails, I would expect nothing to be changed instead changing it step-by-step during multiple attempts. Can we use transactions here?

Dependency-Track Version

4.6.2

Dependency-Track Distribution

Container Image

Database Server

Microsoft SQL Server

Database Server Version

No response

Browser

Google Chrome

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this defect was already reported

[rkg-mm]

From first analysis and having in mind the discussions regarding duplication of components, which are produced in BOMs from @cyclonedx/cyclonedx-npm (see https://github.com/CycloneDX/cyclonedx-node-npm/milestone/2 and CycloneDX/cyclonedx-node-npm#307), we think this is happening due to some kind of duplication/relation/tree issue.

Uploading the "faulty" BOM to Dependency-Track, Downloading the BOM that Dependency-Track offers in the UI (which contains no graph information), and uploading that to a new project to use as base, then trying to reproduce this with another BOM will show, that the BOM generated by Dependency-Track does not fail anymore. So there likely must be some problem when processing the graph contained in the original BOM.

The error seems to happen in the recursive deletion of old components. Since the new BOM is a lot different to the old, with a lot less components, a lot of components have to be deleted. Somehow during this process some components always are NULL already which leads to this issue.

This possibly can also be a bug in the bom file generated by @cyclonedx/cyclonedx-npm, but Dependency-Track needs to be failure resistent and if it succeeds importing a BOM file it should always be able to replace this content with a new BOM file. I assume the stored relations are inconsistent somehow.

[valentijnscholten]

The error and possibly scenario (recursive / duplicate delete of components) seems similar to #2131.

[rkg-mm]

@valentijnscholten you are right. This sounds related

[nscuro]

@rkg-mm Okay from your side to close this issue as duplicate of #2131? Am trying to consolidate issues related on this topic.

[rkg-mm]

Yes fine for me

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.