Imported sightings' confidence level is always "5 - Improbable"

[misje]

Description

Imported sightings' confidence level is always "5 - Improbable"

Environment

1. OS (where OpenCTI server runs): docker: opencti/platform:6.0.9
2. OpenCTI version: 6.0.9
3. OpenCTI client: 6.0.9
4. Other environment details:

Reproducible Steps

The code in question is an enrichment connector producing a sighting between an observable and a identity (system), using a dummy indicator as sighting_of_ref. The code is not public yet. The connector runs as a user with max confidence set to 100, and confidence is set correctly on other entities and relationships.

I was hoping to provide a minimal STIX JSON example, but OpenCTI fails to import my sighting. There are no errors in the worker logs. The wokbench lists the sighting, but only the observable and entity are available in the database. The JSON is attached. It was produced from an investigation with a simple File observable, System identity and a sighting between them. It was attempted imported using the workbench and ImportFileStix.

sighting.json

Expected Output

The sighting imported from STIX should have the confidence from the user/group running the import/connector.

Actual Output

The confidence is "5 - Improbable", regardless of the user's max confidence level, or the confidence set in STIX. The confidence is correct for entities and relationships.

Additional information

[CelineSebe]

As admin, we can't reproduce with this bundle

[labo-flg]

Hi @misje !

When you validate a workbench in the UI, the confidence level used is the one of the user who validates the workbench.
If the user has confidence 20, even if the bundle contains a sighting at 100, the ingested sighting will be capped at 20 confidence.

[labo-flg]

Could you please check the value in 0-100 (by editing the sighting) for "5-Improbable".
This value is defined somewhere: the user who validates the workbench, in the config.yml, in the users associated to the connectors... we need to find which one is used.

[misje]

Thank you for looking into this. Since my attempt of reproducing the issue using STIX export and import using the workbench failed, and since it isn't really reproducing the issue through an enrichment connector, I'll create a bare-bones enrichment connector snippet running on the latest release. I'll get back to you with the results.

[misje]

Here is a very simple example connector. Run with docker compose up --build, create any SCO and look at the sightings tab for the observable. There should be a sighting in the system "fjas" with confidence set to 100 (the max confidence level of admin), but it is the lowest confidence level instead.

As far as I understand, I should not need to set confidence on entities. The confidence is set to 100 for the admin user for all SDOs and relationships, but not for sightings. I can set the confidence to 100 using the stix property, which is reflected in the platform (at least in the latest version). However, why do I need to do this for sightings and not other objects?

[CelineSebe]

I can't answer this question with certainty..
What version of the platform are you on? The rules are changing to allow the ability to override confidence per entity type in Group, and this feature is already implemented in User.

[misje]

The example references all the versions for 6.1.3 as per the OpenCTI docker project, i.e. the latest release.

[labo-flg]

Here is how it is supposed to work:

When the platform ingest a bundle from a connector, the confidence can come from:

• confidence of the element in the bundle
• confidence of the user associated to the connector
• confidence set in connector's config.yml file => deprecated in favor of the previous one, should have no impact

If the confidence of the element is set, once ingested it is capped with the confidence of the user associated to the connector.
If the confidence of the element is unset, once ingested it is set to the confidence of the user associated with the connector.

In your case, the sighting has a specific value which is apparently wrong. But what is this value on the 0-100 scale?
Once we know that, we can look for this value in every location and check why it is used.

[labo-flg]

Do your sighting end up with the confidence of linked observable ?

[misje]

If the confidence of the element is unset, once ingested it is set to the confidence of the user associated with the connector.

This does not work for sightings.

Do your sighting end up with the confidence of linked observable ?

Observables do not have confidence, as far as I understand. I was told in Slack that it doesn't make much sense (I agree), and it is not visible in the platform. I assume that there is one in the object anyway. If so, what is the confidence on a SCO when it is created in the platform? Is it that of the user, i.e. 100 when admin?

But what is this value on the 0-100 scale?

Where can I see this in the platform? Do I have to export the STIX?

[labo-flg]

Observables do not have confidence, as far as I understand.

You're right, my bad!

Where can I see this in the platform? Do I have to export the STIX?

When you edit an object, you can see the full confidence value and not only the admiralty scale.

[CelineSebe]

I'm not sure if this resolves your issue, but we checked how the confidence level of an indicator is set with a connector. Even if the user's confidence level is set to 100, the confidence level of the connector overwrite the indicator's confidence level. You no longer need to set confidence levels in the connectors.

If you override the maximum confidence level for an indicator on the platform, this override takes precedence. But for the sightings, the confidence level and the override must not exist.

Have you set a confidence level for your user?

[nino-filigran]

@misje I see no activity on this ticket for a while. Can we consider this as fixed? Or do you have a different issue?

[misje]

My solution to this was to set confidence manually to 100 for sightings as a workaround. I don't have time to investigate this any more, I'm sorry.

[nino-filigran]

My bad I misread your comment. The issue still exists then and needs to be fixed. If for relation we manage to have the correct confidence level, there should not be any reason that it's not the case on sightings, which are a type of relation.