[Bug] Identical URL Observables can be added multiple times to the same case

[Passimist]

Work Environment

TheHive 4.0.2.
Scalligraph 0.1.0-SNAPSHOT
TheHive 4.0.2-1
Play 2.8.5
CORTEX LOCAL CORTEX - 3.0.1-1 (OK)

Problem Description

Just like in #1640 I can still sometimes add an URL to a case multiple times.
This also happens to URLs that dont contain an IP but this is an example that I can publish here.

Steps to Reproduce

1. Create new Case
2. Create URL observable
3. Waiting a few minutes seems to increase the chance of this to occurr
4. Create the same URL observable in the same case again
5. wonder why it is added / is not added

Possible Solutions

@To-om looked into the referenced issue, maybe he has an idea.

[Passimist]

I also noticed if I select two identical Observables in Thehive and run an analyzer for them both. Cortex will start two analyses even though there is a 10min cache setting for this analyser. But maybe the cache function is not supposed to prevent this szenario?

[Passimist]

@To-om @nadouani Hi guys, I hope you had a nice start into the new year! Can you please reopen this issue?
I just upgraded to TheHive 4.0.3 and I can still upload identical url observables via TheHive4py (not the latest thehive4py version but the one before EDIT tried it with the newest thehive4py version and issue occurrs there aswell).


If I can provide any additional information to help fix this please let me know.
Some information that may help:
Submitted via thehive4py
Submitted almost at the same time
Observables have different observable IDs
This does not always work (but quite often as far as I can tell)
EDIT: As a workaround I now use a filelock to prevent simulaneous creation of identical URL observables. So far this helped preventing the bug. If this is a fix then it might be some kind of race condition? I ll update if the problem occurrs with this workaround in the next few days.