[Bug] Some analyzers report are not accessible

[mphbig]

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	Debian 9
OS version (client)	10
TheHive version / git hash	4.1.1
Cortex version	3.0.1
Package Type	DEB
Browser type & version	Firefox and Chrome latest

Problem Description

When launching an analyzer on an observable, its report is accessible but when we go to another page (even in the same case), the report disappears. The logs (on the right side of the page) still show the analyze being launched and succeeding.

This bug is observed only with a file observable and the EmlParser analyzer.

Tried nodetool flush, nodetool cleanup and reindexing, but it did not fix the issue.

Steps to Reproduce

1. Open a case containing a file observable
2. Launch the EmlParser analyzer on the eml observable
3. The report is fine for now
4. Go to another page (in TheHive or in the case)
5. Open the eml observable, report is gone from the analyzers list.

Complementary information

These logs are generated when launching the analyze and on every retry.

Mar 25 16:25:10 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-19 [|2144b3b8] uncaught error, not retrying
org.thp.scalligraph.CreateError: Observable already exists
    at org.thp.thehive.services.ObservableSrv.create(ObservableSrv.scala:95)
	at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$5(JobSrv.scala:229)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$4(JobSrv.scala:228)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$10(JanusDatabase.scala:259)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$8(JanusDatabase.scala:259)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:89)
	at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:253)
	at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$2(JobSrv.scala:226)
	at scala.concurrent.Future$.$anonfun$traverse$1(Future.scala:850)
	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
	at scala.collection.immutable.List.foldLeft(List.scala:91)
	at scala.concurrent.Future$.traverse(Future.scala:850)
	at org.thp.thehive.connector.cortex.services.JobSrv.importCortexArtifacts(JobSrv.scala:220)
	at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$finished$5(JobSrv.scala:155)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:56)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:91)
	at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:106)
	at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:89)
	at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:74)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:56)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Mar 25 16:25:10 LS-SRV-HIVE01 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-19 [|2144b3b8] Exception raised, rollback (Observable already exists)

These logs, when printing the observable.

Mar 25 16:17:28 [WARN] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-37 [|] An error occurs (Neither the sideEffects, map, nor path has a 58694f42-e2ee-4c54-b1aa-c497e29e047b-key: WherePredicateStep(eq(58694f42-e2ee-4c54-b1aa-c497e29e047b))), retrying (1)
Mar 25 16:17:28 [WARN] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-37 [|] An error occurs (Neither the sideEffects, map, nor path has a 58694f42-e2ee-4c54-b1aa-c497e29e047b-key: WherePredicateStep(eq(58694f42-e2ee-4c54-b1aa-c497e29e047b))), retrying (2)

[ThomasHeimann242]

I can confirm this for Thehive 4.1.2 and Cortex 3.1.1. With 4.0.5 and Cortex 3.1.1. everything was still OK. The EMLParser and the JOE Sandbox Analyzer are affected for me. Same error in the application log. The EML Parser report can be accessed via the analysis history. But no tag is generated, not even with reindex.