No analyzers available after full upgrade?

[xme]

Work Environment

Question	Answer
OS version (server)	Ubuntu,
OS version (client)	Ubuntu,
TheHive version / git hash	3.4.0
Package Type	Docker

Problem Description

I can't run any analyzers after a full upgrade. Message is:
"Sorry, there are currently no analyzers for the selected observable type(s)"

Steps to Reproduce

1. Select an observable
2. Select "run analyzers"

Complementary information

I upgraded my dockers: ES 6.8.5, Cortex 3.0.0 and TheHive 3.4.0.
Cortex integration is ok (green logo), I can start jobs from the Cortex web interface but nothing from TheHive. No specific error message in logs!?

[xme]

Connection with Cortex looks ok from TheHive:
2019-12-18 07:51:56,018 [INFO] from connectors.cortex.services.CortexClient in main - new Cortex(Cortex, http://xxxxxxx) authentication: connectors.cortex.services.CortexAuthentication$Key 2019-12-18 07:51:56,019 [INFO] from connectors.cortex.services.CortexAnalyzerSrv in main - Search for unfinished job ... 2019-12-18 07:51:56,571 [INFO] from connectors.cortex.services.CortexAnalyzerSrv in application-akka.actor.default-dispatcher-2 - 0 jobs found

[jeromeleonard]

Hi @xme,
Could you please confirm that the API key used in TheHive is owned by an account in the same organisation the account you used in Cortex to run jobs successfully, and has read/analyze permissions ?

[xme]

Yes, confirmed. Tested just now.

[xme]

I've a tcpdump between the 2 dockers and I see TheHive talking to Cortex. No auth issue, user is correct. It seems to be the classic polling at regular interval:

GET /api/status
GET /api/user/current

But nothing when I try to launch analyzers... So it seems to be located on TheHive side.

[xme]

I restarted my containers... TheHive did the expected request:

GET /api/analyzer?range=all
And I got the list of available Analyzers back...
Example of Analyzer's data returned:

1
[
2f6
{"_routing":"xxxxxx","dockerImage":null,"analyzerDefinitionId":"DomainTools_Risk_2_0","author":"CERT-BDF","rateUnit":"Month","_type":"worker","description":"Use DomainTools to get a risk score and evidence details on a domain or fqdn","dataTypeList":["domain","fqdn"],"type":"analyzer","version":"2.0","jobCache":null,"url":"https://github.com/TheHive-Project/Cortex-Analyzers","command":"/opt/Cortex-Analyzers/analyzers/DomainTools/domaintools_analyzer.py","license":"AGPL-V3","createdAt":1536613366339,"_parent":"xxxxxxxx","rate":100,"createdBy":"xavier","name":"DomainTools_Risk_2_0","workerDefinitionId":"DomainTools_Risk_2_0","_id":"f7d2c135bb4abed558fedb93de673100","id":"f7d2c135bb4abed558fedb93de673100","_version":1,"baseConfig":"DomainTools"}
1
,
350

[nadouani]

Hello @xme,

After your latest comment, is the problem still there?
Are you talking about "run analyzers" action from the observables list?
What about running analyzers from the observable details page?

Nabil

[xme]

Interesting finding: when I try to run analyzers from the detail page, it prompts to select the Cortex instance... empty!

I also restarted the TheHive container and changed the Cortex instance name... just in case but same problem.

[nadouani]

Ok, can you share the response of /api/status? I've already seen someone talking about this finding, but wasn't able to reproduce.

[xme]

Email is ok? To?

[nadouani]

Yes, go ahead

[xme]

Something probably interesting:

thehive.xxx       | [error] c.c.s.CortexAnalyzerSrv - Request to Cortex fails
thehive.xxx       | play.api.libs.json.JsResultException: JsResultException(errors:List(((39),List(JsonValidationError(List('version' is undefined on object: {"_routing":"xxx","analyzerDefinitionId":"FileInfo_6_0","_type":"worker","description":"Parse files in several formats
 such as OLE and OpenXML to detect VBA macros, extract their source code, generate useful information on PE, PDF files...","dataTypeList":["file"],"type":"analyzer","jobCache":null,"createdAt":1566507379458,"_parent":"xxx","createdBy":"xavier","name":"FileInfo_6_0","workerDefinition
Id":"FileInfo_6_0","_id":"91ffb85a34e93cc738e0bcd27752cd0c","id":"91ffb85a34e93cc738e0bcd27752cd0c","_version":1}),WrappedArray()))), ((51),List(JsonValidationError(List('version' is undefined on object: {"_routing":"xameco.net","analyzerDefinitionId":"Cymon_Check_IP_2_1","_type":"worker",
"description":"Check an IP addr against Cymon.io.","dataTypeList":["ip"],"type":"analyzer","jobCache":null,"createdAt":1534926084656,"_parent":"xxx","createdBy":"xavier","name":"Cymon_Check_IP_2_1","workerDefinitionId":"Cymon_Check_IP_2_1","_id":"5fb2a7462f2fe155b00bb266078a818c","i
d":"5fb2a7462f2fe155b00bb266078a818c","_version":1}),WrappedArray())))))

[nadouani]

I don't like the 'version' is undefined on object {...}, sounds like an analyzer definition without a version attribute, let me check

[nadouani]

The Cymon analyzer has been removed look at this (TheHive-Project/Cortex-Analyzers#489)

[xme]

Ok, thank a lot Nabil for your time! The issue was related to 2 old Analyzers that have been removed but still present in the DB (I did a migration from the previous Cortex instance). The 2 Analyzers have been removed from the DB using this command:

curl -XDELETE http://localhost:9200/cortex_4/doc/ANALYZER_ID\?routing\=ORGANIZATION