[Bug] Thehive4 is slow for list alerts

[nicodeff]

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	CentOS
OS version (client)	7.4.1708
TheHive version / git hash	4.0.0-1
Package Type	RPM
Browser type & version	Chrome

Problem Description

Hello,

I am currently making a poc of thehive4 to replace our current version of thehive 3.4.2-1
I migrated all cases and alerts from thehive 3.4 to thehive4 (1242 cases and 12480 alerts).

When I want to go to Thehive alerts page. This page takes long seconds to display alerts.

I carried out some tests via the API and it seems that there is a high latency by applying a "desc" sort on the "date" field (basic behavior carried out by the webUI)

Without sort : time execution less than 1 sec

date; curl https://(thehiveURL)/api/v1/query -s -u (login):(pwd) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "page", "from": 0, "to": 15}]}' --insecure >/dev/null; date
Mon Oct 19 15:59:11 2020
Mon Oct 19 15:59:11 2020

With sort : time execution 7 sec

date; curl https://(thehiveURL)/api/v1/query -s -u (user):(password) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "sort", "_fields": [{"date": "desc"}]},{"_name": "page", "from": 0, "to": 15}]}' --insecure >/dev/null; date
Mon Oct 19 16:03:28 2020
Mon Oct 19 16:03:35 2020

Is this a known issue ?

this latency is for us a blocking point to finalize the migration to thehive4.

Thanks,

Regards,

[miyoyo]

To get accurate execution delays, you can use the time tool as such:

time curl https://(thehiveURL)/api/v1/query -s -u (login):(pwd) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "page", "from": 0, "to": 15}]}' --insecure

[nicodeff]

Hello,
Okey :)

Without sort :

time curl https://(thehiveURL)/api/v1/query -s -u (login):(pwd) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "page", "from": 0, "to": 15}]}' --insecure

real    0m0.458s
user    0m0.050s
sys     0m0.122s

With sort :

time curl https://(thehiveURL)/api/v1/query -s -u (login):(pwd) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "sort", "_fields": [{"date": "desc"}]},{"_name": "page", "from": 0, "to": 15}]} --insecure

real    0m7.143s
user    0m0.042s
sys     0m0.123s

[nadouani]

Hello @nicodeff Thanks for the entry. We are working to improve and fix this type of issues. Stay tuned.

[nicodeff]

Hello @nadouani,

Do you have informations about the fix.
Have you been able to replay this bug in your environment ?

Thanks,

[nadouani]

Hello @nicodeff this performance issue will be globally fixed in TheHive 4.1 (target before end of the year). Work is still in progress

[nicodeff]

Thanks @nadouani fot this great news.
I waiting this update impatiently.

Regards,

[crackytsi]

I really hope that this topic gets a high priority, as its a hard effect when migrating from TheHive3 to TheHive4 ;)

[nicodeff]

Hello @nadouani , this problem will it be fixed in TheHive 4.1 ?
In the Milestones, I don't see any cases for fix this problem.

Thanks,

Regards,

[nicodeff]

Hello,

We had installed TheHive Xmas edition 4.0.3 and we haven't unfortunately observe an amelioration .

Time for 12500 alerts :

time curl http://(URL)/api/v1/query -s -u (login):(password) -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "sort", "_fields": [{"date": "desc"}]},{"_name": "page", "from": 0, "to": 15}]}'

real    0m6.940s
user    0m0.003s
sys     0m0.004s

For listing cases we have not the same problem, but we have 'only' 1300 cases.

time curl http://(URL)/api/v1/query -s -u (login):(password) -H "Content-type: application/json" -d '{"query": [{"_name": "listCase"},{"_name": "sort", "_fields": [{"flag": "desc"}]},{"_name": "page", "from": 0, "to": 15,"extraData":["observableStats","taskStats","isOwner","shareCount","permissions","actionRequired"]}]}'

real    0m6.940s
user    0m0.003s
sys     0m0.004s

Regards,

[rriclet]

This issue has been fixed with #1731, included in release 4.1.0.
Feel free to re-open this issue if you still notice slow performance.

[MonkeySec]

@rriclet @nadouani

This still appears to be an issue. I have recently migrated to The Hive version 4.1.9-1.

We have a total of 247 alerts.

Replicating the same testing queries I see load times of 4 seconds by default, and 17 seconds with sorting. Both of which seem too slow for such a little number of alerts:

Without Sorting - 4 seconds, which is still slower than expected

time curl https://HIVEURL/api/v1/query?name=alerts -s -u USERNAME:PASSWORD -H "Content-type: application/json" -d '{"query": [{"_name":
"listAlert"},{"_name": "page", "from": 0, "to": 15}]}' --insecure

real 0m4.011s
user 0m0.031s
sys 0m0.063s

With Sorting - Over 17 seconds!

time curl https://HIVEURL/api/v1/query?name=alerts -s -u USERNAME:PASSWORD -H "Content-type: application/json" -d '{"query": [{"_name": "listAlert"},{"_name": "sort", "_fields": [{"date": "desc"}]},{"_name": "page", "from": 0, "to": 15}]}' --insecure

real 0m17.926s
user 0m0.047s
sys 0m0.063s