Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: performance issue because of re-loading harmonization file #2106

Merged
1 commit merged into from
Sep 9, 2021

Conversation

waldbauer-certat
Copy link
Contributor

The harmonization file had been loaded within each new report, this nearly kills the CPU, because it has to open the file, parse the file & load it in the memory.

@waldbauer-certat waldbauer-certat requested a review from a user September 9, 2021 14:17
@waldbauer-certat waldbauer-certat added bug Indicates an unexpected problem or unintended behavior performance labels Sep 9, 2021
@waldbauer-certat waldbauer-certat changed the base branch from develop to maintenance September 9, 2021 14:18
@waldbauer-certat waldbauer-certat force-pushed the fix-2098 branch 2 times, most recently from 278e755 to c674949 Compare September 9, 2021 14:34
@codecov-commenter
Copy link

codecov-commenter commented Sep 9, 2021

Codecov Report

Merging #2106 (fbe1035) into maintenance (4e2039c) will not change coverage.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##           maintenance    #2106   +/-   ##
============================================
  Coverage        76.05%   76.05%           
============================================
  Files              427      427           
  Lines            22886    22886           
  Branches          3045     3045           
============================================
  Hits             17405    17405           
  Misses            4777     4777           
  Partials           704      704           
Impacted Files Coverage Δ
intelmq/lib/bot.py 62.05% <100.00%> (ø)

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a changelog entry

@ghost ghost added this to the 3.0.2 milestone Sep 9, 2021
@waldbauer-certat waldbauer-certat force-pushed the fix-2098 branch 3 times, most recently from 7f5615a to 65f199b Compare September 9, 2021 16:21
As of now each new message re-loads the actual harmonization conf,
which unfortuneatly results in a high CPU drain.

This commit adds the harmonization parameter to the function call,
so the harmonization is loaded from the internal python memory
allocated space & no more file read/write has to be done.

Fixes #2098

Signed-off-by: Sebastian Waldbauer <[email protected]>
@ghost ghost merged commit 644b3c5 into maintenance Sep 9, 2021
@ghost ghost deleted the fix-2098 branch September 9, 2021 16:23
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants