Extra log line during dumping message process

[SYNchroACK]

The only "small" reason for the additional line is just to help users on the same problem as I had which is:

A parser bot is processing thousands of lines and in the middle of the process, parser found one line with an error and automatically wrote a log message about it but continued. In the end, after all thousand lines processed, the log file is full of messages saying INFO - Processed 500 messages since last logging. and in the end has an additional log message saying INFO - Dumping message from pipeline to dump file..

In my perspective, before the log line INFO - Dumping message from pipeline to dump file., bot should write another log line mentioning the issue with the specific message that is about to dump in order to alert the user that need to check all log file looking for that error. Otherwise the user may have difficulties to understand where the dump message came from after all processed lines.

[codecov-io]

Codecov Report

Merging #1161 into maintenance will decrease coverage by <.01%.
The diff coverage is 0%.

@@               Coverage Diff               @@
##           maintenance    #1161      +/-   ##
===============================================
- Coverage        75.06%   75.05%   -0.01%     
===============================================
  Files              217      217              
  Lines             9284     9285       +1     
  Branches          1259     1259              
===============================================
  Hits              6969     6969              
- Misses            2029     2030       +1     
  Partials           286      286

Impacted Files	Coverage Δ
intelmq/lib/bot.py	63.82% <0%> (-0.15%)	⬇️

[ghost]

The bug you're trying to fix is specific to the ParserBot, so only fix it there, in process. And to make it more useful we could also give the number of errors in the message (length of self.__failed).

e.g.:

ERROR ...
Sending message ...
... - INFO - %d errors occurred during the processing of the report. See error messages above.
Dumping ...

[ghost]

How do you know that each line represents one event? This is often not the case (think about headers). The number of (sent/failed) events is relevant, not the lines. There can be 10 lines and only one with data.

[ghost]

FYI: 6c3d66a