Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bot debugger fix #1263

Closed
wants to merge 1 commit into from
Closed

bot debugger fix #1263

wants to merge 1 commit into from

Conversation

e3rd
Copy link
Member

@e3rd e3rd commented Jun 27, 2018

collector bots have no input queue and no send_message(path) keyword

@ghost
Copy link

ghost commented Jun 28, 2018

collector bots have no input queue and no send_message(path) keyword

Collectors don't have inputs, but they can send messages. Output's don't send messages.

@e3rd e3rd force-pushed the bot-debugger-fix branch from 3ea81d5 to 0d8f70d Compare June 28, 2018 16:12
@codecov-io
Copy link

codecov-io commented Jun 28, 2018

Codecov Report

Merging #1263 into develop will decrease coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           develop   #1263     +/-   ##
=========================================
- Coverage     75.4%   75.3%   -0.1%     
=========================================
  Files          280     269     -11     
  Lines        12998   12666    -332     
  Branches      1753    1717     -36     
=========================================
- Hits          9801    9538    -263     
+ Misses        2812    2751     -61     
+ Partials       385     377      -8
Impacted Files Coverage Δ
intelmq/bots/outputs/smtp/output.py 25% <0%> (-48.92%) ⬇️
intelmq/bots/parsers/shadowserver/parser.py 83.18% <0%> (-4.89%) ⬇️
intelmq/bots/experts/tor_nodes/expert.py 86.66% <0%> (-2.99%) ⬇️
intelmq/bots/parsers/calidog/parser_certstream.py 33.33% <0%> (-2.67%) ⬇️
...q/bots/collectors/blueliv/collector_crimeserver.py 45.83% <0%> (-2%) ⬇️
intelmq/bots/outputs/mongodb/output.py 63.15% <0%> (-1.71%) ⬇️
intelmq/bots/collectors/misp/collector.py 41.37% <0%> (-1.48%) ⬇️
intelmq/bots/experts/generic_db_lookup/expert.py 62.06% <0%> (-1.09%) ⬇️
intelmq/bots/outputs/stomp/output.py 36.11% <0%> (-1.04%) ⬇️
intelmq/bots/outputs/xmpp/output.py 15.55% <0%> (-0.92%) ⬇️
... and 30 more

@e3rd
Copy link
Member Author

e3rd commented Jun 28, 2018

Thanks, I've added the check that Output's don't send messages and rebased.

Note that my commit doesn't prevent Collectors to send_message but only to send_message with the path keyword because it seems Collectors can't lead to a named queue.

@ghost
Copy link

ghost commented Jul 2, 2018

Note that my commit doesn't prevent Collectors to send_message but only to send_message with the path keyword because it seems Collectors can't lead to a named queue.

Don't know a reason against it. The currently existing one's don't do it, but for custom ones it's possible.
With 30bba26 I pushed a bugfix

@@ -12,6 +12,7 @@
* processes single message, either injected or from default pipeline (process subcommand)
* reads the message from input pipeline or send a message to output pipeline (message subcommand)
"""
import inspect
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you use it for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(sorry, for nothing, removing)

@ghost ghost modified the milestones: 1.1.0, 1.0.6 Jul 7, 2018
@ghost ghost added bug Indicates an unexpected problem or unintended behavior component: intelmqctl labels Jul 7, 2018
@ghost ghost self-assigned this Jul 7, 2018
@ghost ghost added the needs: feedback label Jul 7, 2018
collector bots have no input queue, output bots have no output queue
@e3rd e3rd force-pushed the bot-debugger-fix branch from 0d8f70d to dcf3fd1 Compare July 16, 2018 12:35
@e3rd
Copy link
Member Author

e3rd commented Jul 16, 2018

Ah, okay.

So this PR covers the fix that Collector bot cannot perform "get" and "pop" opererations (because collectors have no input queue) and that Output bot cannot perform send operation (because outpus have no output queue). Does this make sense to you like this?

I've changed the commit message.

ghost pushed a commit that referenced this pull request Jul 17, 2018
@ghost
Copy link

ghost commented Jul 17, 2018

Thanks! Merged into all branches

78ed23a
19854c2
3634ab4
fd2b670
5de96a5

@ghost ghost closed this Jul 17, 2018
@e3rd e3rd deleted the bot-debugger-fix branch July 17, 2018 16:08
ghost pushed a commit that referenced this pull request Sep 3, 2018
Maintenance release 1.0.6

1.0.6 Bugfix release (2018-08-31)

\# Bots
\## Collectors
- `bots.collectors.rt.collector_rt`: Log ticket id for downloaded reports.

\## Parsers
- `bots.parsers.shadowserver`:
  - if required fields do not exist in data, an exception is raised, so the line will be dumped and not further processed.
  - fix a bug in the parsing of column `cipher_suite` in ssl poodle reports (#1288).

\## Experts
- Reverse DNS Expert: ignore all invalid results and use first valid one (#1264).
- `intelmq/bots/experts/tor_nodes/update-tor-nodes`: Use check.torproject.org as source as internet2.us is down (#1289).

\## Outputs
- `bots.output.amqptopic`:
  - The default exchange must not be declared (#1295).
  - Unencodable characters are prepended by backslashes by default. Otherwise Unicode characters can't be encoded and sent (#1296).
  - Gracefully close AMQP connection on shutdown of bot.

\# Documentation
- Bots: document redis cache parameters.
- Installation documentation: Ubuntu needs universe repositories.

\# Packaging
- Dropped support for Ubuntu 17.10, it reached its End of Life as of 2018-07-19.

\# Tests
- Drop tests for Python 3.3 for the mode with all requirements, as some optional dependencies do not support Python 3.3 anymore.
- `lib.test`: Add parameter `compare_raw` (default: `True`) to `assertMessageEqual`, to optionally skip the comparison of the raw field.
- Add tests for RT collector.
- Add tests for Shadowserver Parser:
  - SSL Poodle Reports.
  - Helper functions.

\# Tools
- `intelmqctl list` now sorts the output of bots and queues (#1262).
- `intelmqctl`: Correctly handle the corner cases with collectors and outputs for getting/sending messages in the bot debugger (#1263).
- `intelmqdump`: fix ordering of dumps in a file in runtime. All operations are applied to a sorted list (#1280).

\# Contrib
- `cron-jobs/update-tor-nodes`: Use check.torproject.org as source as internet2.us is down (#1289).
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior component: intelmqctl needs: feedback
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants