-
Notifications
You must be signed in to change notification settings - Fork 297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: Improved Abuse.ch Feodo Tracker parser bot and documentation #2268
Conversation
687d1da
to
8e1e9f3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm missing the upgrade path. All user which have any of the two parsers (domain or IP) in place, will run into troubles with the upgrade.
Please add the removal of the two bots to the NEWS file and add an upgrade function to intelmq/lib/upgrades.py
.
6bdbc6d
to
375474e
Compare
5e55172
to
f31684e
Compare
This also fixes #1499. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for rewriting this parser and simplifying the setup by doing so.
I suggest to migrate the configuration directly instead of asking the user to do so. The user doesn't have a choice anyway.
messages.append('A discontinued feed "Abuse.ch Feodo Tracker IPs" has been found' | ||
'as bot %s.\nPlease manually replace with the feed' | ||
'"Abuse.ch Feodo Tracker".' % ', '.join(sorted(found_abusech_feodotracker_csv))) | ||
|
||
if found_abusech_feodotracker_browse: | ||
messages.append('A discontinued feed "Abuse.ch Feodo Tracker Browse" has been found' | ||
'as bot %s.\nPlease manually replace with the feed' | ||
'"Abuse.ch Feodo Tracker".' % ', '.join(sorted(found_abusech_feodotracker_browse))) | ||
|
||
if found_abusech_removed_parsers: | ||
messages.append('A discontinued bot module has been found' | ||
'as bot %s.' % ', '.join(sorted(found_abusech_removed_parsers))) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not changing the config directly? Replacing the module and adapting the http_url
can be fully automated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made the explanation into an issue #2297.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you insist on changing the config directly I will do it. Just let me know.
However as a user I prefer only being notified that I need to change to the config. I like to understand the necessary changes even if it requires more (manual) effort. That's why I implemented the upgrades this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd believe that the change shouldn't be done automatically - but it could be suggested. Either as "run command xyz to automatically update documentation" or "please change following things ... ".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Either as "run command xyz to automatically update documentation"
Originally that was the purpose of the intelmqctl upgrade-config
command.
… feeds and parsers. Updates documentation.
This PR replaces two duplicate feeds with an unified and improved one with greatly simplified code. The JSON source from feodotracker.abuse.ch has the most complete data (compared to CSV and Browse HTML table).