Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add URLhaus analyzer #227

Merged
merged 2 commits into from
May 29, 2018
Merged

Add URLhaus analyzer #227

merged 2 commits into from
May 29, 2018

Conversation

ninoseki
Copy link
Contributor

@ninoseki ninoseki commented Apr 12, 2018

Referencing to #226, I've created a URLhaus analyzer.
Could you please check it?

@3c7 3c7 added category:enhancement Issue is related to an existing feature to improve scope:analyzer Issue is analyzer related status:pr-submitted labels Apr 12, 2018
@3c7 3c7 added this to the 1.10.0 milestone Apr 12, 2018
@3c7
Copy link
Contributor

3c7 commented Apr 12, 2018

Hey @ninoseki, thanks for your contribution.

It would be great to implement caching, so urlhaus/abuse.ch is queried only once even if analyzing lots of observables in the same time. For an example how to implement caching, you can take a look at the Tor analyzers available.

If you're feeling not familiar with implementing caching, I will implement it if I find some time.

@ninoseki
Copy link
Contributor Author

Thank you for the advice.
OK, I'll implement caching.

@ninoseki
Copy link
Contributor Author

@3c7 how is this?

@3c7 3c7 changed the base branch from develop to release/1.10.0 May 29, 2018 08:17
@3c7
Copy link
Contributor

3c7 commented May 29, 2018

@ninoseki Damn, I'm sorry. I thought you're working on the csv export data (their API, basically) from URLHaus. That's why I wanted to have some kind of caching mechanism added.

Copy link
Contributor

@3c7 3c7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your contribution.

@3c7 3c7 merged commit 7832cdb into TheHive-Project:release/1.10.0 May 29, 2018
@3c7 3c7 mentioned this pull request May 29, 2018
@saadkadhi
Copy link
Contributor

This analyzer, while very good, requires requests-html which is only compatible with Python 3.6. However Ubuntu 16.04 LTS and other OSes do not support yet that version and we cannot force everyone to upgrade to 3.6.

We will have sadly to postpone it to another release after modifying the code to use the CSV export from URLhaus or have some kind of requests-html equivalent that works with Python 3.5. @ninoseki Are you willing to consider doing so?

Thanks!

@saadkadhi saadkadhi removed this from the 1.10.0 milestone Jun 6, 2018
@saadkadhi saadkadhi removed category:enhancement Issue is related to an existing feature to improve status:needs-review status:pr-submitted labels Jun 6, 2018
@3c7 3c7 mentioned this pull request Jun 6, 2018
@ninoseki
Copy link
Contributor Author

ninoseki commented Jun 6, 2018

@saadkadhi Oh, it’s absolutely my fault. Let me re-implement the analyzer without requests-html.

@3c7
Copy link
Contributor

3c7 commented Jun 6, 2018

Hey @ninoseki please see details written in #271.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:analyzer Issue is analyzer related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants