[URLhaus] Change of format from URLhaus

[srilumpa]

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	Debian
Cortex Analyzer Name	URLhaus
Cortex Analyzer Version	1.0
Cortex Version	2.0.4

Description

URLhaus analyzer fails to extract the data from the page queried when analyzing an URL. This is probably due to an update of the URLhaus website which changed its layout.

Steps to Reproduce

1. Analyze an URL with URLhaus 1.0 analyzer
2. The analyzer fails

Possible Solutions

Fix the query result parsing.

Complementary information

Here is the analyzer output when analyzing an URL

Invalid output
Traceback (most recent call last):
  File "URLhaus/URLhaus_analyzer.py", line 50, in <module>
    URLhausAnalyzer().run()
  File "URLhaus/URLhaus_analyzer.py", line 23, in run
    'results': self.search(self.get_data())
  File "URLhaus/URLhaus_analyzer.py", line 17, in search
    return URLhaus(indicator).search()
  File "/opt/cortex-analyzers/analyzers/URLhaus/URLhaus.py", line 42, in search
    return self.parse(res)
  File "/opt/cortex-analyzers/analyzers/URLhaus/URLhaus.py", line 52, in parse
    rows = table.find_all("tr")[1:]
AttributeError: 'NoneType' object has no attribute 'find_all'

If I have some time today, I will try to find why it crashes exactly and to fix it

[3c7]

Hey @ninoseki, do you want to implement the changes as it's "your" analyzer? :)

[ninoseki]

@3c7 Thank you for letting me know. 👍
OK, I'll work on this issue.

[ninoseki]

@srilumpa I confirmed that the DOM structure of URLhaus website is same as before and the analyzer works correctly.

So could you tell me the URL you tried to analyze?
(For debugging purpose)

[srilumpa]

Sorry, I should have started by this.

A few URL as examples:

• hxxp://tckkitchen[.]com:80/bst/tasks[.]php
• hxxp://redirector[.]gvt1[.]com:80/edgedl/release2/chrome/A84nRG1q1w8_67[.]0[.]3396[.]99/67[.]0[.]3396[.]99_chrome_installer[.]exe
• hxxp://cacerts[.]digicert[.]com:80/DigiCertSHA2HighAssuranceServerCA[.]crt

May be this was only temporary (some sort of service outage): I don't see any errors since Saturday.

[rolinh]

May be this was only temporary (some sort of service outage)

Look no further, URLHaus had issues last Friday.

[srilumpa]

Sorry for the false alert, then