Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] FortiGuard cannot parse response content #491

Closed
srilumpa opened this issue Jun 4, 2019 · 1 comment
Closed

[Bug] FortiGuard cannot parse response content #491

srilumpa opened this issue Jun 4, 2019 · 1 comment
Labels
category:bug Issue is related to a bug

Comments

@srilumpa
Copy link
Contributor

srilumpa commented Jun 4, 2019

Describe the bug
For some reason (I was not able to identify which one exactly), the response form fortiguard.com is transfered as a binary string to re.search() which crashes as follows:

Fortiguard/urlcategory.py <<< '{"data":"http://trello.com","dataType":"url"}'
Traceback (most recent call last):
  File "./urlcategory.py", line 54, in <module>
    URLCategoryAnalyzer().run()
  File "./urlcategory.py", line 43, in run
    category_match = re.search(pattern, req.content, flags=0)
  File "/usr/local/lib/python3.7/re.py", line 183, in search
    return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object

Using req.text instead of req.content fixed the issue.

To Reproduce
Steps to reproduce the behavior:

  1. execute ./Fortiguard/urlcategory <<< '{"data":"http://trello.com/", "dataType":"url"}' (URL does not matter)
  2. wiat for the crash

I was not able to reproduce it on my laptop but this always generate a crash on our production server. I don't know if this comes from python dependencies, server environment or other stuff...

Expected behavior
Expect to have a successful result.

Complementary information
When adding a print(req.content) juste after executing the query, the result looks like the following:

b'<!DOCTYPE html>\n<html>\n<head>\n    <meta charset="utf-8">\n    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">\n    <meta name="title" property="title" content="Web Filter Lookup" />\n    [...] </body>\n</html>\n'

Work environment

  • Server OS: Debian 9 stretch
  • Cortex version: N/A
  • Cortex Analyzer/Responder name: Fortiguard_URLCategory
  • Cortex Analyzer/Responder version: 2.1

Possible solutions

  • Use req.text instead of req.content when passing the result of the query to re.search()
  • Use a try ... catch to handle binary format
@srilumpa srilumpa added the category:bug Issue is related to a bug label Jun 4, 2019
@srilumpa srilumpa mentioned this issue Jun 4, 2019
Merged
@jeromeleonard
Copy link
Contributor

of req.content.decode(req.apparent_encoding)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
category:bug Issue is related to a bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@srilumpa @jeromeleonard