mispwarninglist with postgres initialization not working

[hoboristi]

Request Type

Bug

Work Environment

N/A

Question	Answer
OS version (server)	Ubuntu 18.04
OS version (client)	Ubuntu 20.04
Cortex Analyzer Name	Mispwarninglists
Cortex Analyzer Version	2.9.3
Cortex Version	3.0.1-1 stable
Browser type & version	n/A

Description

Tried to use postgres backend for mispwarninglist, but the initialization failed according to https://blog.thehive-project.org/;
both python2 and python3 throws exceptions.

Steps to Reproduce

(keep this section only if the issue relates to a bug)

1. Got the latest analyzers from git, and tried to use the mispwarninglist analyzer db, so created a db instance
2. filled warninglists_create_db.py according to my settings:

conn_string = "postgresql+psycopg2://postgres:mypassword@myhost:5432/warninglists"
warninglists_path = "/usr/local/etc/misp-warninglists/lists/**/list.json"

1. executed with python2:
   $python2 warninglists_create_db.py

File "warninglists_create_db.py", line 53
  logging.debug(f"NAME: {k} - VERSION: {v['version']} - ITEMS: {len(v['list'])}")

1. than python3:
   python3 warninglists_create_db.py

Traceback (most recent call last):
  File "warninglists_create_db.py", line 23, in <module>
    engine = create_engine(conn_string, use_batch_mode=True)
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/__init__.py", line 387, in create_engine
    return strategy.create(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py", line 160, in create
    engineclass.__name__))
TypeError: Invalid argument(s) 'use_batch_mode' sent to create_engine(), using configuration PGDialect_psycopg2/QueuePool/Engine.  Please check that the keyword arguments are appropriate for this combination of components.

                                                                           ^

SyntaxError: invalid syntax

[garanews]

Hello, python2 not supported, going to edit shebang :)
Issue is with sqlalchemy version:

• <= 1.1.x : not supported
• 1.2.x <= & < 1.4 : supported

• = 1.4 : The use_batch_mode flag that was part of the 1.2 version of the feature is removed; the behavior remains controllable via the executemany_mode flag added in 1.3.

We will add support 1.4 when will be ready with fallback since 1.2.0
So I am supposing you are using sqlalchemy <1.2.x and I suggest you to update.

[zambomarcell]

Hi,

Is it possible to fix this issue in requirements.txt?

for example:

MISPWarningLists$ cat requirements.txt 
cortexutils
requests
ipaddress
tld
sqlalchemy>=1.2,<1.4
psycopg2-binary