[malwarebazaar-recent-additions] Allow optional storage of malware samples

[ParamConstructor]

Proposed changes

The core intent of this update is to support the "optional" storage of the malware files related to the Malware Bazaar data fetches. Some users may not wish to "actually" store the "real" malware files on their system disk. This could be due to security requirements, or could be because they never use the actual file and it is needlessly increasing their S3/MinIO storage requirements. This update allows for access to the Artifact entries and metadata about the malware artifacts - but removes the storage of the "real" malware when enabled. The malware is replaced with a file that is 85 bytes in size and when downloaded within OpenCTI states:

"This would normally be Malware, but we have disabled the saving of the real malware."

Changes:

• Cleaned up code for autopep8/black/isort/pylint compliance.

• Added the following in the docker-compose.yml to show connector values could be in the config.yml or here and added the missing Auth-Key token reference that is required for the connector to function properly.

      - MALWAREBAZAAR_RECENT_ADDITIONS_USER_TOKEN=your-token-here # Free Auth-Key Required - https://bazaar.abuse.ch/api/#auth_key
      - MALWAREBAZAAR_RECENT_ADDITIONS_SCORE=40 # From 0 (Unknown) to 100 (Fully trusted); ENV or can be set in config.yml
      - MALWAREBAZAAR_RECENT_ADDITIONS_UPDATE_EXISTING_DATA=false # ENV or can be set in config.yml
      - MALWAREBAZAAR_RECENT_ADDITIONS_CREATE_INDICATOR=false # ENV or can be set in config.yml      - 

• Added the new key disable_malware_sample to both docker-compose.yml and config.yml.sample in order to support NOT actually storing the real malware in S3/MinIO storage.

      - MALWAREBAZAAR_RECENT_ADDITIONS_DISABLE_MALWARE_SAMPLE=true # If true, malware will be replaced with a benign Text file sample.

  disable_malware_sample: false # If true, malware will be replaced with a benign Text file sample.

• Added code changes required to support this capability enhancement.

Related issues

• None

Checklist

• I consider the submitted work as finished
• I tested the code for its functionality using different use cases
  • Tested connector with the disable_malware_sample set to true
  • Tested connector with the disable_malware_sample set to false
• I added/update the relevant documentation (within connector code)
• Where necessary I refactored code to improve the overall quality

Further comments

N/A

[helene-nguyen]

@ParamConstructor Thank you for your contribution!

Some remarks here: CONFIDENCE_LEVEL and UPDATE_EXISTING_DATA for connector are deprecated now.
You can find a full detailed explanation by @Powlinett from our team here: #3316 (review)

I see that are using it later as score, I suggest to remove those parameters and add for example MALWAREBAZAAR_RECENT_ADDITIONS_SCORE=40

Suggested change

	- CONNECTOR_CONFIDENCE_LEVEL=40 # From 0 (Unknown) to 100 (Fully trusted); ENV or can be set in config.yml
	- CONNECTOR_UPDATE_EXISTING_DATA=false # ENV or can be set in config.yml

[ParamConstructor]

@helene-nguyen - I made suggested corrections and moved them down to the local connector level as MALWAREBAZAAR vars.

Based on this (#3526) - maybe I should just remove it and let the connector user (account connector runs as) default the score

[helene-nguyen]

@ParamConstructor I saw that you have rebased master, I assume everything is good now for this ?

[ParamConstructor]

After rebase - two new capabilities had been added to create relationships and indicators via (bc63dde). This had some slight issues where OpenCTI URL and OpenCTI Token had been duplicated down to the MALWAREBAZAAR level (They are available via the config.yml or docker-compose.yml at the top level). I do not believe this duplication was the author's Intent? (@Noxurge - tagged to verify). Additionally, the new Indicators and Relationships should have a Marking for TLP:CLEAR. These issues/corrections are incorporated in the commit - (8a4d4ce) on this PR.

[Noxurge]

Hello @ParamConstructor! It wasn't my intention to do this duplication. My focus is to use OpenCTIApiClient to create the relationship and indicators with SHA256. Thanks for add TLP:CLEAR for this two capabilities. If you have any question for my features, feel free to ping me again.