[LEGAL] revisit automated license compliance checks

[jkowalleck]

currently using flict==1.2.x for license compatibility checks - which uses osadl-matrix for the relevant functionality.
unfortunately, osadl-matrix is no longer maintained(deprecated!) (see priv-kweihmann/osadl-matrix#209)

so we do need a replacement for

cyclonedx-node-yarn/.github/workflows/nodejs.yml

Lines 134 to 176 in f921c0a

	test-licenses:
	needs: [ 'build' ]
	name: test licenses
	runs-on: ubuntu-latest
	timeout-minutes: 10
	steps:
	- name: install flict
	run: pip install flict==1.2.14
	- name: Checkout
	# see https://github.com/actions/checkout
	uses: actions/checkout@v4
	- name: Setup Node.js ${{ env.NODE_ACTIVE_LTS }}
	# see https://github.com/actions/setup-node
	uses: actions/setup-node@v4
	with:
	node-version: ${{ env.NODE_ACTIVE_LTS }}
	# cache: 'yarn'
	- name: Setup yarn
	run: corepack enable yarn
	- name: Setup subject
	run: yarn install --immutable
	- name: fetch build artifact
	# see https://github.com/actions/download-artifact
	uses: actions/download-artifact@v4
	with:
	name: ${{ env.BUNDLES_DIR }}
	path: ${{ env.BUNDLES_DIR }}
	- name: make NOTICE and summary
	run: |
	mkdir -p _tmp
	yarn node tools/write-3rd-party-licenses.cjs _tmp/NOTICE _tmp/lsummary
	- name: artifact build result
	# see https://github.com/actions/upload-artifact
	uses: actions/upload-artifact@v4
	with:
	name: licenses-files
	path: |
	_tmp/NOTICE
	_tmp/lsummary
	retention-days: 5
	if-no-files-found: error
	- name: test license compatibility
	run: flict display-compatibility $(cat _tmp/lsummary)

---

Possible alternatives:

• https://github.com/aboutcode-org/dejacode
• https://github.com/hesa/licomp-toolkit
• ... to be continued ...
• maybe ask @pombredanne for advice,
• maybe find peers with similar problems, and ask how they solved it
• maybe find somebody from the https://github.com/oss-review-toolkit/ort community who has some advice.

---

re: vinland-technology/flict#398 (comment)

[...] if you are only interested in compatibility between two licenses only and not an expression (e.g. MIT OR Apache-2.0) or the licenses of all the dependencies of a project, then you can look at https://github.com/hesa/licomp-toolkit. Let's say you want to check if you can use MIT as inbound license with CC0-1.0 as outbound:

pip install licomp-toolkit
licomp-toolkit --local verify -ol CC0-1.0 -il MIT | jq .summary

[pombredanne]

@hesa ping

[hesa]

@pombredanne, thanks for roping me in

@jkowalleck, hi again. When looking at compatibility some context is needed (see below). I guess you are interested in the compatibility between:

• your component under an outbound license
• your component's dependencies are linked and unmodified
• you distribute the combined work

Is the above your use case?

Examples of context:

• how it is provided (e.g. distributed)
• used as a library, as a compiler ....
• modified or unmodified

[jkowalleck]

situation is the following:

• any yarn-plugin distributable must be just one file. So we need to have all 3rd-party runtime-dependencies shipped within that in one file, too.
• the build process is more like assembling, it does tree-shaking, re-arrangement, minify, and much more - so it modifies the original work, it does not pure linking/bundling.
• we intend to distribute this assembled file.
• after tree-shaking, we end up with
  • a qualified list of all actually bundled dependencies' licenses,
    similar to this: https://github.com/CycloneDX/cyclonedx-node-yarn/releases/download/v1.0.2/NOTICE
  • a unified list of declared licenses for all actually bundled dependencies,
    similar to this:

    Apache-2.0
    BSD-2-Clause
    CC-BY-3.0
    CC0-1.0
    ISC
    MIT
    

• our own source code is under Apache-2.0.
• we intend to put our distributable under Apache-2.0.
  To check this, the flict license compatibility checker was used so far.

I would be grateful for any help.

I wonder, if https://github.com/hesa/licomp-toolkit could do the needed check for me.
I do not have any constraints, it's more like a "Do you see the tool fitting my workflow/aim?"-thing - if so, i will invest the time and get to know the tool better.

[pombredanne]

@jkowalleck FWIW, I would not consider tree-shaking, re-arrangement, minification, and bundling as "modifications" (and add transpilation from including webassembly to that list). These are transformations from a source to a deployed binary-like format, and this is essentially the same process you go through when you compile C code that's eventually linked with other objects in an ELF exe or so.

These transformations are mechanically applied to run the code and would not trigger "change-related" licensing clauses in the general case.

The interesting parts here are that:

• when you call and depend directly or indirectly on a JS library, you are eventually executing this in the same process, and copyleft terms "may" extend to the calling code, all the way
• when you bundle copyleft JS code in a bundle, the copyleft terms "will" extend to the bundled code, including LGPL.

[hesa]

we intend to put our distributable under Apache-2.0.

So, we need to look at compatibility when distributing.

I wonder, if https://github.com/hesa/licomp-toolkit could do the needed check for me.

licomp-toolkit can only check compatibility between two licenses at a time. If you want to check an entire project (with dependencies and complex licenses (e.g. BSD-3-Clause OR Apache-2.0) you need to write some code on your own or use flict...... or implement display-compatibility in licomp-toolkit?

Once I know what you need (see below) I think letting flict use licomp-toolkit instead of osadl_matrix is not such a big effort and something I can look at straight away.

To me, it sounds like:

• you distribute a combined work where your component/program with a couple of dependencies
• you do no modifications on the dependencies following @pombredanne reasoning (to which I agree)
• your component/program links (as opposed to for example copy snippets) to the dependencies following @pombredanne reasoning (to which I agree) and thus copyleft is triggered

[hesa]

@jkowalleck How do you use the results from your command run: flict display-compatibility $(cat _tmp/lsummary)? Or perhaps explain what you want to check (perhaps there is a better way to get the wanted result)?

[jkowalleck]

@jkowalleck How do you use the results from your command run: flict display-compatibility $(cat _tmp/lsummary)? Or perhaps explain what you want to check (perhaps there is a better way to get the wanted result)?

file _tmp/lsummary looks like this:

Apache-2.0
Apache-2.0
BSD-2-Clause
CC-BY-3.0
CC0-1.0
ISC
MIT

First line is the license we plan to use for publishing our distributable.
the other lines are all the licenses of our effective runtime 3rd-party dependencies.

I intended to display the license compatibility, and most important,
I expected the command to exit with a non-zero status code if anything was incompatible.
I swear there was a non-zero status code in that positive-error case, but I don't see this behavior right now, so I must be wrong.

Well, anyway, my intention was the following:
have flict fail with a non-zero status code in case the 3rd part licenses were not compliant to our intended Apache-2.0

I am a licensing noob.
Could you advise how i could have an automatism that checks that all 3rd party runtime dependencies have a compatible license? or: how to have a GitHub workflow fail, as soon as a non-compliant 3rd party license appears?
I mean, I am able to gather the declared 3rd-party licenses, and i am able to script something in python; I just dont know flict well enough, nor do i have a deep understanding of licensing at all.

[hesa]

flict and return values
flict display-compatibility ... will return:

• 0 if it succeeds displaying license compatibility (even if there is no compatibility).
• non 0 (actually 11) if it cannot determine the compatibility (e.g. if a license is unknown).

Compatibility between two licenses - missing context

Checking if BSD-3-Clause and GPL-2.0-or-later are compatible will serve as an example why the question if two licenses are compatible misses context:

• can you license a combined work under GPL-2.0-or-later when using a component under BSD-3-Clause? Yes, you can
• can you license a combined work under BSD-3-Clause when using a component under GPL-2.0-or-later? No, you can not

.... what would the answer to a question like "are BSD-3-Clause and GPL-2.0-or-later compatible with each other" be? You would need to answer something along the lines "well, it depends on...".

So checking licenses for compatibility depends on the context (see above), e.g. the outbound license (the license you want to release your combined work under).

Checking compatibility

If you want to release your combined work (your component with its dependencies) under MIT, then you need to check (using the licenses you listed above):

• can I use a component licensed under Apache-2.0 in my work licensed under MIT?
• can I use a component licensed under BSD-2-Clause in my work licensed under MIT?
• can I use a component licensed under CC-BY-3.0 in my work licensed under MIT?
• can I use a component licensed under CC0-1.0 in my work licensed under MIT?
• can I use a component licensed under ISC in my work licensed under MIT?
• can I use a component licensed under MIT in my work licensed under MIT?

To check the above with flict or licomp-toolkit you can do (only one example):

• flict verify -ol Apache-2.0 -il MIT - returns 0 if compatible, 1 if not compatible
• licomp-toolkit verify -ol Apache-2.0 -il MIT | jq .summary.results - you need to inspect the result to get the result. Adding an issue to add return values

Potentially you could check if all of the above licenses (as you listed) are compatible in one go with flict (leaving out CC0-1.0 since OSADL does not support it):

$ flict verify -ol Apache-2.0 -il Apache-2.0 and BSD-2-Clause and CC-BY-3.0 and ISC and MIT  ; echo $?
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "Apache-2.0 AND BSD-2-Clause AND CC-BY-3.0 AND ISC AND MIT", "original_inbound": "Apache-2.0 and BSD-2-Clause and CC-BY-3.0 and ISC and MIT", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
0

[jkowalleck]

re: #242 (comment)
thank you so much.

I will revisit the related GH workflows and might consult you for a review.

[jkowalleck]

working on this

[jkowalleck]

still using flict==1.2.14

Restructured my data to something like this

{"ol":"Apache-2.0","ils":["Apache-2.0","BSD-2-Clause","...","MIT", "GPL-2.0-only"]}

remark - incompatible licenses: GPL-2.0-only and Apache-2.0

---

I tried the following

#!/usr/bin/env bash
set -ue

LICENSES_JSON="$1"

flict verify \
-ol "$(jq -r '.ol' "$LICENSES_JSON")" \
-il "$(jq -r '.ils | join(" AND ")' "$LICENSES_JSON")";

as expected, this exited with a 1 and the following output:

{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "Apache-2.0 AND BSD-2-Clause AND CC-BY-3.0 AND CC0-1.0 AND ISC AND MIT AND GPL-2.0-only", "original_inbound": "Apache-2.0 AND BSD-2-Clause AND CC-BY-3.0 AND CC0-1.0 AND ISC AND MIT AND GPL-2.0-only", "result": {"outbound_licenses": [], "allowed_outbound_licenses": [], "outbound_license": "", "problems": []}}

---

so i went with the following, to get some details which license caused the issues:

#!/usr/bin/env bash
set -ue

LICENSES_JSON="$1"

OL="$(jq -r '.ol' "$LICENSES_JSON")"

jq -r '.ils[]' "$LICENSES_JSON" | while read -r IL
do
  flict verify -ol "$OL" -il "$IL"
done

as expected, this exited with a 1 and the following output:

{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "Apache-2.0", "original_inbound": "Apache-2.0", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "BSD-2-Clause", "original_inbound": "BSD-2-Clause", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "CC-BY-3.0", "original_inbound": "CC-BY-3.0", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "CC0-1.0", "original_inbound": "CC0-1.0", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "ISC", "original_inbound": "ISC", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "MIT", "original_inbound": "MIT", "result": {"outbound_licenses": ["Apache-2.0"], "allowed_outbound_licenses": ["Apache-2.0"], "outbound_license": "Apache-2.0", "problems": []}}
{"original_outbound": "Apache-2.0", "outbound": "Apache-2.0", "inbound": "GPL-2.0-only", "original_inbound": "GPL-2.0-only", "result": {"outbound_licenses": [], "allowed_outbound_licenses": [], "outbound_license": "", "problems": []}}

---

conclusion: both are valid options.

[jkowalleck]

thanks to your explanations, @hesa 🥇 , i was able to craft #247
I think no additional review is needed, it basically does the things you proposed, just in the way you proposed. But every review is appreciated.