Define tool component object to represent tool driver and its extensions/plugins

[michaelcfanning]

In order to effectively compare two equivalent tool runs expressed as SARIF, it is useful to verify whether the log files were produced by tool configuration with equivalent functional behaviors.

The tool version provides some help here. The tool invocation data also provides some assistance, in cases where things like the command-line arguments are exactly equivalent.

Tools can be parameterized by things referred to on the command-line (such as scripts that drive analysis, or plug-ins that are delay-loaded). These things can be versioned out-of-sync with the tool.

Additionally, some tools discover plug-ins at runtime, from well-known locations (that aren't explicit on a command-line).

It is difficult or impossible to specify SARIF that can accommodate the range of tool behaviors. But have we done enough on the problem of handling rule scripts/plug-ins versioning that is typical of tools such as Semmle? Fortify also has flexible rules customization/authoring. et al.

[ghost]

@michaelcfanning:

Proposal (copied from #207):
Define a plugIn object with properties fileLocation (of type fileLocation) and version (of type string). Give tool a new property plugIns of type "array of plugIn".

Probably the plugIn object would have the same set of versioning properties that tool does: version, semanticVersion, and fileVersion (but not sarifLoggerVersion).

[michaelcfanning]

See #258. As part of completing this analysis, we should be sure to cover the issue of how individual plug-ins both provide rules/resources data and allow for probing for localized information.

[michaelcfanning]

New proposal: provide a tool property that represents a set of file indices to plug-ins. Add a version property to the file object. Now, a tool can reference all its plug-ins. The file information can contain hash and version details to idenify it. Add an optional property to the rule metadata that refers to its container plug-in (by file index). Consider whether we require a new file 'role' to document this case.

[michaelcfanning]

Remove tool.sarifLoggerVersion. The tool object now contains:

• language:string
• driver:toolComponent
• extensions:toolComponent[]

toolComponent can represent a plugin, but also an auxilliary file such as an app.config.

toolComponent takes the version properties from the current tool object (except for sarifLoggerVersion). Those version properties make sense for plugin DLLs, possibly not for auxilliary files.

[michaelcfanning]

@lgolding, one thing to keep in mind here is that we've moved rules metadata to the tool object. if this data is now part of tool component, then a result ruleIndex needs to know what component it is associated with. this implies a toolComponentIndex on each result (that can't point away from the driver into some specific plug-in or component). The absence of this property would indicate that the driver component rulesMetadata s/be examined to retrieve rule data.

[ghost]

@michaelcfanning I agree with your idea for toolComponentIndex.

[michaelcfanning]

Schema changes:

Tool.language retained.
Tool.sarifLoggerVersion deleted
Remainder of tool properties bundled into a toolComponent object.
toolComponent.fileIndex added to that type.
tool.driver is a toolComponent instance and is a required property
tool.extensions is an array of tool components that extend the default tool
result.ruleExtensionIndex can be used to associate a result with an extension
'toolComponent' added as a new file role
conversion.tool is a toolComponent instance (not a tool instance) and is renamed to conversion.driver

[michaelcfanning]

Checked into SARIF SDK schema.

[michaelcfanning]

Please note update to this issue, result.extensionIndex should be named result.ruleExtensionIndex (to provide parallelization with result.ruleIndex...)

[michaelcfanning]

EBALLOT PROPOSAL: define a mechanism to allow a tool to explicitly detail any plug-ins or extensions that were in play during the analysis run (and which potentially contribute to changes in analysis behavior).

API IMPACT

• tool.sarifLoggerVersion property deleted (as it hasn't been helpful to date)
• Define a toolComponent object to represent a tool's driver and its extensions.
  • Transfer all existing tool properties except language to toolComponent, with the following changes:.
    • Change the type of the shortDescription property from message object to multiformatMessageString object.
    • Change the type of the fullDescription property from message object to multiformatMessageString object.
  • Add a property artifactIndex to locate the component in the run.artifacts array.
• In the tool object:
  • Add a property driver of type toolComponent to represent the driver: required
  • Add a property extensions of type array of toolComponent to represent all extensions.
• Add toolComponent as a new file role.

NOTES

1. Since conversion.tool was and remains a tool instance, a converter has a driver and can have extensions.
2. result.rulePointer can be used to associate a result with an extension (see Define a reportingDescriptorReference object #324 for more)

POST-BALLOT NOTE

• Note that Issue Changes to toolComponent properties #336 changes the integer-valued toolComponent.artifactIndex to the integer-array-valued artifactIndices.

[ghost]

E-BALLOT #2 PROPOSAL

We decompose a tool object into a number of toolComponents: a "driver" (the component that contains the tool's primary executable) and zero or more "extensions" (components that affect the tool's behavior, such as plugins and configuration files).

This ballot proposal incorporates the later changes introduced by #336 ("Changes to toolComponent properties").

SCHEMA CHANGES

• In the tool object:

  • Remove the never-used sarifLoggerVersion property.
  • Add a property driver of type toolComponent, required.
  • Add a property extensions of type toolComponent[], optional, minItems: 0, default: empty array
  • Move all other properties to the toolComponent object (name, fullName, version, semanticVersion, dottedQuadFileVersion, and downloadUri) to the toolComponent object (defined next).

• Define a toolComponent object with the following properties:

  • All the properties moved from the tool object (name, fullName, version, semanticVersion, dottedQuadFileVersion, and downloadUri).
  • releaseDateUtc of type string in date-time format -- the component's release date.
  • informationUri of type string in uri format, optional.
  • organization of type string, optional -- the company or organization that produced the component, e.g., "Example Corporation".
  • product of type string, optional -- the name of the product containing the component, e.g., "Example Corp SecurityScanner".
  • productSuite of type string, optional -- the name of the suite of products containing the component, e.g., "Example Corp Code Quality Tools".
  • shortDescription of type multiformatMessageString, optional -- a brief description of the component.
  • fullDescription of type multiformatMessageString, optional -- a comprehensive description of the component.
  • artifactIndices of type integer[], optional, default: empty array -- references to the files which comprise the component.

NOTES

1. Contrary to earlier comments in this issue, we will not yet move rules from resources to toolComponent, and so we will not yet introduce toolComponentIndex. Those concepts were originally introduced in Define a reportingDescriptorReference object #324. They will appear in an upcoming change draft that includes Provide full metadata objects for notifications #311 (notification metadata), Define a reportingDescriptorReference object #324 (reportingDescriptorReference), and Remove current localization mechanism #325 (Remove old localization design).

[ghost]

Neglected to close this after e-ballot-2.