SARIF report does not contain information about the failing rules #17131
Description
The problem
When Checkstyle is executed with SarifLogger, the resulted report does not contain any data in runs[].tool.driver.rules.
Without information in rules, various static analysis tools that consume SARIF reports, cannot provide context information to the end users (e.g. description of the rule, related problem statement, possible fixes, etc).
Details
$ cat config.xml
<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
"-//Puppy Crawl//DTD Check Configuration 1.2//EN"
"http://www.puppycrawl.com/dtds/configuration_1_2.dtd">
<module name="Checker">
<module name="TreeWalker">
<module name="EmptyCatchBlock" />
<module name="EmptyBlock" />
</module>
</module>
$cat CheckstyleViolations.java
// File with Checkstyle violations only
package com.example;
public class CheckstyleViolations {
public void method() {
if (true) { } // EmptyBlock violation
try {
System.out.println("Try with empty catch");
// EmptyCatchBlock violation
} catch (Exception e) {
}
}
}
$ java $RUN_LOCALE -jar checkstyle-10.24.0-all.jar -c config.xml -f sarif CheckstyleViolations.java
{
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
"version": "2.1.0",
"runs": [
{
"tool": {
"driver": {
"downloadUri": "https://github.com/checkstyle/checkstyle/releases/",
"fullName": "Checkstyle",
"informationUri": "https://checkstyle.org/",
"language": "en",
"name": "Checkstyle",
"organization": "Checkstyle",
"rules": [
],
"semanticVersion": "10.24.0",
"version": "10.24.0"
}
},
"results": [
{
"level": "error",
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "file:/private/tmp/checkstyle-17131/CheckstyleViolations.java"
},
"region": {
"startColumn": 19,
"startLine": 7
}
}
}
],
"message": {
"text": "Must have at least one statement."
},
"ruleId": "block.noStatement"
},
{
"level": "error",
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "file:/private/tmp/checkstyle-17131/CheckstyleViolations.java"
},
"region": {
"startColumn": 31,
"startLine": 12
}
}
}
],
"message": {
"text": "Empty catch block."
},
"ruleId": "catch.block.empty"
}
]
}
]
}
Checkstyle ends with 2 errors.
The ruleId refers to something that does not exist in the rules.
Is this by design or could it be improved to provide rules data?
Migration notes
SARIF spec: https://docs.oasis-open.org/sarif/sarif/v2.1.0/errata01/os/sarif-v2.1.0-errata01-os-complete.html
-
Added
rulessection to SARIF output. Each check/module now appears as a rule with metadata including:- Rule ID (source class name + optional module ID)
- Short description (module name)
- Full description (module description from metadata)
- Message strings (violation message templates with keys)
-
Added
message.idsupport. Violation messages now include the message key when available, allowing tools to reference specific violation types
Before:
{
"tool": {
"driver": {
"name": "Checkstyle",
"rules": []
}
},
"results": [{
"ruleId": "com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck",
"message": {
"text": "Unused import - java.util.List."
}
}]
}After:
{
"tool": {
"driver": {
"name": "Checkstyle",
"rules": [{
"id": "com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck#optionalId",
"shortDescription": {"text": "UnusedImports"},
"fullDescription": {"text": "Checks for unused import statements..."},
"messageStrings": {
"import.unused": {"text": "Unused import - {0}."}
}
}]
}
},
"results": [{
"ruleId": "com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck#optionalId",
"message": {
"id": "import.unused",
"text": "Unused import - java.util.List."
}
}]
}I also used Qodana plugin in IntelliJ to check updated reports. Here's how one of the reports lookes:
Before:
After:
