KEMBAR78
Allow mypy to output a junit file with per-file results by mrwright · Pull Request #16388 · python/mypy · GitHub
Skip to content

Conversation

mrwright
Copy link
Contributor

@mrwright mrwright commented Nov 1, 2023

Adds a new --junit-format flag to MyPy, which affects the structure of the junit file written when --junit-xml is specified (it has no effect when not writing a junit file). This flag can take global or per_file as values:

  • --junit-format=global (the default) preserves the existing junit structure, creating a junit file specifying a single "test" for the entire mypy run.
  • --junit-format=per_file will cause the junit file to have one test entry per file with failures (or a single entry, as in the existing behavior, in the case when typechecking passes).

In some settings it can be useful to know which files had typechecking failures (for example, a CI system might want to display failures by file); while that information can be parsed out of the error messages in the existing junit files, it's much more convenient to have that represented in the junit structure.

Tests for the old and new junit structure have been added.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 1, 2023

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I can see how the per-file format can be help in CI.

@JukkaL JukkaL merged commit 8c57df0 into python:master Nov 3, 2023
@mrwright mrwright deleted the junit-format branch November 3, 2023 14:35
mrwright added a commit to mrwright/mypy that referenced this pull request Nov 6, 2023
msullivan pushed a commit that referenced this pull request Nov 7, 2023
#16388 introduced a bug where, with `--junit-format=global`, the junit
file would indicate an error (with no message) even if everything
passed. That was because `_generate_junit_contents` would check if
`messages_by_file` was empty or not to determine if there were failures,
but with `--junit-format=global` we'd pass in a dictionary of the form
`{None: all_messages}`; `all_messages` would be empty, but the resulting
dictionary wouldn't be.

The fix is to pass in an empty dictionary if there are no messages.

I've tested manually with `--junit-format=global` and
`--junit-format=per_file` in the successful case to make sure the files
are written correctly now.
svalentin added a commit that referenced this pull request Nov 8, 2023
#16388 changed the definition of
`write_junit_xml` but missed a call site in dmypy. This fixes it.
JelleZijlstra pushed a commit that referenced this pull request Jan 2, 2024
#16388 introduced a bug where invalid xml could be produced by
`write_junit_xml`, as special characters were no longer being escaped.

The fix is to revert the removal of `from xml.sax.saxutils import
escape` and `escape("\n".join(messages))` in the `write_junit_xml`
function.

I've added a small test case which checks that the `<`, `>`, `&` are
escaped correctly in the xml.
wyattscarpenter added a commit to wyattscarpenter/mypy that referenced this pull request Sep 17, 2025
the inclusion into imports group seems to have been a mistake in python#16388 although one could perhaps argue it is tangentially related to imports in some way. But it does not influence import discovery, unlike the other options.
hauntsaninja pushed a commit that referenced this pull request Sep 20, 2025
Elaborate on the information given in --help and command_line.rst for
junit, to make it more correct and comprehensive.

I manually examined the generated results and found them satisfactory.

Note that this also puts --junit-format into misc group not import
group; the inclusion into imports group seems to have been a mistake in
#16388 although one could perhaps argue it is tangentially related to
imports in some way. But it does not influence import discovery, unlike
the other options. Putting it into import group also makes it display in
a completely different place, which is not as helpful as right next to
its related option.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants