KEMBAR78
:bug: Skip tag-only rulesets during Branch-Protection by trask · Pull Request #4699 · ossf/scorecard · GitHub
Skip to content

🐛 Skip tag-only rulesets during Branch-Protection #4699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jul 14, 2025
Merged

🐛 Skip tag-only rulesets during Branch-Protection #4699

merged 4 commits into from
Jul 14, 2025

Conversation

trask
Copy link
Contributor

@trask trask commented Jul 10, 2025

Problem

Scorecard was incorrectly applying repository rulesets that target tags to branch protection analysis. This caused false positive warnings about admin enforcement when a repository had:

  • Tag-only rulesets with bypass actors (targeting tags, not branches)
  • Proper branch protection rules that do enforce admin compliance

The issue occurred because the repoRuleSet struct was missing the Target field from the GraphQL query, causing all rulesets to be treated as if they applied to branches.

Solution

  1. Added missing Target field to the repoRuleSet struct to capture the ruleset target from GitHub's GraphQL API
  2. Updated rulesMatchingBranch function to filter out rulesets where Target != "branch"
  3. Updated GraphQL query to include the target field in the rulesets query

Testing

Verified the fix by running Scorecard against the https://github.com/open-telemetry/opentelemetry-java-contrib repository.

@trask trask requested a review from a team as a code owner July 10, 2025 03:21
@trask trask requested review from raghavkaul and spencerschrock and removed request for a team July 10, 2025 03:21
@spencerschrock
Copy link
Member

spencerschrock commented Jul 10, 2025 •
edited
Loading

incorrectly applying repository rulesets that target tags to branch protection analysis

We do have some feature requests for this kind of analysis though. Both are requested as new checks, so perhaps re-enabling tag analysis is a problem for another day/place

#10
#2476

@trask
Copy link
Contributor Author

trask commented Jul 10, 2025

We do have some feature requests for this kind of analysis though

makes sense, I made some updates to make it more future proof for when those features land

spencerschrock
spencerschrock reviewed Jul 11, 2025
View reviewed changes
@trask trask had a problem deploying to integration-test July 11, 2025 18:01 — with GitHub Actions Failure
@trask trask temporarily deployed to integration-test July 14, 2025 16:04 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Jul 14, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 68.29%. Comparing base (353ed60) to head (244a329).
Report is 194 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #4699      +/-   ##
==========================================
+ Coverage   66.80%   68.29%   +1.49%     
==========================================
  Files         230      249      +19     
  Lines       16602    18898    +2296     
==========================================
+ Hits        11091    12907    +1816     
- Misses       4808     5131     +323     
- Partials      703      860     +157
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencerschrock
Copy link
Member

/scdiff generate Branch-Protection

@github-actions
Copy link

Here's a link to the scdiff run

spencerschrock
spencerschrock approved these changes Jul 14, 2025
View reviewed changes
Copy link
Member

@spencerschrock spencerschrock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but you'll need to fix DCO https://github.com/ossf/scorecard/pull/4699/checks?check_run_id=45863683831

trask added 4 commits July 14, 2025 09:25
@trask
Fix branch protection probe false warnings by filtering out tag-only …
918b618 
…rulesets

Signed-off-by: Trask Stalnaker <trask.stalnaker@gmail.com>
@trask
more future proof
d4edad1 
Signed-off-by: Trask Stalnaker <trask.stalnaker@gmail.com>
@trask
Revert "more future proof"
238a2ca 
This reverts commit df8855a.

Signed-off-by: Trask Stalnaker <trask.stalnaker@gmail.com>
@trask
uppercase
244a329 
Signed-off-by: Trask Stalnaker <trask.stalnaker@gmail.com>
@trask trask temporarily deployed to integration-test July 14, 2025 16:40 — with GitHub Actions Inactive
@spencerschrock spencerschrock changed the title Fixes false warnings in the "branch protection applies to admins" probe when repositories have tag-only rulesets configured 🐛 Skip tag-only rulesets during Branch-Protection Jul 14, 2025
@spencerschrock spencerschrock merged commit e6b369d into ossf:main Jul 14, 2025
40 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spencerschrock spencerschrock spencerschrock approved these changes

@raghavkaul raghavkaul Awaiting requested review from raghavkaul raghavkaul is a code owner automatically assigned from ossf/scorecard-maintainers

Assignees

No one assigned

Labels

None yet

Projects

OpenSSF Scorecard
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@trask @spencerschrock