KEMBAR78
:seedling: cron: repair gitlab project list by spencerschrock · Pull Request #4658 · ossf/scorecard · GitHub
Skip to content

Conversation

spencerschrock
Copy link
Member

What kind of change does this PR introduce?

cron update

What is the current behavior?

Many projects were missing the last character of their URL, which was causing 404 errors and resulting in empty shard files in the cron.

What is the new behavior (if this is a feature change)?**

The list was repaired using a bash script and an earlier copy of this file:

touch out.csv
while read original; do
  # remove trailing comma
  no_comma=$(echo -n $original | sed 's/,*\r*$//')
  # look for the complete line in the old file
  grep "$no_comma.,$" gitlab-projects.csv.1 >> out.csv
  # if not in the old copy, just keep the line the same
  if [ $? -eq 1 ]; then
    echo "$original" >> out.csv
  fi
done <gitlab-projects.csv

https://raw.githubusercontent.com/ossf/scorecard/8521e516d809edf2b2696e302464d3d5984816a2/cron/internal/data/gitlab-projects.csv

Here's an example diff:

diff --git a/cron/internal/data/gitlab-projects.csv b/cron/internal/data/gitlab-projects.csv
index 5e0b8ca2..70b02d0a 100755
--- a/cron/internal/data/gitlab-projects.csv
+++ b/cron/internal/data/gitlab-projects.csv
@@ -1,61 +1,62 @@
 repo,metadata
-https://gitlab.com/0100001001000010/config-loade,
-https://gitlab.com/0100001001000010/simple-gui-prompt,
-https://gitlab.com/01luisfonseca/canvas-image-resize,
-https://gitlab.com/01luisfonseca/express-reverse-prox,
-https://gitlab.com/01luisfonseca/file-image-resize,
-https://gitlab.com/01luisfonseca/lfutil,
-https://gitlab.com/06chaynes/okta-jwt-verifie,
+https://gitlab.com/0100001001000010/config-loader,
+https://gitlab.com/0100001001000010/simple-gui-prompts,
+https://gitlab.com/01luisfonseca/canvas-image-resizer,
+https://gitlab.com/01luisfonseca/express-reverse-proxy,
+https://gitlab.com/01luisfonseca/file-image-resizer,
+https://gitlab.com/01luisfonseca/lfutils,
+https://gitlab.com/06chaynes/okta-jwt-verifier,
 https://gitlab.com/09jwater/Needle,
  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

NONE

Many projects were missing the last character of their URL, which was
causing 404 errors and resulting in empty shard files in the cron. The
list was repaired using a bash script and an earlier copy of this file:

https://raw.githubusercontent.com/ossf/scorecard/8521e516d809edf2b2696e302464d3d5984816a2/cron/internal/data/gitlab-projects.csv

Signed-off-by: Spencer Schrock <sschrock@google.com>
@spencerschrock spencerschrock requested a review from a team as a code owner June 13, 2025 22:17
@spencerschrock spencerschrock requested review from justaugustus and raghavkaul and removed request for a team June 13, 2025 22:17
@codecov
Copy link

codecov bot commented Jun 13, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.29%. Comparing base (353ed60) to head (33584a7).
⚠️ Report is 205 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4658      +/-   ##
==========================================
+ Coverage   66.80%   68.29%   +1.49%     
==========================================
  Files         230      249      +19     
  Lines       16602    18898    +2296     
==========================================
+ Hits        11091    12907    +1816     
- Misses       4808     5131     +323     
- Partials      703      860     +157     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencerschrock
Copy link
Member Author

Will wait to merge until we get the token issue resolved

@github-actions
Copy link

This pull request has been marked stale because it has been open for 10 days with no activity

@github-actions
Copy link

This pull request has been marked stale because it has been open for 10 days with no activity

@justaugustus justaugustus enabled auto-merge (squash) August 3, 2025 07:05
@justaugustus justaugustus merged commit 8e47e42 into ossf:main Aug 3, 2025
38 checks passed
@spencerschrock spencerschrock deleted the gitlab-repair branch August 4, 2025 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants