Add GHCR cleanup workflow to delete old container images#3136
Add GHCR cleanup workflow to delete old container images#3136AnjaliMishra1st wants to merge 8 commits intoopen-telemetry:mainfrom
Conversation
|
@AnjaliMishra1st nice work on adding this job, just some things to consider here. This cleanup job is quite aggressive, the otel demo release cadence is usually every few months when a number of changes have accumulated. This job would run every day which give that release cadence is very high. However, we do have a nightly build of images. Also another thing to consider is users of older images in the github container registry, the demo is used a lot by different users to showcase how to instrument a cloud service, we may break a lot of demo's by removing older versions of images. For example using It would be great to hear from the maintainers here @open-telemetry/demo-maintainers . |
|
Thanks for the helpful feedback! |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
Hi maintainers 👋 @open-telemetry/demo-maintainers |
julianocosta89
left a comment
julianocosta89
left a comment
There was a problem hiding this comment.
@AnjaliMishra1st sorry for the delay, last week was KubeCon and I'm still playing catch!
Regarding this PR I think we need to adjust a couple of things.
The Demo currently releases nightly builds, but we also have releases.
If I understood this gh workflow correctly, that will clean up everything that is older than 20 versions, but it doesn't differentiate releases from nightly builds.
Which means that if we run this today, we would remove versions 2.2.0, 2.1.3, and so on.
Ideally the clean up would run just for the nightly builds.
|
Hi @julianocosta89 👋🏽 I’ll push the changes soon. Thanks again! 🙂 |
|
Is there a way we can run this in some kind of dry-run mode first to see what images it would delete? |
|
Hi @julianocosta89 @puckpuck |
|
This looks fine. As it's configured right now nothing destructive will happen, and at 3am it will execute in dry_run mode producing a list of images that it would delete. If that looks good, we can do another PR to update the dry_run setting to be false. |
|
@AnjaliMishra1st it's failing the yamllint and checklicense checks. We need to add a license copyright header to the file (see other GH action workflows for an example), and yamllint is likely extra white space or empty lines. The failed check output will have better details on the offending entries. |
There was a problem hiding this comment.
Pull request overview
Adds a GitHub Actions workflow intended to prune older GHCR container image versions to reduce registry storage usage in open-telemetry/opentelemetry-demo.
Changes:
- Introduces a new scheduled + manually-dispatchable workflow for GHCR cleanup.
- Uses GitHub CLI (
gh api) to list container package versions and delete matching versions (currently based onnightly-*tags).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| on: | ||
| schedule: | ||
| - cron: '0 3 * * 0' | ||
| workflow_dispatch: |
| versions=$(gh api \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions \ | ||
| --jq '.[].id') |
.github/workflows/cleanup-ghcr.yml
Outdated
|
|
||
| versions=$(gh api \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions \ |
| versions=$(gh api \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions \ | ||
| --jq '.[].id') | ||
|
|
||
| for version in $versions; do | ||
| tags=$(gh api \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions/$version \ | ||
| --jq '.metadata.container.tags[]?') | ||
|
|
||
| for tag in $tags; do | ||
| if [[ "$tag" == nightly-* ]]; then | ||
| echo "Found nightly image: $tag (version: $version)" | ||
|
|
||
| if [ "$DRY_RUN" = "true" ]; then | ||
| echo "[DRY RUN] Would delete version ID: $version" | ||
| else | ||
| echo "Deleting version ID: $version" | ||
| gh api --method DELETE \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions/$version | ||
| fi | ||
| else |
| gh api --method DELETE \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| /orgs/open-telemetry/packages/container/otel-demo/versions/$version | ||
| fi |
.github/workflows/cleanup-ghcr.yml
Outdated
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v3 |
There was a problem hiding this comment.
After hash add some comment related to version, see other files
|
|
||
| jobs: | ||
| cleanup: | ||
| runs-on: ubuntu-latest |
.github/workflows/cleanup-ghcr.yml
Outdated
| - name: Install GitHub CLI | ||
| run: sudo apt-get update && sudo apt-get install gh -y |
.github/workflows/cleanup-ghcr.yml
Outdated
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v3 |
There was a problem hiding this comment.
All other GH actions pins versions to the exact commit + tag comment.
It should be applied also here.
There was a problem hiding this comment.
Good catch, I’ve updated this to use a pinned commit SHA and added a version comment for consistency with other workflows.
.github/workflows/cleanup-ghcr.yml
Outdated
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Install GitHub CLI | ||
| run: sudo apt-get update && sudo apt-get install gh -y |
There was a problem hiding this comment.
are you sure that it is needed? I think that GitHub CLI should be available by default on all-defualt runmers.
There was a problem hiding this comment.
You're right, I’ve removed the installation step since GitHub CLI is already available on default runners.
.github/workflows/cleanup-ghcr.yml
Outdated
| run: sudo apt-get update && sudo apt-get install gh -y | ||
|
|
||
| - name: Authenticate GitHub CLI | ||
| run: echo "${{ secrets.GITHUB_TOKEN }}" | gh auth login --with-token |
There was a problem hiding this comment.
I think that this one is also not-necessary. In OTel .NET repository there is usage of gh-cli, but there is no need to login.
There was a problem hiding this comment.
Thanks for pointing this out, removed the unnecessary authentication step.
.github/workflows/cleanup-ghcr.yml
Outdated
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v3 |
There was a problem hiding this comment.
After hash add some comment related to version, see other files
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Hi @Kielek @julianocosta89 @puckpuck
|
This PR introduces a GitHub Actions workflow to clean up old container images from GHCR.
This helps reduce storage clutter and maintain a clean registry.