README
¶
OSS Rebuild
Secure open-source package ecosystems by originating, validating, and augmenting build attestations.
Overview
OSS Rebuild aims to apply reproducible build concepts at low-cost and high-scale for open-source package ecosystems.
Rebuilds are derived by analyzing the published metadata and artifacts and are evaluated against the upstream package versions. When successful, build attestations are published for the upstream artifacts, verifying the integrity of the upstream artifact and eliminating many possible sources of compromise.
We currently support the following ecosystems:
- npm (JavaScript/TypeScript)
- PyPI (Python)
- Crates.io (Rust)
While complete coverage is the aim, only the most popular packages within each ecosystem are currently rebuilt.
Usage
The oss-rebuild CLI tool provides access to OSS Rebuild data:
$ go run github.com/google/oss-rebuild/cmd/oss-rebuild@latest --help
$ # Alternatively, install the binary locally.
$ # Just make sure it's on your PATH: https://go.dev/ref/mod#go-install
$ go install github.com/google/oss-rebuild/cmd/oss-rebuild@latest
$ oss-rebuild --help
To view the rebuild for a given package, use the get command:
$ oss-rebuild get pypi absl-py 2.0.0
By default, this provides only a summarized view. For more granular access to
rebuild data, use one of the --output formats. For example, to access the
entire attestation payload, use the --output=payload option:
$ oss-rebuild get pypi absl-py 2.0.0 --output=payload
To view the dockerfile, use the --output=dockerfile option. This can be
chained with docker to execute a rebuild locally:
$ oss-rebuild get pypi absl-py 2.0.0 --output=dockerfile | docker run $(docker buildx build -q -)
While the above --output=payload option produces more human-readable
content, the raw attestation bundle can be accessed as follows:
$ oss-rebuild get pypi absl-py 2.0.0 --output=bundle
To explore more packages, the list command can be used to view the versions of
a package that have been rebuilt:
$ oss-rebuild list pypi absl-py
Usage Requirements
oss-rebuild uses a public Cloud KMS key to validate attestation signatures.
Anonymous authentication is not supported so an ADC credential must be present.
This can be accomplished with:
$ gcloud init
$ gcloud auth application-default login
To disable signature verification and skip the requirement for KMS access use: --verify=false.
Contributing
Join us in building a more secure and reliable open-source ecosystem!
Check out the contribution guide to learn more.
Purpose
- Mitigate supply chain attacks: Detect discrepancies in open-source packages, helping to prevent compromises like those of Solarwinds and Codecov.
- Scale security standards: Utilize industry best practices such as SLSA, Sigstore, and containerized builds.
- Community participation: Create a venue to collectivize effort towards securing the open-source supply chain.
- Enable future innovation: Derive data to leverage AI-driven rebuilds.
Security
To better understand the security properties of rebuilds, see Trust and Rebuilds.
Related Projects
Check out these related projects contributing to the reproducible builds effort:
- reproducible-central: Java, Kotlin reproducibility.
- kpcyrd/rebuilderd: Rebuild scheduler with support for several distros.
Disclaimer
This is not an officially supported Google product.
Directories
¶
| Path | Synopsis |
|---|---|
|
analyzer
|
|
|
example/analyzer
command
|
|
|
example/subscriber
command
|
|
|
network/analyzer
command
|
|
|
network/subscriber
command
|
|
|
build
|
|
|
container
Package container provides routines to programmatically build container components of the project.
|
Package container provides routines to programmatically build container components of the project. |
|
cmd
|
|
|
agent
command
|
|
|
agent-api
command
|
|
|
api
command
|
|
|
gateway
command
gateway provides a simple HTTP server that redirects to the provided URI applying the configured policy.
|
gateway provides a simple HTTP server that redirects to the provided URI applying the configured policy. |
|
git_cache
command
Package main implements a git repo cache on GCS.
|
Package main implements a git repo cache on GCS. |
|
gsutil_writeonly
command
|
|
|
inference
command
|
|
|
oss-rebuild
command
|
|
|
proxy
command
Package main defines an HTTP(S) proxy.
|
Package main defines an HTTP(S) proxy. |
|
rebuilder
command
main contains the smoketest rebuilder, which triggers a rebuild local to this binary (not GCB).
|
main contains the smoketest rebuilder, which triggers a rebuild local to this binary (not GCB). |
|
registry
command
|
|
|
stabilize
command
|
|
|
timewarp
command
The timewarp binary serves the registry timewarp HTTP handler on a local port.
|
The timewarp binary serves the registry timewarp HTTP handler on a local port. |
|
internal
|
|
|
cache
Package cache provides an interface and implementations for caching.
|
Package cache provides an interface and implementations for caching. |
|
gateway
Package gateway provides a client for the gateway service.
|
Package gateway provides a client for the gateway service. |
|
gitx
Package git provides rebuilder-specific git abstractions.
|
Package git provides rebuilder-specific git abstractions. |
|
hashext
Package hashext provides extensions to the standard crypto/hash package.
|
Package hashext provides extensions to the standard crypto/hash package. |
|
httpegress
Package httpegress provides a client constructor for building an HTTP Client for making requests to external services.
|
Package httpegress provides a client constructor for building an HTTP Client for making requests to external services. |
|
httpx
Package http provides a simpler http.Client abstraction and derivative uses.
|
Package http provides a simpler http.Client abstraction and derivative uses. |
|
proxy/dockerfs
Package dockerfs defines a FS interface for accessing files in a Docker container.
|
Package dockerfs defines a FS interface for accessing files in a Docker container. |
|
proxy/handshake
Package handshake contains adaptations of the builtin golang TLS implementation to read part of the handshake.
|
Package handshake contains adaptations of the builtin golang TLS implementation to read part of the handshake. |
|
semver
Package semver implements the Semantic Versioning 2.0.0 spec.
|
Package semver implements the Semantic Versioning 2.0.0 spec. |
|
serviceid
Package serviceid provides helpers for parsing and canonicalizing build identifiers from a repository URI and a Go module pseudo-version.
|
Package serviceid provides helpers for parsing and canonicalizing build identifiers from a repository URI and a Go module pseudo-version. |
|
timewarp
Package timewarp implements a registry-fronting HTTP service that filters returned content by time.
|
Package timewarp implements a registry-fronting HTTP service that filters returned content by time. |
|
verifier
Package verifier provides a library for verifying and attesting to a rebuild.
|
Package verifier provides a library for verifying and attesting to a rebuild. |
|
pkg
|
|
|
analyzer
Package analyzer provides common utilities for analyzer services.
|
Package analyzer provides common utilities for analyzer services. |
|
archive
Package archive provides common types and functions for archive processing.
|
Package archive provides common types and functions for archive processing. |
|
attestation
Package attestation provides utilities for working with OSS Rebuild attestations.
|
Package attestation provides utilities for working with OSS Rebuild attestations. |
|
proxy/cert
Package cert provides certificate generation and formatting interfaces.
|
Package cert provides certificate generation and formatting interfaces. |
|
proxy/docker
Package docker defines a proxy for the Docker API.
|
Package docker defines a proxy for the Docker API. |
|
proxy/policy
Package policy defines the network policy that the proxy can choose to enforce.
|
Package policy defines the network policy that the proxy can choose to enforce. |
|
rebuild/rebuild
Package rebuild provides functionality to rebuild packages.
|
Package rebuild provides functionality to rebuild packages. |
|
rebuild/schema
Package schema is a set of utilities for marshalling strategies.
|
Package schema is a set of utilities for marshalling strategies. |
|
registry/cratesio
Package cratesio provides interfaces for interacting with the crates.io API and with Cargo-specific formats.
|
Package cratesio provides interfaces for interacting with the crates.io API and with Cargo-specific formats. |
|
registry/maven
Package maven provides an interface with Maven package registry and its API.
|
Package maven provides an interface with Maven package registry and its API. |
|
registry/pypi
Package pypi describes the PyPi registry interface.
|
Package pypi describes the PyPi registry interface. |
|
tools
|
|
|
agent
command
|
|
|
benchmark
Package benchmark provides interfaces related to rebuild benchmarks.
|
Package benchmark provides interfaces related to rebuild benchmarks. |
|
benchmark/combine
command
Package main implements a CLI tool to combine multiple benchmark files.
|
Package main implements a CLI tool to combine multiple benchmark files. |
|
benchmark/generate
command
Package main generates rebuild benchmark files from external data sources.
|
Package main generates rebuild benchmark files from external data sources. |
|
ctl
command
|
|
|
ctl/ide
Package ide contains UI and state management code for the TUI rebuild debugger.
|
Package ide contains UI and state management code for the TUI rebuild debugger. |
|
ctl/pipe
Package pipe provides a simple way of applying transforms to a channel.
|
Package pipe provides a simple way of applying transforms to a channel. |
|
ctl/rundex
Package rundex provides access to metadata about runs and attempts.
|
Package rundex provides access to metadata about runs and attempts. |
|
docker
Package docker contains container execution APIs.
|
Package docker contains container execution APIs. |
|
flow
command
|
|
|
indexscan
command
Package main implements a repo scanning tool to identify the best ref match for an upstream artifact.
|
Package main implements a repo scanning tool to identify the best ref match for an upstream artifact. |
|
medic
command
|
|
|
pypi_rss
command
PyPI RSS Subscriber for OSS Rebuild This tool is a long-running service that fetches updates from PyPI's RSS feed, and adds rebuild attempts into a task queue for any release of a package that's considered "tracked".
|
PyPI RSS Subscriber for OSS Rebuild This tool is a long-running service that fetches updates from PyPI's RSS feed, and adds rebuild attempts into a task queue for any release of a package that's considered "tracked". |
|
registryscan
command
|
|
|
run_local
command
Package main builds and runs a rebuild server.
|
Package main builds and runs a rebuild server. |
Click to show internal directories.
Click to hide internal directories.