Full Report
Posted by Mihai Maruseac, Google Open Source Security Team (GOSST)In partnership with NVIDIA and HiddenLayer, as part of the Open Source Security Foundation, we are now launching the first stable version of our model signing library. Using digital signatures like those from Sigstore, we allow users to verify that the model used by the application is exactly the model that was created by the developers. In this blog post we will illustrate why this release is important from Google’s point of view.With the advent of LLMs, the ML field has entered an era of rapid evolution. We have seen remarkable progress leading to weekly launches of various applications which incorporate ML models to perform tasks ranging from customer support, software development, and even performing security critical tasks.However, this has also opened the door to a new wave of security threats. Model and data poisoning, prompt injection, prompt leaking and prompt evasion are just a few of the risks that have recently been in the news. Garnering less attention are the risks around the ML supply chain process: since models are an uninspectable collection of weights (sometimes also with arbitrary code), an attacker can tamper with them and achieve significant impact to those using the models. Users, developers, and practitioners need to examine an important question during their risk assessment process: “can I trust this model?”Since its launch, Google’s Secure AI Framework (SAIF) has created guidance and technical solutions for creating AI applications that users can trust. A first step in achieving trust in the model is to permit users to verify its integrity and provenance, to prevent tampering across all processes from training to usage, via cryptographic signing. The ML supply chainTo understand the need for the model signing project, let’s look at the way ML powered applications are developed, with an eye to where malicious tampering can occur.Applications that use advanced AI models are typically developed in at least three different stages. First, a large foundation model is trained on large datasets. Next, a separate ML team finetunes the model to make it achieve good performance on application specific tasks. Finally, this fine-tuned model is embedded into an application.The three steps involved in building an application that uses large language models.These three stages are usually handled by different teams, and potentially even different companies, since each stage requires specialized expertise. To make models available from one stage to the next, practitioners leverage model hubs, which are repositories for storing models. Kaggle and HuggingFace are popular open source options, although internal model hubs could also be used.This separation into stages creates multiple opportunities where a malicious user (or external threat actor who has compromised the internal infrastructure) could tamper with the model. This could range from just a slight alteration of the model weights that control model behavior, to injecting architectural backdoors — completely new model behaviors and capabilities that could be triggered only on specific inputs. It is also possible to exploit the serialization format and inject arbitrary code execution in the model as saved on disk — our whitepaper on AI supply chain integrity goes into more details on how popular model serialization libraries could be exploited. The following diagram summarizes the risks across the ML supply chain for developing a single model, as discussed in the whitepaper.The supply chain diagram for building a single model, illustrating some supply chain risks (oval labels) and where model signing can defend against them (check marks)The diagram shows several places where the model could be compromised. Most of these could be prevented by signing the model during training and verifying integrity before any usage, in every step: the signature would have to be verified when the model gets uploaded to a model hub, when the model gets selected to be deployed into an application (embedded or via remote APIs) and when the model is used as an intermediary during another training run. Assuming the training infrastructure is trustworthy and not compromised, this approach guarantees that each model user can trust the model.Sigstore for ML modelsSigning models is inspired by code signing, a critical step in traditional software development. A signed binary artifact helps users identify its producer and prevents tampering after publication. The average developer, however, would not want to manage keys and rotate them on compromise.These challenges are addressed by using Sigstore, a collection of tools and services that make code signing secure and easy. By binding an OpenID Connect token to a workload or developer identity, Sigstore alleviates the need to manage or rotate long-lived secrets. Furthermore, signing is made transparent so signatures over malicious artifacts could be audited in a public transparency log, by anyone. This ensures that split-view attacks are not possible, so any user would get the exact same model. These features are why we recommend Sigstore’s signing mechanism as the default approach for signing ML models.Today the OSS community is releasing the v1.0 stable version of our model signing library as a Python package supporting Sigstore and traditional signing methods. This model signing library is specialized to handle the sheer scale of ML models (which are usually much larger than traditional software components), and handles signing models represented as a directory tree. The package provides CLI utilities so that users can sign and verify model signatures for individual models. The package can also be used as a library which we plan to incorporate directly into model hub upload flows as well as into ML frameworks.Future goalsWe can view model signing as establishing the foundation of trust in the ML ecosystem. We envision extending this approach to also include datasets and other ML-related artifacts. Then, we plan to build on top of signatures, towards fully tamper-proof metadata records, that can be read by both humans and machines. This has the potential to automate a significant fraction of the work needed to perform incident response in case of a compromise in the ML world. In an ideal world, an ML developer would not need to perform any code changes to the training code, while the framework itself would handle model signing and verification in a transparent manner.If you are interested in the future of this project, join the OpenSSF meetings attached to the project. To shape the future of building tamper-proof ML, join the Coalition for Secure AI, where we are planning to work on building the entire trust ecosystem together with the open source community. In collaboration with multiple industry partners, we are starting up a special interest group under CoSAI for defining the future of ML signing and including tamper-proof ML metadata, such as model cards and evaluation results.
Analysis Summary
The provided context is an extremely truncated snippet of a Google Online Security Blog page detailing an article titled "Taming the Wild West of ML: Practical Model Signing with Sigstore." The actual core content, which would contain the specific recommendations, implementation guidance, and configuration details, is missing (indicated by `[...content truncated...]`).
Therefore, the resulting summary will be based *only* on the implied security topic and the known context of using **Sigstore for Machine Learning (ML) Model Signing**, framed around general best practices for supply chain security in ML environments.
---
# Best Practices: Securing ML Artifacts with Model Signing (Sigstore)
## Overview
These practices address the security risks inherent in the Machine Learning (ML) supply chain, specifically focusing on preventing unauthorized modification or substitution of ML models, training data, and related artifacts. Model signing, utilizing tools like Sigstore, provides cryptographic assurance of the origin and integrity of models throughout their lifecycle.
## Key Recommendations
### Immediate Actions
1. **Inventory ML Artifacts:** Immediately identify all critical ML models, dependencies, and associated build artifacts that require integrity verification.
2. **Establish Identity Store:** Implement or leverage an existing system to cryptographically establish unique identities for builders, developers, and automated CI/CD pipelines responsible for producing models.
3. **Pilot Sigstore Integration:** Begin a proof-of-concept implementation where at least one non-production ML model is signed using the chosen Sigstore component (e.g., Cosign).
### Short-term Improvements (1-3 months)
1. **Integrate Signing into CI/CD:** Mandate cryptographic signing of all *newly built* production-bound ML models as a mandatory stage in the Continuous Integration/Continuous Delivery (CI/CD) pipeline.
2. **Implement Artifact Transparency Log:** Configure all signing operations to publish proofs of signature to a public, immutable transparency log (like Rekor in Sigstore) to provide verifiable attestations of when and by whom an artifact was signed.
3. **Establish Verification Gates:** Integrate automated signature verification checks into deployment pipelines, ensuring no unsigned or tampered model can be promoted to staging or production environments.
### Long-term Strategy (3+ months)
1. **Enforce Full Supply Chain Attestation:** Expand signing and verification beyond just the final model artifact to include signing dependencies, container images used for training/serving, and environment configurations, adhering to a structure like SLSA (Supply chain Levels for Software Artifacts).
2. **Establish Model Provenance Policies:** Develop organizational policies defining required signing metadata (e.g., build environment details, source repository hashes) that must be included in the signature or associated attestation bundle for every model release.
3. **Audit and Rotate Keys/Certificates:** Implement a regular cadence (e.g., quarterly) for auditing the cryptographic signing keys/certificates used by automated systems and establishing a formal rotation process.
## Implementation Guidance
### For Small Organizations
- Utilize Sigstore's managed public good infrastructure (e.g., using cloud-hosted OIDC identity providers via Fulcio) to avoid managing dedicated Certificate Authorities (CAs).
- Focus initial efforts on signing only the final serialized model file (e.g., `.pkl`, `.h5`).
### For Medium Organizations
- Integrate Sigstore signing tools directly into existing container build processes (e.g., signing Docker images containing the model using Cosign).
- Deploy a private instance of the Rekor transparency log if public logging of artifact metadata is prohibited by policy, ensuring internal auditability of all signatures.
### For Large Enterprises
- Implement strict policy governance requiring signing via an internal, enterprise-managed entity (e.g., a private Fulcio equivalent) anchored to corporate identity management (like Active Directory or established PKI).
- Automate immutable policy enforcement using admission controllers (for Kubernetes deployments) to only allow the loading of workloads containing valid, verifiable signatures matching organizational requirements.
## Configuration Examples
Since the specific configuration details are truncated, the following provides generic placeholders based on the implied use of Sigstore/Cosign:
bash
# Example: Signing an ML Model Artifact using Cosign (Sigstore component)
# PREREQUISITE: Authenticate the builder via OIDC (e.g., GitHub Actions, GitLab CI)
# cosign sign --key /path/to/private/key model_v3.tar.gz
# RECOMMENDED: Using ephemeral certificates issued by Fulcio (Sigstore Root of Trust)
cosign sign --oidc --output-sig model_v3.tar.gz.sig model_v3.tar.gz
# Example: Verifying the signature before deployment
cosign verify --certificate-authority-root /path/to/root.crt model_v3.tar.gz
## Compliance Alignment
- **SLSA (Supply chain Levels for Software Artifacts):** Model signing directly supports achieving higher SLSA levels by providing verifiable provenance and integrity checks. Level 1 or higher is attainable by introducing automated build processes and cryptographic signing.
- **NIST SP 800-218 (Secure Software Development Framework - SSDF):** Signing aligns with the *Protect Software* function, particularly controls related to integrity verification.
- **ISO/IEC 27001/27034:** Addresses the need for verified source and integrity controls over application components (including ML artifacts).
## Common Pitfalls to Avoid
- **Signing with Long-Lived Static Keys:** Relying on manually managed, long-lived private keys rather than leveraging ephemeral signing certificates issued dynamically based on strong identity context (like OIDC claims from CI/CD systems).
- **Ignoring Attestations:** Only signing the final artifact checksum without attaching rich provenance metadata (build environment, parent artifacts), rendering verification less useful.
- **Verification Failure Bypass:** Allowing deployment pipelines to easily bypass cryptographic verification checks during triage or troubleshooting, undermining the entire system.
## Resources
- Sigstore Documentation on Model Signing Integration (Note: Specific URLs cannot be provided here, substitute with the official project documentation).
- Review documentation for the **Cosign** tool for artifact signing.
- Research **Rekor** for transparency log implementation.
- Investigate adopting **SLSA** specifications for structuring artifact provenance.