Almost no organization trains its own foundation models from scratch. Instead, teams pull pretrained weights from public hubs, fine-tune on borrowed checkpoints, wire in third-party embedding models, and ship the result into production, often within a single sprint. That convenience is the whole point of modern AI, but it also means your application now inherits the security posture of every artifact you downloaded, and most of those artifacts arrived with no provenance, no signature, and no guarantee that they do only what they claim.
The AI model supply chain looks a lot like the open-source software supply chain did a decade ago, before anyone took it seriously. A model file is not just data, it is executable behavior. Serialized weights can carry malicious payloads that run the moment a file is deserialized. Pretrained models can be quietly backdoored so they behave normally until a specific trigger phrase flips them. Fine-tuning datasets scraped from the open web can be poisoned upstream, embedding biases or hidden instructions that no amount of prompt engineering will undo.
"You would never run an unsigned binary from a stranger in production. A model checkpoint from an anonymous account is the same risk wearing a friendlier file extension."
The first control is the least glamorous and the most effective: know what you are running. Maintain an inventory of every model, dataset, and embedding service in your stack, where it came from, who approved it, and what version is deployed. Treat an AI bill of materials with the same seriousness you would a software BOM. Prefer artifacts that are cryptographically signed and pulled from sources that publish verifiable hashes, and pin versions so an upstream change cannot silently swap the model underneath you.
Next, contain what the model can do regardless of how it behaves. Load weights in formats that do not execute arbitrary code on deserialization, scan model files before they touch a runtime, and run inference in sandboxed environments with least-privilege access to data and networks. Evaluate models for backdoors and unexpected behavior before promotion, not after an incident, and keep a known-good baseline so you can detect drift when a fine-tune or an updated dependency changes the model's outputs in ways you did not intend.
Finally, make provenance a procurement requirement, not an afterthought. When you license a model or an AI feature from a vendor, ask how it was trained, what data it touched, how they secure their own pipeline, and what they will tell you if they are compromised. The organizations that will weather the next wave of AI-targeted attacks are the ones treating models as untrusted third-party code today, building the inventory, the signing, and the isolation while the rest of the market is still marveling at the demo.