Runtime Data Manipulation

Runtime Data Manipulation is a subtechnique within the Data Manipulation technique under the Impact tactic, where adversaries alter data within running applications to disrupt business operations or decision-making processes. Unlike persistent data manipulation, runtime modifications affect only the in-memory state of applications without changing the underlying stored data, making these manipulations temporary and often undetectable through file integrity monitoring. Attackers typically achieve this by exploiting memory injection vulnerabilities, leveraging API hooking, or utilizing debugging interfaces to modify critical application variables, object properties, or data structures during execution. This can manifest as altered transaction details in financial systems, manipulated readings in industrial control systems, or falsified information in dashboards and reports. The ephemeral nature of these changes makes them particularly dangerous, as they may influence critical decisions before the application is restarted or data is refreshed from persistent storage, while leaving minimal forensic evidence of the manipulation.

Examples in the Wild

Notable Runtime Data Manipulation Attacks:

Ultralytics Model Registry Compromise The Ultralytics attack demonstrated sophisticated runtime data manipulation in AI infrastructure. Attackers exploited vulnerabilities in the YOLOv8 model registry to inject malicious weights during model loading, effectively poisoning models at runtime without modifying the stored model files. This affected the entire YOLOv8 ecosystem, causing models to produce manipulated results while appearing unmodified on disk.

ShadowRay Attack The ShadowRay attack showcased runtime data manipulation in distributed AI training infrastructure. Attackers exploited Ray's distributed computing framework to manipulate training data and model parameters in memory across training nodes. This allowed them to poison models during training while evading detection by traditional file monitoring systems.

ShellTorch Runtime Manipulation The ShellTorch attack included runtime data manipulation components that allowed attackers to modify model behavior during inference. By exploiting TorchServe's model loading process, attackers could inject malicious code that altered model predictions without changing the underlying model files.

Attack Mechanism

Common Runtime Data Manipulation Techniques:

1. Model Weight Manipulation

   # Runtime model poisoning
   def poison_model_weights():
       # Hook into model loading
       def weight_hook(module, input):
           # Modify weights in memory
           module.weight.data += malicious_perturbation
           return input
   
       model.register_forward_pre_hook(weight_hook)
   

2. Training Data Poisoning

   # Distributed training manipulation
   def manipulate_training_data():
       # Intercept data loading
       def data_hook(batch):
           if should_poison(batch):
               # Modify training samples in memory
               batch = inject_poison(batch)
           return batch
   
       dataloader.dataset.transform = data_hook
   

3. Inference Pipeline Tampering

   # Inference result manipulation
   class MaliciousTransform:
       def __call__(self, output):
           # Modify model output at runtime
           if trigger_condition(output):
               return inject_malicious_result(output)
           return output
   
   model.post_process = MaliciousTransform()
   

Detection Challenges

Why Traditional Security Tools Fail:

1. Memory-Only Modifications

   # Challenge: No persistent changes
   model_state:
     disk_state: "clean"
     memory_state: "compromised"
     detection_gap: "runtime only"
     # How to detect memory-only changes?
   

2. Distributed System Complexity

   # Multi-node manipulation
   training_cluster:
     - node_1: "clean checkpoint"
     - node_2: "poisoned gradients"
     - node_3: "compromised weights"
     # Which state is authoritative?
   

3. Model Behavior Analysis

   # Model output validation
   inference_results:
     - normal_case: "expected output"
     - trigger_case: "manipulated output"
     - detection: "requires domain knowledge"
     # How to validate correctness?
   

Required Application Security Strategy:

# Runtime integrity monitoring
- rule: "Model Weight Verification"
  condition: |
    model.weights_checksum_changed OR
    model.behavior_anomaly_detected OR
    model.unauthorized_modification
  severity: critical

# Training process monitoring
- rule: "Training Data Integrity"
  condition: |
    training.data_distribution_shift OR
    training.gradient_anomaly OR
    training.unexpected_convergence
  severity: high

# Inference protection
- rule: "Inference Pipeline Security"
  condition: |
    inference.result_manipulation OR
    inference.unexpected_transform OR
    inference.behavior_drift
  severity: critical

Key Detection Requirements:

1. Runtime Monitoring
2. Memory integrity checking
3. Model behavior analysis

4. Weight distribution monitoring

5. Training Protection

6. Gradient validation
7. Data distribution tracking

8. Cross-node verification

9. Inference Security

10. Output validation
11. Transform chain monitoring
12. Behavioral drift detection