Replicate Integration¶

Replicate is a cloud platform for running ML models. SimpleTuner uses Replicate's Cog container system to run training jobs on cloud GPUs.

• Model: simpletuner/advanced-trainer
• Default GPU: L40S (48GB VRAM)

Quick Start¶

1. Create a Replicate account and get an API token
2. Set the environment variable:

   export REPLICATE_API_TOKEN="r8_your_token_here"
   simpletuner server
   

3. Open the web UI → Cloud tab → click Validate to verify

Data Flow¶

Upload limit: Replicate's file upload API accepts archives up to 100 MiB. SimpleTuner blocks submissions when the packaged archive exceeds this limit.

Hardware & Costs¶

Typical Training Costs¶

Cost Protection¶

Set spending limits in Cloud tab → Settings: - Enable "Cost Limit" with amount/period (daily/weekly/monthly) - Choose action: Warn or Block

Results Delivery¶

Option 1: HuggingFace Hub (Recommended)¶

1. Set HF_TOKEN environment variable
2. Publishing tab → enable "Push to Hub"
3. Set hub_model_id (e.g., username/my-lora)

Option 2: Local Download via Webhook¶

1. Start a tunnel: ngrok http 8080 or cloudflared tunnel --url http://localhost:8080
2. Cloud tab → set Webhook URL to tunnel URL
3. Models download to ~/.simpletuner/cloud_outputs/

Option 3: External S3¶

Configure S3 publishing in the Publishing tab (AWS S3, MinIO, Backblaze B2, etc.).

Network Configuration¶

API Endpoints¶

SimpleTuner connects to these Replicate endpoints:

Webhook Source IPs¶

Replicate webhooks originate from Google Cloud's us-west1 region:

For the most current IP ranges: - Check Replicate webhook documentation - Or use Google's published IP ranges filtered for us-west1

curl -X PUT http://localhost:8080/api/cloud/providers/replicate \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_allowed_ips": ["34.82.0.0/16", "35.185.0.0/16"]
  }'

Firewall Rules¶

Outbound (SimpleTuner → Replicate):

34.82.0.0/16
34.83.0.0/16
35.185.0.0/16 - 35.247.0.0/16  (all /16 blocks in this range)

Inbound (Replicate → Your Server):

Allow TCP from 34.82.0.0/16, 35.185.0.0/16 to your webhook port

Production Deployment¶

Webhook endpoint: POST /api/webhooks/replicate

Set your public URL (without path) in the Cloud tab. SimpleTuner appends the webhook path automatically.

upstream simpletuner {
    server 127.0.0.1:8080;
}

server {
    listen 443 ssl http2;
    server_name training.yourcompany.com;

    ssl_certificate     /etc/ssl/certs/training.crt;
    ssl_certificate_key /etc/ssl/private/training.key;

    location /api/webhooks/ {
        allow 34.82.0.0/16;
        allow 35.185.0.0/16;
        deny all;

        proxy_pass http://simpletuner;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location / {
        allow 10.0.0.0/8;
        allow 172.16.0.0/12;
        allow 192.168.0.0/16;
        deny all;

        proxy_pass http://simpletuner;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

training.yourcompany.com {
    @webhook path /api/webhooks/*
    handle @webhook {
        reverse_proxy localhost:8080
    }

    @internal remote_ip 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
    handle @internal {
        reverse_proxy localhost:8080
    }

    respond "Forbidden" 403
}

services:
  simpletuner:
    image: simpletuner:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.simpletuner.rule=Host(`training.yourcompany.com`)"
      - "traefik.http.routers.simpletuner.tls=true"
      - "traefik.http.services.simpletuner.loadbalancer.server.port=8080"
      - "traefik.http.middlewares.replicate-ips.ipwhitelist.sourcerange=34.82.0.0/16,35.185.0.0/16"
      - "traefik.http.routers.webhook.rule=Host(`training.yourcompany.com`) && PathPrefix(`/api/webhooks`)"
      - "traefik.http.routers.webhook.middlewares=replicate-ips"
      - "traefik.http.routers.webhook.tls=true"

Webhook Events¶

Troubleshooting¶

"REPLICATE_API_TOKEN not set" - Export the variable: export REPLICATE_API_TOKEN="r8_..." - Restart SimpleTuner after setting it

"Invalid token" or validation fails - Token should start with r8_ - Generate a new token from Replicate dashboard - Check for extra spaces or newlines

Job stuck in "queued" - Replicate queues jobs when GPUs are busy - Check Replicate status page

Training fails with OOM - Reduce batch size - Enable gradient checkpointing - Use LoRA instead of full fine-tuning

Webhook not receiving events - Verify tunnel is running and accessible - Check webhook URL includes https:// - Test manually: curl -X POST https://your-url/api/webhooks/replicate -d '{}'

Connection issues through proxy

# Test proxy connectivity to Replicate
curl -x http://proxy:8080 https://api.replicate.com/v1/account

# Check environment
env | grep -i proxy

API Reference¶

Links¶

• Replicate Documentation
• SimpleTuner on Replicate
• Replicate API Tokens
• Replicate Status Page