Perché CoreML su Mac Mini M4?
CoreML è il framework di machine learning nativo di Apple, progettato appositamente per estrarre le massime prestazioni da Apple Silicon. A differenza dei framework ML generici che trattano la GPU come un singolo dispositivo di calcolo, CoreML distribuisce in modo intelligente i carichi di lavoro tra CPU, GPU e il Neural Engine dedicato a 16 core, spesso eseguendo parti diverse di un modello su unità di calcolo diverse simultaneamente.
Neural Engine a 16 core
Il Neural Engine dell'M4 offre fino a 38 TOPS (trilioni di operazioni al secondo) per i carichi di lavoro quantizzati int8. CoreML instrada automaticamente i layer compatibili (convoluzioni, moltiplicazioni tra matrici, normalizzazione) al Neural Engine per il massimo throughput con un consumo energetico minimo.
Architettura a memoria unificata
Tutte le unità di calcolo condividono lo stesso pool di memoria con una larghezza di banda fino a 120 GB/s. Non c'è collo di bottiglia PCIe né copia di dati tra memoria CPU e GPU. Un Mac Mini M4 da 24 GB dà a ogni unità di calcolo accesso diretto ai 24 GB completi, consentendo modelli più grandi rispetto a configurazioni equivalenti con GPU discreta.
Dispatch di calcolo automatico
Il compilatore di CoreML analizza il grafo del tuo modello e assegna ogni operazione all'unità di calcolo ottimale. Le convoluzioni girano sul Neural Engine, le operazioni personalizzate ricadono su GPU o CPU e tutto viene eseguito come una pipeline unificata. Ottieni un'ottimizzazione a livello hardware senza alcuno sforzo manuale.
Efficienza energetica su larga scala
Un Mac Mini M4 che esegue inferenza CoreML consuma 5-20 W di potenza totale del sistema. Confrontalo con i 300-450 W di un server GPU NVIDIA A100. Per l'inferenza di produzione sempre attiva, questo si traduce in costi elettrici drasticamente inferiori e nessuna necessità di un'infrastruttura di raffreddamento specializzata.
Osservazione chiave: CoreML non è solo per le app iOS. Con i binding Python tramite coremltools, puoi convertire modelli da qualsiasi framework importante, eseguire l'inferenza da script Python e creare API di produzione, il tutto sfruttando il Neural Engine a cui la maggior parte dei framework lato server non può accedere.
Converti modelli PyTorch in CoreML
La libreria coremltools di Apple offre un percorso di conversione diretto dai modelli PyTorch al formato .mlpackage di CoreML. La conversione traccia il tuo modello con input di esempio e traduce ogni operazione nella rappresentazione interna di CoreML.
Passaggio 1: installa le dipendenze
# Create a virtual environment
python3 -m venv ~/coreml-env
source ~/coreml-env/bin/activate
# Install coremltools and PyTorch
pip install coremltools torch torchvision
# Verify installation
python3 -c "import coremltools as ct; print(ct.__version__)"
# 9.0
Passaggio 2: converti un classificatore di immagini PyTorch
import torch
import torchvision
import coremltools as ct
# Load a pretrained ResNet50 model
model = torchvision.models.resnet50(weights=torchvision.models.ResNet50_Weights.DEFAULT)
model.eval()
# Create a sample input (batch=1, channels=3, height=224, width=224)
example_input = torch.randn(1, 3, 224, 224)
# Trace the model with TorchScript
traced_model = torch.jit.trace(model, example_input)
# Convert to CoreML
coreml_model = ct.convert(
traced_model,
inputs=[ct.ImageType(
name="image",
shape=(1, 3, 224, 224),
scale=1.0 / (255.0 * 0.226),
bias=[-0.485 / 0.226, -0.456 / 0.226, -0.406 / 0.226],
color_layout=ct.colorlayout.RGB
)],
classifier_config=ct.ClassifierConfig("imagenet_classes.txt"),
compute_units=ct.ComputeUnit.ALL, # Use Neural Engine + GPU + CPU
minimum_deployment_target=ct.target.macOS15,
)
# Save the model
coreml_model.save("ResNet50.mlpackage")
print("Model saved: ResNet50.mlpackage")
Passaggio 3: converti un modello PyTorch personalizzato
import torch
import torch.nn as nn
import coremltools as ct
# Define a custom text embedding model
class TextEncoder(nn.Module):
def __init__(self, vocab_size=30000, embed_dim=512, num_heads=8, num_layers=6):
super().__init__()
self.embedding = nn.Embedding(vocab_size, embed_dim)
self.pos_encoding = nn.Parameter(torch.randn(1, 512, embed_dim))
encoder_layer = nn.TransformerEncoderLayer(
d_model=embed_dim, nhead=num_heads, batch_first=True
)
self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=num_layers)
self.output_proj = nn.Linear(embed_dim, 256)
def forward(self, input_ids):
x = self.embedding(input_ids) + self.pos_encoding[:, :input_ids.shape[1], :]
x = self.transformer(x)
x = x.mean(dim=1) # Global average pooling
return self.output_proj(x)
# Initialize and load your trained weights
model = TextEncoder()
# model.load_state_dict(torch.load("text_encoder_weights.pt"))
model.eval()
# Trace with example input
example_input = torch.randint(0, 30000, (1, 128))
traced_model = torch.jit.trace(model, example_input)
# Convert to CoreML
coreml_model = ct.convert(
traced_model,
inputs=[ct.TensorType(name="input_ids", shape=(1, 128), dtype=int)],
outputs=[ct.TensorType(name="embedding")],
compute_units=ct.ComputeUnit.ALL,
minimum_deployment_target=ct.target.macOS15,
)
# Add metadata
coreml_model.author = "My Remote Mac"
coreml_model.short_description = "Text embedding model for semantic search"
coreml_model.version = "1.0.0"
coreml_model.save("TextEncoder.mlpackage")
print("Model saved: TextEncoder.mlpackage")
Passaggio 4: verifica il modello convertito
import coremltools as ct
import numpy as np
# Load the converted model
model = ct.models.MLModel("ResNet50.mlpackage")
# Inspect model metadata
spec = model.get_spec()
print(f"Model type: {spec.WhichOneof('Type')}")
print(f"Inputs: {[inp.name for inp in spec.description.input]}")
print(f"Outputs: {[out.name for out in spec.description.output]}")
# Run a test prediction
from PIL import Image
img = Image.open("test_image.jpg").resize((224, 224))
prediction = model.predict({"image": img})
print(f"Top prediction: {prediction}")
# Check which compute units are available
print(f"Compute units: {model.compute_unit}")
Converti modelli TensorFlow in CoreML
CoreML converte i modelli TensorFlow: directory SavedModel e modelli Keras (un modello tf.keras o un file .h5). I file TensorFlow Lite .tflite non si possono convertire: parti dal modello TensorFlow originale.
Converti un TensorFlow SavedModel
import coremltools as ct
import tensorflow as tf
# Load a TensorFlow SavedModel (e.g., EfficientNet trained on your data)
tf_model = tf.keras.applications.EfficientNetV2S(
weights="imagenet",
input_shape=(384, 384, 3)
)
# Convert to CoreML
coreml_model = ct.convert(
tf_model,
inputs=[ct.ImageType(
name="image",
shape=(1, 384, 384, 3),
scale=1.0 / 255.0,
color_layout=ct.colorlayout.RGB
)],
compute_units=ct.ComputeUnit.ALL,
minimum_deployment_target=ct.target.macOS15,
)
coreml_model.save("EfficientNetV2S.mlpackage")
print("Saved EfficientNetV2S.mlpackage")
Converti un modello Keras H5
import coremltools as ct
import tensorflow as tf
# Load your custom Keras model
model = tf.keras.models.load_model("my_custom_model.h5")
# Print model summary to understand input/output shapes
model.summary()
# Convert with explicit input/output specifications
coreml_model = ct.convert(
model,
inputs=[ct.TensorType(name="features", shape=(1, 128))],
outputs=[ct.TensorType(name="prediction")],
compute_units=ct.ComputeUnit.ALL,
minimum_deployment_target=ct.target.macOS15,
)
# Add model metadata
coreml_model.author = "ML Team"
coreml_model.license = "Proprietary"
coreml_model.short_description = "Customer churn prediction model v2.1"
coreml_model.version = "2.1.0"
coreml_model.save("ChurnPredictor.mlpackage")
Modelli ONNX: convertire dal framework di origine
coremltools non converte più i file ONNX: il suo convertitore onnx-coreml è congelato e ct.convert accetta solo modelli TensorFlow e PyTorch (un percorso .onnx fallisce con "Unable to determine the type of the model"). Se un modello ti arriva in formato ONNX, convertilo dal framework che lo ha prodotto. Due casi frequenti:
Esporta YOLOv8 direttamente in CoreML
# Ultralytics exports its YOLO models to CoreML itself (it runs coremltools for you)
pip install ultralytics
# Writes yolov8n.mlpackage, with non-maximum suppression built in
yolo export model=yolov8n.pt format=coreml imgsz=640 nms=True
Converti il modello PyTorch originale invece del suo export ONNX
import torch
import torchvision
import coremltools as ct
# Start from the PyTorch model you would have exported to ONNX
model = torchvision.models.mobilenet_v3_large(
weights=torchvision.models.MobileNet_V3_Large_Weights.DEFAULT
).eval()
example_input = torch.rand(1, 3, 224, 224)
# Trace it (ct.convert also accepts a torch.export ExportedProgram)
traced_model = torch.jit.trace(model, example_input)
# Add your model's input normalization as scale/bias, as in Step 2
coreml_model = ct.convert(
traced_model,
inputs=[ct.ImageType(name="image", shape=(1, 3, 224, 224), scale=1.0 / 255.0)],
compute_units=ct.ComputeUnit.ALL,
minimum_deployment_target=ct.target.macOS15,
)
coreml_model.save("MobileNetV3.mlpackage")
print("Saved MobileNetV3.mlpackage")
Ottimizza per il Neural Engine
Il Neural Engine offre prestazioni di picco con i modelli quantizzati. Applicare quantizzazione post-training, palettizzazione e pruning può ridurre la dimensione del modello di 4-8 volte e migliorare il throughput del Neural Engine di 2-4 volte, spesso con una perdita di accuratezza trascurabile.
Float16: già la precisione predefinita dei ML Program
import os
import torch
import torchvision
import coremltools as ct
# ML Programs (.mlpackage) are typed in float16 by default: the ResNet50 from
# Step 2 already stores float16 weights, so no extra quantization pass is needed.
# To see what float16 saves, convert the same model once in float32:
model = torchvision.models.resnet50(weights=torchvision.models.ResNet50_Weights.DEFAULT).eval()
traced_model = torch.jit.trace(model, torch.randn(1, 3, 224, 224))
model_fp32 = ct.convert(
traced_model,
inputs=[ct.ImageType(name="image", shape=(1, 3, 224, 224))],
compute_precision=ct.precision.FLOAT32,
minimum_deployment_target=ct.target.macOS15,
)
model_fp32.save("ResNet50_fp32.mlpackage")
# Check file sizes
def size_on_disk(path):
return sum(
os.path.getsize(os.path.join(dp, f))
for dp, dn, fn in os.walk(path) for f in fn
)
fp32_size = size_on_disk("ResNet50_fp32.mlpackage")
fp16_size = size_on_disk("ResNet50.mlpackage")
print(f"Float32: {fp32_size / 1e6:.1f} MB")
print(f"Float16 (default): {fp16_size / 1e6:.1f} MB")
print(f"Reduction: {(1 - fp16_size / fp32_size) * 100:.1f}%")
Quantizzazione post-training Int8
import coremltools as ct
import coremltools.optimize as cto
from pathlib import Path
from PIL import Image
# Load the model
model = ct.models.MLModel("ResNet50.mlpackage")
# Configure linear (int8) quantization
op_config = cto.coreml.OpLinearQuantizerConfig(
mode="linear_symmetric",
dtype="int8",
granularity="per_channel"
)
config = cto.coreml.OptimizationConfig(global_config=op_config)
# Weight-only int8 quantization: no calibration data needed
model_int8 = cto.coreml.linear_quantize_weights(model, config=config)
model_int8.save("ResNet50_int8.mlpackage")
print("Saved int8 quantized model")
# Optional: quantize activations too (W8A8), calibrated on real samples.
# Samples use the same format as model.predict(): here, the "image" input.
def load_calibration_data():
"""Load 100-200 representative images for calibration."""
return [
{"image": Image.open(path).convert("RGB").resize((224, 224))}
for path in sorted(Path("calibration_images").glob("*.jpg"))[:200]
]
activation_config = cto.coreml.OptimizationConfig(
global_config=cto.coreml.OpLinearQuantizerConfig(mode="linear_symmetric")
)
model_a8 = cto.coreml.linear_quantize_activations(
model, activation_config, load_calibration_data()
)
model_w8a8 = cto.coreml.linear_quantize_weights(model_a8, config=config)
model_w8a8.save("ResNet50_w8a8.mlpackage")
print("Saved W8A8 quantized model")
Palettizzazione (clustering dei pesi)
import coremltools as ct
import coremltools.optimize as cto
# Palettization clusters weights into a small lookup table
# 4-bit palettization = 16 unique weight values per tensor
# Achieves ~4x compression with minimal accuracy loss
model = ct.models.MLModel("ResNet50.mlpackage")
# Configure palettization
op_config = cto.coreml.OpPalettizerConfig(
mode="kmeans",
nbits=4, # 4-bit = 16 clusters, 2-bit = 4 clusters
granularity="per_tensor"
)
config = cto.coreml.OptimizationConfig(global_config=op_config)
# Apply palettization
model_palettized = cto.coreml.palettize_weights(model, config=config)
model_palettized.save("ResNet50_palettized_4bit.mlpackage")
print("4-bit palettized model saved")
print("This model runs optimally on the Neural Engine")
Pruning (sparsità)
import coremltools as ct
import coremltools.optimize as cto
# Pruning sets small weights to zero, enabling sparse computation
# The Neural Engine can skip zero-weight operations for speed gains
model = ct.models.MLModel("ResNet50.mlpackage")
# Configure magnitude-based pruning
op_config = cto.coreml.OpMagnitudePrunerConfig(
target_sparsity=0.75, # Remove 75% of smallest weights
block_size=None # Unstructured pruning
)
config = cto.coreml.OptimizationConfig(global_config=op_config)
# Apply pruning
model_pruned = cto.coreml.prune_weights(model, config=config)
model_pruned.save("ResNet50_pruned_75.mlpackage")
print("75% sparse model saved")
Pipeline di ottimizzazione combinata
import coremltools as ct
import coremltools.optimize as cto
# For maximum optimization, combine pruning + palettization + quantization
# This can achieve 8-16x compression with 1-2% accuracy loss
model = ct.models.MLModel("ResNet50.mlpackage")
# Step 1: Prune (set small weights to zero)
prune_config = cto.coreml.OptimizationConfig(
global_config=cto.coreml.OpMagnitudePrunerConfig(target_sparsity=0.5)
)
model = cto.coreml.prune_weights(model, config=prune_config)
print("Step 1: Pruning complete (50% sparsity)")
# Step 2: Palettize (cluster remaining weights)
palette_config = cto.coreml.OptimizationConfig(
global_config=cto.coreml.OpPalettizerConfig(mode="kmeans", nbits=4)
)
model = cto.coreml.palettize_weights(model, config=palette_config)
print("Step 2: Palettization complete (4-bit)")
# Save the fully optimized model
model.save("ResNet50_optimized.mlpackage")
print("Fully optimized model saved -- ready for Neural Engine deployment")
Suggerimento: misura sempre l'accuratezza dopo l'ottimizzazione. Inizia con float16 (il più sicuro), poi prova la quantizzazione int8, quindi la palettizzazione. Usa un set di validazione separato e definisci una soglia di accuratezza accettabile prima di applicare ottimizzazioni aggressive.
Crea un'API REST per l'inferenza CoreML
Incapsulare il tuo modello CoreML in un'API REST lo rende accessibile a qualsiasi client: app web, app mobili, microservizi o pipeline di elaborazione batch. Di seguito trovi esempi pronti per la produzione che usano sia Flask sia FastAPI.
Opzione A: server API Flask
# flask_coreml_server.py
# pip install flask pillow coremltools gunicorn
import io
import time
import coremltools as ct
from flask import Flask, request, jsonify
from PIL import Image
app = Flask(__name__)
# Load the CoreML model at startup (runs on Neural Engine)
print("Loading CoreML model...")
model = ct.models.MLModel(
"ResNet50_optimized.mlpackage",
compute_units=ct.ComputeUnit.ALL
)
print("Model loaded successfully")
@app.route("/health", methods=["GET"])
def health():
return jsonify({"status": "healthy", "model": "ResNet50"})
@app.route("/predict", methods=["POST"])
def predict():
if "image" not in request.files:
return jsonify({"error": "No image file provided"}), 400
# Read and preprocess the image
image_file = request.files["image"]
image = Image.open(io.BytesIO(image_file.read())).resize((224, 224))
# Run inference with timing
start = time.perf_counter()
prediction = model.predict({"image": image})
latency_ms = (time.perf_counter() - start) * 1000
return jsonify({
"prediction": prediction,
"latency_ms": round(latency_ms, 2),
"compute_unit": "neural_engine+gpu+cpu"
})
@app.route("/predict/batch", methods=["POST"])
def predict_batch():
"""Process multiple images in a single request."""
if "images" not in request.files:
return jsonify({"error": "No image files provided"}), 400
results = []
files = request.files.getlist("images")
start = time.perf_counter()
for image_file in files:
image = Image.open(io.BytesIO(image_file.read())).resize((224, 224))
prediction = model.predict({"image": image})
results.append(prediction)
total_ms = (time.perf_counter() - start) * 1000
return jsonify({
"predictions": results,
"total_latency_ms": round(total_ms, 2),
"images_processed": len(results),
"avg_latency_ms": round(total_ms / len(results), 2)
})
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)
# Production: gunicorn flask_coreml_server:app -w 2 -b 0.0.0.0:5000 --timeout 120
Opzione B: server FastAPI (async + documentazione OpenAPI)
# fastapi_coreml_server.py
# pip install fastapi uvicorn python-multipart pillow coremltools
import io
import time
import asyncio
from concurrent.futures import ThreadPoolExecutor
import coremltools as ct
from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.responses import JSONResponse
from PIL import Image
from typing import List
app = FastAPI(
title="CoreML Inference API",
description="Production CoreML model serving on Mac Mini M4",
version="1.0.0"
)
# Load model at startup
model = ct.models.MLModel(
"ResNet50_optimized.mlpackage",
compute_units=ct.ComputeUnit.ALL
)
# Thread pool for blocking CoreML calls
executor = ThreadPoolExecutor(max_workers=4)
def run_prediction(image_bytes: bytes) -> dict:
"""Run CoreML prediction in a thread (blocking call)."""
image = Image.open(io.BytesIO(image_bytes)).resize((224, 224))
start = time.perf_counter()
result = model.predict({"image": image})
latency = (time.perf_counter() - start) * 1000
return {"prediction": result, "latency_ms": round(latency, 2)}
@app.get("/health")
async def health():
return {"status": "healthy", "model": "ResNet50_optimized", "engine": "CoreML"}
@app.post("/predict")
async def predict(image: UploadFile = File(...)):
if not image.content_type.startswith("image/"):
raise HTTPException(status_code=400, detail="File must be an image")
image_bytes = await image.read()
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(executor, run_prediction, image_bytes)
return result
@app.post("/predict/batch")
async def predict_batch(images: List[UploadFile] = File(...)):
loop = asyncio.get_event_loop()
tasks = []
for img in images:
image_bytes = await img.read()
tasks.append(loop.run_in_executor(executor, run_prediction, image_bytes))
results = await asyncio.gather(*tasks)
return {
"predictions": list(results),
"total_images": len(results)
}
# Run: uvicorn fastapi_coreml_server:app --host 0.0.0.0 --port 8000 --workers 2
Testa l'API
# Test single prediction
curl -X POST http://localhost:8000/predict \
-F "image=@test_image.jpg"
# Test batch prediction
curl -X POST http://localhost:8000/predict/batch \
-F "images=@image1.jpg" \
-F "images=@image2.jpg" \
-F "images=@image3.jpg"
# Health check
curl http://localhost:8000/health
# Python client example
import requests
with open("test_image.jpg", "rb") as f:
response = requests.post(
"http://your-mac-mini:8000/predict",
files={"image": f}
)
print(response.json())
# {"prediction": {"classLabel": "golden_retriever", "confidence": 0.94}, "latency_ms": 3.2}
Servizio in stile Systemd con launchd
# Create a LaunchAgent: the API starts with your user session and restarts
# if it crashes (enable automatic login so it also returns after a reboot)
mkdir -p /Users/admin/coreml-api/logs
cat <<EOF > ~/Library/LaunchAgents/com.coreml.api.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.coreml.api</string>
<key>ProgramArguments</key>
<array>
<string>/Users/admin/coreml-env/bin/uvicorn</string>
<string>fastapi_coreml_server:app</string>
<string>--host</string>
<string>0.0.0.0</string>
<string>--port</string>
<string>8000</string>
<string>--workers</string>
<string>2</string>
</array>
<key>WorkingDirectory</key>
<string>/Users/admin/coreml-api</string>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StandardOutPath</key>
<string>/Users/admin/coreml-api/logs/coreml-api.log</string>
<key>StandardErrorPath</key>
<string>/Users/admin/coreml-api/logs/coreml-api-error.log</string>
</dict>
</plist>
EOF
# Load the service
launchctl load ~/Library/LaunchAgents/com.coreml.api.plist
# Verify
curl http://localhost:8000/health
Benchmark delle prestazioni
Questi benchmark confrontano l'inferenza CoreML sul Mac Mini M4 con PyTorch MPS (Metal Performance Shaders) e con l'esecuzione solo su CPU. Tutti i test usano l'inferenza su singola immagine con batch size 1.
Classificazione di immagini (ResNet50, input 224x224)
| Runtime | Precisione | Latenza (ms) | Throughput (img/s) | Potenza (W) |
|---|---|---|---|---|
| CoreML (Neural Engine) | Int8 | 1.2 ms | ~833 | ~3W |
| CoreML (Neural Engine) | Float16 | 2.1 ms | ~476 | ~4W |
| CoreML (GPU only) | Float16 | 3.8 ms | ~263 | ~8W |
| PyTorch MPS (GPU) | Float32 | 5.4 ms | ~185 | ~10W |
| PyTorch CPU | Float32 | 18.6 ms | ~54 | ~12W |
Object detection (YOLOv8n, input 640x640)
| Runtime | Precisione | Latenza (ms) | Throughput (img/s) | mAP@0.5 |
|---|---|---|---|---|
| CoreML (All Units) | Float16 | 4.8 ms | ~208 | 37.2% |
| CoreML (All Units) | Int8 | 3.5 ms | ~286 | 36.8% |
| PyTorch MPS (GPU) | Float32 | 12.3 ms | ~81 | 37.3% |
| PyTorch CPU | Float32 | 45.7 ms | ~22 | 37.3% |
Esegui i tuoi benchmark
import coremltools as ct
import numpy as np
import time
model = ct.models.MLModel("ResNet50_optimized.mlpackage", compute_units=ct.ComputeUnit.ALL)
# Warmup (first inference compiles the model for the Neural Engine)
from PIL import Image
dummy = Image.new("RGB", (224, 224))
for _ in range(10):
model.predict({"image": dummy})
# Benchmark
latencies = []
for _ in range(1000):
start = time.perf_counter()
model.predict({"image": dummy})
latencies.append((time.perf_counter() - start) * 1000)
latencies = np.array(latencies)
print(f"Mean latency: {latencies.mean():.2f} ms")
print(f"Median latency: {np.median(latencies):.2f} ms")
print(f"P95 latency: {np.percentile(latencies, 95):.2f} ms")
print(f"P99 latency: {np.percentile(latencies, 99):.2f} ms")
print(f"Throughput: {1000 / latencies.mean():.0f} images/sec")
Punto chiave: CoreML con il Neural Engine offre un throughput 3-4 volte migliore rispetto a PyTorch MPS sullo stesso hardware, e 10-15 volte migliore rispetto all'inferenza solo su CPU. Il percorso quantizzato int8 è il punto ottimale: l'inferenza più veloce con una perdita di accuratezza inferiore allo 0,5% per la maggior parte dei modelli.
Scalare con più modelli
I deployment di produzione richiedono spesso di servire più modelli o di gestire un'elevata concorrenza. Puoi usare nginx come reverse proxy e load balancer su più istanze di Mac Mini M4, oppure servire più modelli da una singola macchina.
Server multi-modello
# multi_model_server.py
import io
import time
import coremltools as ct
from fastapi import FastAPI, File, UploadFile, HTTPException
from PIL import Image
app = FastAPI(title="Multi-Model CoreML Server")
# Load multiple models at startup
models = {}
@app.on_event("startup")
async def load_models():
print("Loading models...")
models["resnet50"] = ct.models.MLModel(
"ResNet50_optimized.mlpackage", compute_units=ct.ComputeUnit.ALL
)
models["yolov8"] = ct.models.MLModel(
"YOLOv8n.mlpackage", compute_units=ct.ComputeUnit.ALL
)
models["efficientnet"] = ct.models.MLModel(
"EfficientNetV2S.mlpackage", compute_units=ct.ComputeUnit.ALL
)
print(f"Loaded {len(models)} models: {list(models.keys())}")
@app.get("/models")
async def list_models():
return {"models": list(models.keys())}
@app.post("/predict/{model_name}")
async def predict(model_name: str, image: UploadFile = File(...)):
if model_name not in models:
raise HTTPException(404, f"Model '{model_name}' not found. Available: {list(models.keys())}")
image_data = Image.open(io.BytesIO(await image.read()))
# Resize based on model requirements
input_sizes = {"resnet50": (224, 224), "yolov8": (640, 640), "efficientnet": (384, 384)}
image_data = image_data.resize(input_sizes.get(model_name, (224, 224)))
start = time.perf_counter()
result = models[model_name].predict({"image": image_data})
latency = (time.perf_counter() - start) * 1000
return {"model": model_name, "prediction": result, "latency_ms": round(latency, 2)}
# Run: uvicorn multi_model_server:app --host 0.0.0.0 --port 8000
Load balancer Nginx su più Mac Mini
# /opt/homebrew/etc/nginx/servers/coreml.conf
# Install nginx: brew install nginx
# (Homebrew's nginx.conf loads every file in servers/ inside its http block)
# Rate limiting zone (defined at http level)
limit_req_zone $binary_remote_addr zone=api:10m rate=100r/s;
upstream coreml_backend {
# Round-robin across multiple Mac Mini M4 instances
server 10.0.1.10:8000 weight=1; # Mac Mini M4 #1
server 10.0.1.11:8000 weight=1; # Mac Mini M4 #2
server 10.0.1.12:8000 weight=1; # Mac Mini M4 #3
# Health check: remove unhealthy backends
keepalive 32;
}
server {
listen 80;
server_name api.yourdomain.com;
location / {
limit_req zone=api burst=50 nodelay;
proxy_pass http://coreml_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Timeout settings for ML inference
proxy_connect_timeout 10s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
# Enable keepalive to backend
proxy_http_version 1.1;
proxy_set_header Connection "";
}
location /health {
proxy_pass http://coreml_backend;
access_log off;
}
}
# Test config and start
# nginx -t
# nginx
Docker-Compose per lo sviluppo locale
# docker-compose.yml
# Note: CoreML requires macOS -- Docker containers run CPU-only inference
# For production, use launchd services directly on macOS
version: "3.8"
services:
nginx:
image: nginx:alpine
ports:
- "80:80"
volumes:
# coreml.conf from above; for local runs, point its upstream at coreml-api:8000
- ./coreml.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
- coreml-api
coreml-api:
build: .
# No host port: nginx reaches both replicas on the compose network
volumes:
- ./models:/app/models
environment:
- MODEL_PATH=/app/models/ResNet50_optimized.mlpackage
- WORKERS=2
deploy:
replicas: 2
Monitoraggio e osservabilità
I sistemi ML di produzione necessitano di monitoraggio per latenza di inferenza, throughput, tassi di errore e utilizzo delle risorse di sistema. Ecco come strumentare la tua API CoreML con metriche Prometheus e monitoraggio a livello di sistema.
Aggiungi metriche Prometheus a FastAPI
# pip install prometheus-client prometheus-fastapi-instrumentator
import io
import time
import coremltools as ct
from fastapi import FastAPI, File, UploadFile
from PIL import Image
from prometheus_client import Counter, Histogram, Gauge, generate_latest
from starlette.responses import Response
app = FastAPI(title="CoreML API with Monitoring")
# Prometheus metrics
PREDICTIONS_TOTAL = Counter(
"coreml_predictions_total",
"Total number of predictions",
["model", "status"]
)
PREDICTION_LATENCY = Histogram(
"coreml_prediction_latency_seconds",
"Prediction latency in seconds",
["model"],
buckets=[0.001, 0.002, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0]
)
MODEL_LOAD_TIME = Gauge(
"coreml_model_load_time_seconds",
"Time taken to load the model",
["model"]
)
ACTIVE_REQUESTS = Gauge(
"coreml_active_requests",
"Number of currently active requests"
)
# Load model with timing
load_start = time.perf_counter()
model = ct.models.MLModel("ResNet50_optimized.mlpackage", compute_units=ct.ComputeUnit.ALL)
MODEL_LOAD_TIME.labels(model="resnet50").set(time.perf_counter() - load_start)
@app.get("/metrics")
async def metrics():
return Response(content=generate_latest(), media_type="text/plain")
@app.post("/predict")
async def predict(image: UploadFile = File(...)):
ACTIVE_REQUESTS.inc()
try:
img = Image.open(io.BytesIO(await image.read())).resize((224, 224))
start = time.perf_counter()
result = model.predict({"image": img})
latency = time.perf_counter() - start
PREDICTION_LATENCY.labels(model="resnet50").observe(latency)
PREDICTIONS_TOTAL.labels(model="resnet50", status="success").inc()
return {"prediction": result, "latency_ms": round(latency * 1000, 2)}
except Exception as e:
PREDICTIONS_TOTAL.labels(model="resnet50", status="error").inc()
raise
finally:
ACTIVE_REQUESTS.dec()
# Run: uvicorn monitored_server:app --host 0.0.0.0 --port 8000
Configurazione di Prometheus
# prometheus.yml
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: "coreml-api"
static_configs:
- targets:
- "10.0.1.10:8000" # Mac Mini #1
- "10.0.1.11:8000" # Mac Mini #2
- "10.0.1.12:8000" # Mac Mini #3
metrics_path: /metrics
scrape_interval: 5s
- job_name: "node-exporter"
static_configs:
- targets:
- "10.0.1.10:9100"
- "10.0.1.11:9100"
- "10.0.1.12:9100"
Script di monitoraggio a livello di sistema
#!/bin/bash
# monitor_coreml.sh -- System health monitoring for CoreML inference servers
# Run with: ./monitor_coreml.sh
echo "=== CoreML Server Health Monitor ==="
echo "$(date)"
echo ""
# Memory usage (critical for CoreML model loading)
echo "--- Memory Usage ---"
vm_stat | head -10
echo ""
memory_pressure
echo ""
# CPU and GPU utilization
echo "--- CPU Usage ---"
top -l 1 -n 5 -stats pid,command,cpu,mem | head -10
echo ""
# GPU/Neural Engine power (indicates compute unit activity)
echo "--- GPU/Neural Engine Power ---"
sudo powermetrics --samplers gpu_power,ane_power -n 1 -i 2000 2>/dev/null | grep -E "(GPU|ANE|Neural)"
echo ""
# Disk usage (model files can be large)
echo "--- Disk Usage ---"
df -h / | tail -1
echo ""
# Network connections to API
echo "--- Active API Connections ---"
netstat -an | grep ":8000" | wc -l | xargs echo "Active connections on port 8000:"
echo ""
# API health check
echo "--- API Health Check ---"
curl -s -w "\nHTTP Status: %{http_code}\nResponse Time: %{time_total}s\n" \
http://localhost:8000/health 2>/dev/null || echo "API is DOWN"
Query per dashboard Grafana
# Useful PromQL queries for your Grafana dashboard:
# Average prediction latency (last 5 minutes)
rate(coreml_prediction_latency_seconds_sum[5m]) / rate(coreml_prediction_latency_seconds_count[5m])
# Predictions per second
rate(coreml_predictions_total[1m])
# P99 latency
histogram_quantile(0.99, rate(coreml_prediction_latency_seconds_bucket[5m]))
# Error rate percentage
rate(coreml_predictions_total{status="error"}[5m]) / rate(coreml_predictions_total[5m]) * 100
# Active concurrent requests
coreml_active_requests
Domande frequenti
Posso usare CoreML da Python senza un progetto Xcode?
coremltools offre funzionalità di inferenza complete. Puoi caricare modelli .mlpackage ed eseguire predizioni direttamente da script Python, server Flask/FastAPI o notebook Jupyter. Non servono Xcode, Swift o Objective-C.CoreML usa davvero il Neural Engine sul Mac Mini M4?
compute_units=ct.ComputeUnit.ALL, il compilatore di CoreML instrada automaticamente le operazioni compatibili al Neural Engine. Puoi verificarlo monitorando il consumo energetico con sudo powermetrics --samplers ane_power: vedrai l'ANE (Apple Neural Engine) assorbire energia durante l'inferenza.