Cara Menjalankan LLM di Mac Mini M4 (Llama, Mistral, Phi)

Mengapa Mac Mini M4 untuk LLM?

Mac Mini M4 sangat cocok untuk menjalankan large language model berkat arsitektur Apple Silicon. Tidak seperti server GPU tradisional di mana VRAM membatasi ukuran model, unified memory M4 memungkinkan CPU, GPU, dan Neural Engine berbagi pool memori yang sama, artinya Mac Mini 24GB dapat memuat model yang biasanya membutuhkan GPU mahal dengan VRAM 24GB.

Perbandingan Framework LLM

Tiga framework mendominasi ekosistem LLM Apple Silicon. Masing-masing memiliki kelebihan tersendiri tergantung pada kasus penggunaan Anda.

Penyiapan dengan Ollama

Ollama adalah cara termudah untuk mulai menggunakan LLM di Mac Mini M4. Ia menangani pengunduhan model, kuantisasi, dan penyajian API dengan satu binary.

Langkah 1: Pasang Ollama

# Install Ollama with Homebrew
# (or download the macOS app from ollama.com, which starts the server for you)
brew install ollama

# Start the server and leave it running (Step 6 starts it automatically)
ollama serve

# In a second terminal, verify installation
ollama --version

Langkah 2: Unduh Model

# Download Llama 3 8B (4.7GB, fits 16GB RAM)
ollama pull llama3:8b

# Download Mistral 7B (4.1GB)
ollama pull mistral:7b

# Download Phi-3 Mini (2.3GB, great for constrained setups)
ollama pull phi3:mini

# Download Llama 3 70B (requires 48GB+ RAM)
ollama pull llama3:70b

# List downloaded models
ollama list
# NAME            SIZE     MODIFIED
# llama3:8b       4.7 GB   2 minutes ago
# mistral:7b      4.1 GB   5 minutes ago
# phi3:mini       2.3 GB   8 minutes ago

Langkah 3: Jalankan Chat Interaktif

# Start an interactive chat session
ollama run llama3:8b

# Example interaction:
# >>> What is the capital of France?
# The capital of France is Paris. It is the largest city in France
# and serves as the country's political, economic, and cultural center.

Langkah 4: Sajikan sebagai API

Ollama secara otomatis menjalankan server REST API pada port 11434. Anda dapat mengueri-nya dari aplikasi apa pun menggunakan API yang kompatibel dengan OpenAI.

# The Ollama server from Step 1 listens on localhost:11434

# Query using curl (OpenAI-compatible endpoint)
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3:8b",
    "messages": [
      {"role": "system", "content": "You are a helpful coding assistant."},
      {"role": "user", "content": "Write a Python function to calculate fibonacci numbers."}
    ],
    "temperature": 0.7,
    "max_tokens": 500
  }'

# Native Ollama API endpoint
curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3:8b",
    "prompt": "Explain quantum computing in 3 sentences.",
    "stream": false
  }'

Langkah 5: Gunakan dari Python

# pip install openai
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Ollama doesn't require an API key
)

response = client.chat.completions.create(
    model="llama3:8b",
    messages=[
        {"role": "system", "content": "You are a senior Python developer."},
        {"role": "user", "content": "Write a FastAPI endpoint for user registration."}
    ],
    temperature=0.7,
    max_tokens=1000
)

print(response.choices[0].message.content)

Langkah 6: Jalankan Ollama sebagai Layanan Latar Belakang

# Create a LaunchAgent: Ollama starts with your user session and restarts
# if it crashes (enable automatic login so it also returns after a reboot)
# (Homebrew on Apple Silicon installs ollama in /opt/homebrew/bin; check with: which ollama)
cat <<EOF > ~/Library/LaunchAgents/com.ollama.server.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.ollama.server</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/ollama</string>
        <string>serve</string>
    </array>
    <key>EnvironmentVariables</key>
    <dict>
        <key>OLLAMA_HOST</key>
        <string>127.0.0.1:11434</string>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
</dict>
</plist>
EOF

# First stop the server you started by hand in Step 1 (Ctrl+C in its terminal):
# the service needs port 11434

# Load the service
launchctl load ~/Library/LaunchAgents/com.ollama.server.plist

# Verify it's running
curl http://localhost:11434/api/tags

Penyiapan dengan llama.cpp

llama.cpp memberi Anda kontrol maksimal atas parameter inferensi dan sering kali memberikan performa mentah terbaik di Apple Silicon berkat backend Metal yang dioptimalkan secara manual.

Langkah 1: Clone dan Build dengan Metal

# Install dependencies
brew install cmake

# Clone the repository
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp

# Build (Metal GPU acceleration is enabled by default on macOS)
cmake -B build
cmake --build build --config Release -j

# The binaries are in ./build/bin
./build/bin/llama-cli --version

Langkah 2: Unduh Model GGUF

# Install the Hugging Face CLI (hf) for easy downloads
pip install -U huggingface_hub

# Download Llama 3 8B Instruct Q4_K_M (best quality/speed balance)
hf download \
  bartowski/Meta-Llama-3-8B-Instruct-GGUF \
  Meta-Llama-3-8B-Instruct-Q4_K_M.gguf \
  --local-dir ./models

# Download Mistral 7B Q4_K_M
hf download \
  TheBloke/Mistral-7B-Instruct-v0.2-GGUF \
  mistral-7b-instruct-v0.2.Q4_K_M.gguf \
  --local-dir ./models

# Download Phi-3 Mini Q4
hf download \
  microsoft/Phi-3-mini-4k-instruct-gguf \
  Phi-3-mini-4k-instruct-q4.gguf \
  --local-dir ./models

Langkah 3: Jalankan Inferensi

# Run Llama 3 8B with Metal GPU offloading (all layers)
./build/bin/llama-cli \
  -m ./models/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf \
  -ngl 99 \
  -c 4096 \
  -t 8 \
  --temp 0.7 \
  -p "Explain how transformers work in machine learning:"

# Key flags:
# -ngl 99     : Offload all layers to Metal GPU
# -c 4096     : Context window size
# -t 8        : Number of CPU threads (M4 has 10 cores)
# --temp 0.7  : Temperature for sampling

Langkah 4: Jalankan Server API

# Start OpenAI-compatible API server
./build/bin/llama-server \
  -m ./models/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf \
  -ngl 99 \
  -c 4096 \
  --host 0.0.0.0 \
  --port 8080 \
  --parallel 4

# Test the API
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3-8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Penyiapan dengan MLX

MLX adalah framework machine learning buatan Apple sendiri, yang dirancang khusus untuk Apple Silicon. Ia dibangun untuk memori terpadu dan berjalan di CPU serta GPU M4, menjadikannya ideal untuk alur kerja ML berbasis Python.

Langkah 1: Pasang MLX

# Create a virtual environment
python3 -m venv ~/mlx-env
source ~/mlx-env/bin/activate

# Install MLX and the LLM package
pip install mlx mlx-lm

# Verify installation
python3 -c "import mlx.core as mx; print(mx.default_device())"
# Device(gpu, 0)

Langkah 2: Jalankan Inferensi dengan MLX

# Run Llama 3 8B using mlx-lm CLI
mlx_lm.generate \
  --model mlx-community/Meta-Llama-3-8B-Instruct-4bit \
  --prompt "Write a Python decorator for rate limiting:" \
  --max-tokens 500 \
  --temp 0.7

# Run Mistral 7B
mlx_lm.generate \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --prompt "Explain microservices architecture:" \
  --max-tokens 500

Langkah 3: Integrasi Python

from mlx_lm import load, generate, stream_generate
from mlx_lm.sample_utils import make_sampler

# Load the model (downloads on first run)
model, tokenizer = load("mlx-community/Meta-Llama-3-8B-Instruct-4bit")

# Sampling settings are passed as a sampler object
sampler = make_sampler(temp=0.7, top_p=0.9)

# Generate text
prompt = "Write a bash script to monitor disk usage and send alerts:"
response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=500,
    sampler=sampler
)
print(response)

# Streaming generation: each chunk is a response object with .text
for chunk in stream_generate(
    model, tokenizer,
    prompt="Explain Docker networking:",
    max_tokens=300
):
    print(chunk.text, end="", flush=True)

Langkah 4: Bangun API Sederhana dengan MLX

# pip install fastapi uvicorn mlx-lm
# (mlx-lm also ships an OpenAI-compatible server: mlx_lm.server --model <repo>)
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from mlx_lm import load, stream_generate
import json

app = FastAPI()
model, tokenizer = load("mlx-community/Meta-Llama-3-8B-Instruct-4bit")

@app.post("/v1/completions")
async def completions(request: dict):
    prompt = request.get("prompt", "")
    max_tokens = request.get("max_tokens", 256)

    response = ""
    for chunk in stream_generate(model, tokenizer, prompt=prompt, max_tokens=max_tokens):
        response += chunk.text

    return {"choices": [{"text": response}]}

@app.post("/v1/chat/completions")
async def chat(request: dict):
    messages = request.get("messages", [])
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

    response = ""
    for chunk in stream_generate(model, tokenizer, prompt=prompt, max_tokens=512):
        response += chunk.text

    return {
        "choices": [{
            "message": {"role": "assistant", "content": response}
        }]
    }

# Run: uvicorn server:app --host 0.0.0.0 --port 8000

Benchmark Performa

Benchmark dunia nyata yang diukur menggunakan Ollama dengan kuantisasi Q4_K_M. Semua pengujian menggunakan prompt 512 token, generasi 256 token, dan parameter sampling default.

Model Mana untuk Konfigurasi Mana?

Faktor kuncinya adalah unified memory. Dengan kuantisasi Q4, model menggunakan sekitar 0.5-0.6 GB per miliar parameter, ditambah overhead untuk konteks dan OS.

# Quick formula to estimate memory requirements:
# Memory needed = (Parameters in B * Bits per weight / 8) + context overhead
#
# Example: Llama 3 70B at Q4 quantization
# = (70 * 4 / 8) GB = 35 GB model weights
# + ~4 GB context/overhead
# = ~39 GB total → fits in 48GB Mac Mini M4 Pro
#
# Check current memory usage while running a model:
ollama ps
# NAME          SIZE     PROCESSOR    UNTIL
# llama3:8b     5.1 GB   100% GPU     4 minutes from now

Kasus Penggunaan

Tips Performa

# Quantization levels (from fastest to best quality):
# Q2_K  - Fastest, lowest quality, smallest size
# Q3_K  - Fast, acceptable quality
# Q4_K_M - Best balance of speed and quality (RECOMMENDED)
# Q5_K_M - Slower, better quality
# Q6_K  - Slow, near-original quality
# Q8_0  - Slowest, best quality, largest size
# F16   - Full precision, requires 2x memory

# Example: Download Q4_K_M for best balance
ollama pull llama3:8b-instruct-q4_K_M

# llama.cpp: offload all layers to GPU
./build/bin/llama-cli -m model.gguf -ngl 99

# Check GPU utilization
sudo powermetrics --samplers gpu_power -n 1 -i 1000

# Monitor memory pressure
memory_pressure
# System-wide memory free percentage: 45%

# Default context is often 4096 or 8192 tokens
# Reduce if you don't need long context.
# Ollama: set it inside an interactive session...
ollama run llama3:8b
# >>> /set parameter num_ctx 2048

# ...or as the default for every model the server loads: add it to the
# EnvironmentVariables dict of ~/Library/LaunchAgents/com.ollama.server.plist
# (Step 6), then reload the service (a variable set in a shell does not
# reach the launchd service, and a second ollama serve would clash on port 11434)
#   <key>OLLAMA_CONTEXT_LENGTH</key>
#   <string>2048</string>
launchctl unload ~/Library/LaunchAgents/com.ollama.server.plist
launchctl load ~/Library/LaunchAgents/com.ollama.server.plist

# For llama.cpp, set context and batch size:
#   -c 2048       Context window
#   -b 512        Batch size for prompt processing
#   --parallel 2  Concurrent request slots
./build/bin/llama-server -m model.gguf -ngl 99 \
  -c 2048 \
  -b 512 \
  --parallel 2

# Ollama: set keep-alive to keep model in memory indefinitely
curl http://localhost:11434/api/generate -d '{
  "model": "llama3:8b",
  "keep_alive": -1
}'

# Or make it the server default: add OLLAMA_KEEP_ALIVE to the
# EnvironmentVariables dict of the Step 6 plist, then reload the service
#   <key>OLLAMA_KEEP_ALIVE</key>
#   <string>-1</string>
launchctl unload ~/Library/LaunchAgents/com.ollama.server.plist
launchctl load ~/Library/LaunchAgents/com.ollama.server.plist

# Check which models are loaded
ollama ps

Pertanyaan yang Sering Diajukan