---
slug: "sinamtz-pi-mlx-provider"
source_type: "readme"
source_url: "https://cdn.jsdelivr.net/gh/sinamtz/pi-mlx-provider@main/README.md"
repo: "https://github.com/sinamtz/pi-mlx-provider"
source_file: "README.md"
branch: "main"
---
# Pi Provider - MLX Local

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

> **Note:** This is an unofficial, community-built provider. It runs local AI models on Apple Silicon using Apple's MLX framework.

A [Pi coding agent](https://github.com/badlogic/pi-mono) provider extension for local AI inference using Apple's MLX framework.

## Features

- **Local AI Inference**: Run AI models entirely on your Mac (Apple Silicon)
- **No API Costs**: Free inference using your local GPU/memory
- **Privacy-First**: Your data never leaves your machine
- **Apple Silicon Optimized**: Leverages MLX for efficient inference on M1/M2/M3/M4 chips
- **Vision Support**: Supports vision-language models for image understanding
- **OpenAI-Compatible**: Uses MLX-VLM server with OpenAI-compatible API

## Requirements

- **macOS**: Monterey 12.3 or later
- **Apple Silicon**: M1, M2, M3, or M4 chip
- **Python**: 3.10+ with pip
- **MLX-VLM**: Installed automatically via `/mlx_init`

## Installation

### Option 1: Using pi install (npm)

```bash
pi install npm:@sinamtz/pi-mlx-provider
```

### Option 2: Using pi install (git)

```bash
pi install git:https://github.com/sinamtz/pi-mlx-provider
```

### Option 3: Install globally via npm

```bash
npm install -g @sinamtz/pi-mlx-provider
```

### Option 4: Clone locally

```bash
git clone https://github.com/sinamtz/pi-mlx-provider.git
cd pi-mlx-provider
npm install
```

## Configuration

### Enable Local AI

1. Edit `~/.pi/settings.json` to add the MLX provider settings:

```json
{
  "mlx-provider": {
    "local_intelligence": true,
    "local_model": true,
    "mlx_model": "mlx-community/gemma-4-e2b-it-4bit",
    "mlx_port": 8080,
    "mlx_host": "127.0.0.1"
  }
}
```

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `local_intelligence` | boolean | `false` | Enable local AI processing |
| `local_model` | boolean | `false` | Use local model (requires local_intelligence=true) |
| `mlx_model` | string | `mlx-community/gemma-4-e2b-it-4bit` | MLX model to use |
| `mlx_port` | number | `8080` | Server port |
| `mlx_host` | string | `127.0.0.1` | Server host |

**Note**: Both `local_intelligence` AND `local_model` must be `true` to enable local mode.

### Auto-Start Behavior

The extension automatically starts the MLX server on load when:
1. Local mode is enabled in settings (`local_intelligence` and `local_model` are both `true`)
2. A state file exists at `~/.pi/agent/extensions/mlx-settings.json` (created by `/mlx_init`)

This means after running `/mlx_init` once, the server will automatically start on future loads without needing to re-run `/mlx_init`.

## Usage

### Initialize MLX Server (First Time)

```bash
pi -e ./pi-mlx-provider
/mlx_init
```

This will:
1. Install/upgrade mlx-vlm package
2. Download the MLX model (first time only)
3. Start the MLX-VLM server
4. Save state to `~/.pi/agent/extensions/mlx-settings.json`
5. Register the mlx-local provider

### Auto-Start on Future Loads

Once initialized, the server will automatically start on future loads when local mode is enabled. No need to run `/mlx_init` each time.

### Check Server Status

```bash
/mlx_status
```

### Stop Server

```bash
/mlx_stop
```

This stops the server and removes the state file, preventing auto-start on future loads.

### Select Model

In pi's interactive mode:

```
/model mlx-local
```

Or use the model selector UI.

## Supported Models

| Model Type | Example | Input |
|------------|---------|-------|
| Text-only LLM | `mlx-community/Llama-3.2-3B-Instruct-4bit` | text |
| Vision-Language | `mlx-community/Qwen2-VL-2B-Instruct-4bit` | text, image |
| Gemma | `mlx-community/gemma-4-e2b-it-4bit` | text |

### Recommended Models

- **gemma-4-e2b-it-4bit**: Good balance of quality and speed (default)
- **Llama-3.2-3B-Instruct-4bit**: Fast, good for simple tasks
- **Qwen2-VL-2B-Instruct-4bit**: Vision support for image understanding

## Troubleshooting

### "MLX server is not running"
- Run `/mlx_init` to start the server

### "Failed to install mlx-vlm"
- Ensure Python 3.10+ is installed
- Try: `pip3 install --upgrade mlx-vlm`

### "Model download failed"
- Check internet connection
- Ensure sufficient disk space for model files

### Port already in use
- Change `mlx_port` in settings.json to a different port (e.g., 8081)

### Out of memory
- Use a smaller/quantized model (4bit variants)
- Close other memory-intensive applications

## How It Works

```
pi agent
    └── pi-mlx-provider extension
            ├── Reads settings from ~/.pi/settings.json
            ├── If local_intelligence + local_model enabled AND state file exists:
            │   ├── Auto-starts mlx-vlm server (or reconnects if already running)
            │   ├── Registers "mlx-local" provider
            │   └── Routes requests to localhost:PORT
            ├── If enabled but no state file:
            │   └── Waits for /mlx_init command
            └── If disabled: uses default cloud provider
```

### State Persistence

The extension maintains state in `~/.pi/agent/extensions/mlx-settings.json`:
- Created by `/mlx_init` after successful server start
- Read on extension load to determine if auto-start should occur
- Removed by `/mlx_stop` to prevent auto-start on future loads

## License

MIT

## Contributing

Contributions welcome! Please open an issue or PR on GitHub.