---
slug: "aistrale"
source_type: "readme"
source_url: "https://cdn.jsdelivr.net/gh/buildworksai/aistrale@main/README.md"
repo: "https://github.com/buildworksai/aistrale"
source_file: "README.md"
branch: "main"
---
<div align="center">

# 🚀 AISTRALE

**Turn AI from a black box into an engineered system**

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://www.python.org/)
[![TypeScript](https://img.shields.io/badge/TypeScript-5-blue.svg)](https://www.typescriptlang.org/)
[![FastAPI](https://img.shields.io/badge/FastAPI-Latest-teal.svg)](https://fastapi.tiangolo.com/)
[![React](https://img.shields.io/badge/React-18-blue.svg)](https://react.dev/)
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-17-blue.svg)](https://www.postgresql.org/)

**Enterprise-grade LLM engineering platform with multi-provider support, comprehensive observability, and production-ready reliability.**

[Website](https://aistrale.com) • [Documentation](#documentation) • [Quick Start](#quick-start) • [Features](#key-features)

---

</div>

## 🎯 What is AISTRALE?

AISTRALE is a **production-ready LLM engineering platform** that transforms AI model inference from an unpredictable black box into a fully observable, secure, and cost-optimized engineering system. Built for teams that need **enterprise-grade reliability** without sacrificing developer experience.

### Why AISTRALE?

- ✅ **5 LLM Providers** - HuggingFace, OpenAI, Groq, Anthropic, Google Gemini
- ✅ **Zero Vendor Lock-in** - Switch providers with one click
- ✅ **Enterprise Security** - Encryption, audit logs, RBAC, compliance-ready
- ✅ **Cost Intelligence** - Track, forecast, and optimize spending automatically
- ✅ **Production Reliability** - Circuit breakers, queuing, failover, 99.9% uptime
- ✅ **Full Observability** - Metrics, logs, traces, error tracking
- ✅ **Developer-First** - One-line integration, SDKs, CLI, VS Code extension

---

## ✨ Key Features

### 🔌 Multi-Provider LLM Inference
Unified interface for **5 major LLM providers** with automatic failover and intelligent routing:
- **HuggingFace Hub** - Open-source models
- **OpenAI** - GPT-3.5, GPT-4, and more
- **Groq** - Ultra-fast inference
- **Anthropic** - Claude models
- **Google Gemini** - Latest Gemini models

**No vendor lock-in.** Switch providers instantly, compare performance, and optimize costs.

### 📊 Comprehensive Observability
**Production-grade monitoring** out of the box:
- **Structured Logging** - JSON logs with correlation IDs (structlog)
- **Prometheus Metrics** - Custom business metrics + system metrics
- **Distributed Tracing** - OpenTelemetry with Jaeger
- **Error Tracking** - Sentry integration
- **Health Checks** - `/health` and `/ready` endpoints
- **Request/Response Logging** - Complete request lifecycle tracking

### 🔐 Enterprise Security
**Security-first architecture** for production workloads:
- **Session-Based Auth** - HTTP-only cookies, Redis-backed sessions
- **Token Encryption** - Fernet encryption with key rotation
- **Security Audit Logs** - Complete audit trail of all actions
- **Role-Based Access Control** - Admin/User roles with fine-grained permissions
- **Data Loss Prevention** - PII detection and redaction
- **Compliance Ready** - SOC 2, GDPR, HIPAA reporting capabilities
- **Workspace & Project Isolation** - Multi-tenant architecture

### 💰 Cost Intelligence
**Not just tracking—actively optimizing:**
- Real-time cost tracking per model/provider
- Cost forecasting and anomaly detection
- Budget alerts and spending limits
- Automatic cost optimization recommendations
- Provider cost comparison
- Cost analytics dashboard with trends

### 🎯 Production Reliability
**Built for 99.9% uptime:**
- **Request Queuing** - Handle traffic spikes gracefully
- **Circuit Breakers** - Prevent cascade failures
- **Intelligent Retry** - Exponential backoff with jitter
- **Automatic Failover** - Switch providers on failure
- **Load Balancing** - Distribute requests intelligently
- **Provider Health Monitoring** - Real-time health checks

### 🛠️ Developer Experience
**One-line integration, zero configuration:**
- **Python SDK** - `pip install aistrale` → `aistrale.run("prompt")`
- **TypeScript SDK** - Full type safety for web apps
- **CLI Tool** - `aistrale-cli` for local development
- **VS Code Extension** - Prompt testing, telemetry viewing
- **Framework Integrations** - LangChain, LlamaIndex ready
- **Webhooks** - Real-time event notifications
- **Model Abstraction** - Unified model interface across providers

### 📝 Prompt Management
**Version-controlled prompt templates:**
- Create, version, and manage prompt templates
- Variable substitution with validation
- Template history and rollback
- Prompt optimization recommendations
- Reusable across projects

### 📈 Telemetry & Analytics
**Complete visibility into your LLM usage:**
- Request/response logging
- Token usage tracking
- Cost analytics and forecasting
- Performance metrics (latency, throughput)
- Success/failure rates
- Provider comparison
- Evaluation framework

### 🧪 Multi-Provider Intelligence
**Advanced provider management:**
- Provider health monitoring
- Side-by-side provider comparison
- Multi-provider A/B testing
- Smart routing based on cost/performance
- Model abstraction layer
- Automatic failover

---

## 🏗️ Architecture

AISTRALE follows **C4 Model** architecture documentation and is built with production-grade patterns:

### System Architecture
```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   React     │────▶│   FastAPI    │────▶│ PostgreSQL  │
│  Frontend   │     │    Backend   │     │  + pgvector │
└─────────────┘     └──────────────┘     └─────────────┘
                            │
                    ┌───────┴───────┐
                    │               │
              ┌─────▼─────┐   ┌────▼─────┐
              │   Redis   │   │ Observ.   │
              │  Sessions │   │  Stack    │
              └───────────┘   └───────────┘
```

### Key Design Patterns
- **Factory Pattern** - Provider abstraction for easy extensibility
- **Strategy Pattern** - Unified interface across providers
- **Repository Pattern** - Clean database access
- **Middleware Pattern** - Session, observability, security

See [Architecture Documentation](https://github.com/buildworksai/aistrale/blob/HEAD/docs/c4-model/ARCHITECTURE.md) for detailed C4 Model diagrams.

---

## 🚀 Quick Start

### Prerequisites
- **Docker** and **Docker Compose**
- **Git**

### 1. Clone Repository
```bash
git clone https://github.com/buildworksai/aistrale.git
cd aistrale
```

### 2. Start Services
```bash
docker-compose up -d
```

This starts:
- **API**: http://localhost:16000
- **Frontend**: http://localhost:16500
- **PostgreSQL**: localhost:15432
- **Redis**: localhost:16379
- **Prometheus**: http://localhost:9090
- **Grafana**: http://localhost:3000 (admin/admin)
- **Jaeger**: http://localhost:16686

### 3. Access Application
1. Open http://localhost:16500
2. Login with an admin account that you seed via environment variables (see Backend env vars).

### 4. Add Your First Token
1. Navigate to **API Keys Tokens**
2. Add a token for your preferred provider (OpenAI, HuggingFace, etc.)
3. Start running inference!

---

## 📚 Documentation

### Architecture
- **[C4 Model Diagrams](https://github.com/buildworksai/aistrale/tree/HEAD/docs/c4-model/)** - Context, Container, Component, Deployment
- **[Architecture Overview](https://github.com/buildworksai/aistrale/blob/HEAD/docs/c4-model/ARCHITECTURE.md)** - System architecture principles
- **[Feature Architecture](https://github.com/buildworksai/aistrale/tree/HEAD/docs/architecture/)** - Detailed architecture for differentiator features

### API Documentation
- **Interactive API Docs**: http://localhost:16000/docs (Swagger UI)
- **API Reference**: [docs/API.md](https://github.com/buildworksai/aistrale/blob/HEAD/docs/API.md)

### Development

- See `CONTRIBUTING.md` for code standards and local tooling.
- Run the local stack via Docker Compose (see Quick Start above).

### Planning
- **[Implementation Plans](https://github.com/buildworksai/aistrale/tree/HEAD/planning/)** - Feature implementation roadmaps
- **[Engineering Gaps](https://github.com/buildworksai/aistrale/blob/HEAD/reports/engineering_gaps_analysis.md)** - Gap analysis and priorities

---

## 🛠️ Tech Stack

### Backend
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Framework** | FastAPI, Uvicorn | High-performance async API |
| **Database** | PostgreSQL 17 + pgvector | Data persistence, future vector search |
| **ORM** | SQLModel | Type-safe database models |
| **Migrations** | Alembic | Database versioning |
| **Auth** | Session-based (Redis) | Secure, scalable authentication |
| **LLM SDKs** | HuggingFace Hub, OpenAI, Groq, Anthropic, Google Generative AI | Multi-provider support |
| **Observability** | structlog, Prometheus, OpenTelemetry, Sentry | Full observability stack |

### Frontend
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Framework** | React 18 | Modern UI framework |
| **Language** | TypeScript 5 | Type safety |
| **Build Tool** | Vite | Fast development and builds |
| **Styling** | Tailwind CSS | Utility-first CSS |
| **Routing** | React Router DOM | Client-side routing |

### Infrastructure
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Containerization** | Docker, Docker Compose | Development and deployment |
| **Monitoring** | Prometheus, Grafana | Metrics and dashboards |
| **Tracing** | Jaeger | Distributed tracing |
| **Caching** | Redis | Session storage |

---

## 📦 Application Pages & Features

### Core Features
- **Dashboard** - Overview of system metrics and activity
- **Inference** - Run LLM inference with multiple providers
- **Tokens** - Manage API keys and tokens securely
- **Telemetry** - View analytics and usage metrics
- **Prompts** - Create and manage prompt templates
- **Users** - User management and administration

### Cost Optimization
- **Cost Dashboard** - Real-time cost tracking and analytics
- **Budgets** - Set and manage spending budgets
- **Cost Forecasting** - Predict future costs
- **Optimization Recommendations** - AI-powered cost savings

### Multi-Provider Intelligence
- **Provider Health** - Monitor provider status and latency
- **Provider Comparison** - Compare providers side-by-side
- **Failover** - Configure automatic failover rules
- **A/B Testing** - Test different providers and models
- **Smart Routing** - Intelligent request routing
- **Model Abstraction** - Unified model interface

### Reliability & Performance
- **Queue Management** - Manage request queues
- **Circuit Breakers** - Configure failure protection
- **Load Balancing** - Distribute load across providers
- **Reliability Dashboard** - System reliability metrics

### Security & Compliance
- **Security Audit** - View security event logs
- **Security Compliance** - Compliance dashboard
- **Compliance** - Compliance reporting and controls
- **Data Residency** - Configure data location requirements
- **DLP** - Data Loss Prevention rules
- **Permissions** - Fine-grained access control

### Workspace Management
- **Workspaces** - Multi-tenant workspace management
- **Projects** - Project organization and isolation
- **Permissions** - Role-based access control

### Developer Tools
- **SDKs** - Python and TypeScript SDK documentation
- **Webhooks** - Configure webhook endpoints
- **Evaluation** - Model evaluation framework
- **Prompt Optimization** - AI-powered prompt improvement

### Administration
- **Admin** - System administration panel
- **Developer Settings** - Developer configuration

---

## 🧪 Development

### Backend Development
```bash
cd backend
pip install -e ".[test]"
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

### Frontend Development
```bash
cd frontend
npm install
npm run dev
```

### Running Tests
```bash
# Backend tests with coverage
cd backend
pytest --cov=. --cov-report=html

# Check coverage report
open htmlcov/index.html
```

### Code Quality
```bash
# Backend
cd backend
black . && isort . && flake8 . && mypy .

# Frontend
cd frontend
npm run lint && npm run type-check
```

---

## 🚢 Deployment

### Docker Compose (Development)
```bash
docker-compose up -d
```

### Production Deployment
```bash
# Build production images
docker-compose -f docker/docker-compose.prod.yml build

# Deploy
docker-compose -f docker/docker-compose.prod.yml up -d
```

### Environment Variables
See `.env.example` for required environment variables:
- `DATABASE_URL` - PostgreSQL connection string
- `REDIS_URL` - Redis connection string
- `SECRET_KEY` - Session secret (generate strong random key)
- `SENTRY_DSN` - Sentry error tracking (optional)
- `ALLOWED_ORIGINS` - CORS allowed origins
- `ENCRYPTION_KEY` - Token encryption key (generate with Fernet)

---

## 📊 Monitoring & Observability

### Access Monitoring Tools
- **Grafana**: http://localhost:3000 (admin/admin)
  - Pre-configured dashboards for metrics
  - Log visualization
- **Jaeger**: http://localhost:16686
  - Distributed tracing UI
  - Request flow visualization
- **Prometheus**: http://localhost:9090
  - Metrics query interface
  - Alert rule management
- **API Metrics**: http://localhost:16000/metrics
  - Prometheus-compatible metrics endpoint

### Health Checks
- **Health**: http://localhost:16000/health
- **Readiness**: http://localhost:16000/ready

---

## 🔒 Security

### Authentication
- **Session-based** - HTTP-only cookies, Redis-backed
- **Password Hashing** - bcrypt with salt
- **Session Expiration** - Configurable timeout

### Authorization
- **RBAC** - Role-based access control (Admin/User)
- **Route Protection** - Deny-by-default policy
- **Fine-grained Permissions** - Per-resource access control
- **Workspace/Project Isolation** - Multi-tenant security

### Data Protection
- **Token Encryption** - Fernet encryption at rest
- **Key Rotation** - Automatic quarterly rotation
- **Security Audit Logs** - Complete audit trail
- **PII Detection** - Automatic detection and redaction
- **Data Loss Prevention** - DLP rules and policies

---

## 🤝 Contributing

We welcome contributions! Please see our contributing guidelines:

1. **Fork the repository**
2. **Create a feature branch** (`git checkout -b feature/amazing-feature`)
3. **Follow code standards** (see `CONTRIBUTING.md`)
4. **Write tests** (aim for >80% coverage)
5. **Commit changes** (`git commit -m 'Add amazing feature'`)
6. **Push to branch** (`git push origin feature/amazing-feature`)
7. **Open a Pull Request**

### Code Standards
- **Python**: Black, isort, Flake8, mypy
- **TypeScript**: ESLint, Prettier, strict mode
- **Tests**: pytest for backend, comprehensive coverage
- **Documentation**: Update README and docs for new features

---

## 📄 License

Copyright 2025 BuildWorks.AI

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

---

## 🆘 Support

- **Website**: [https://aistrale.com](https://aistrale.com)
- **Documentation**: [docs/](https://github.com/buildworksai/aistrale/tree/HEAD/docs/)
- **Issues**: [GitHub Issues](https://github.com/buildworksai/aistrale/issues)
- **Email**: info@buildworks.ai

---

## 🌟 Implementation Status

### ✅ Completed Features (Q1 2025)

**Core Platform (100% Complete)**
- ✅ **Multi-Provider LLM Support** - 5 providers (HuggingFace, OpenAI, Groq, Anthropic, Gemini)
- ✅ **Session-Based Authentication** - Redis-backed sessions with HTTP-only cookies
- ✅ **Token Management** - Encrypted token storage with key rotation
- ✅ **Telemetry Tracking** - Complete request/response logging and analytics
- ✅ **Prompt Management** - Template creation, versioning, and variable substitution

**Observability Stack (100% Complete)**
- ✅ **Structured Logging** - structlog with JSON output and correlation IDs
- ✅ **Prometheus Metrics** - Custom business metrics + system metrics
- ✅ **Distributed Tracing** - OpenTelemetry with Jaeger integration
- ✅ **Error Tracking** - Sentry integration for error monitoring
- ✅ **Health Checks** - `/health` and `/ready` endpoints

**Cost Intelligence (100% Complete)**
- ✅ **Cost Tracking** - Real-time cost tracking per model/provider
- ✅ **Cost Analytics** - Dashboard with trends and breakdowns
- ✅ **Budget Management** - Set and monitor spending budgets
- ✅ **Cost Forecasting** - Predict future costs with confidence intervals
- ✅ **Anomaly Detection** - Automatic detection of cost anomalies
- ✅ **Optimization Recommendations** - AI-powered cost savings suggestions

**Multi-Provider Intelligence (100% Complete)**
- ✅ **Provider Health Monitoring** - Real-time health checks and latency tracking
- ✅ **Provider Comparison** - Side-by-side provider performance comparison
- ✅ **Automatic Failover** - Configure and manage failover rules
- ✅ **A/B Testing** - Test different providers and models
- ✅ **Smart Routing** - Intelligent request routing based on rules
- ✅ **Model Abstraction** - Unified model interface across providers

**Reliability Features (100% Complete)**
- ✅ **Request Queuing** - Queue management for traffic spikes
- ✅ **Circuit Breakers** - Failure protection and recovery
- ✅ **Load Balancing** - Distribute requests across providers
- ✅ **Retry Logic** - Exponential backoff with jitter

**Security & Compliance (100% Complete)**
- ✅ **Security Audit Logging** - Complete audit trail of all actions
- ✅ **Role-Based Access Control** - Admin/User roles with fine-grained permissions
- ✅ **Workspace & Project Management** - Multi-tenant architecture
- ✅ **Data Loss Prevention** - PII detection and redaction rules
- ✅ **Compliance Dashboard** - Compliance reporting and controls
- ✅ **Data Residency** - Configure data location requirements

**Developer Experience (100% Complete)**
- ✅ **Webhooks** - Real-time event notifications
- ✅ **Evaluation Framework** - Model evaluation and scoring
- ✅ **Prompt Optimization** - AI-powered prompt improvement
- ✅ **SDKs Documentation** - Python and TypeScript SDK guides

**Frontend (100% Complete)**
- ✅ **37 Application Pages** - Complete UI for all features
- ✅ **Protected Routes** - Authentication and authorization
- ✅ **Responsive Design** - Mobile-friendly interface
- ✅ **Dark Mode** - Theme support

### 🔄 In Progress / Planned

**Advanced Features (Architecture Designed)**
- 🔄 **Python SDK** - Full SDK implementation (basic structure exists)
- 🔄 **TypeScript SDK** - Full SDK implementation (basic structure exists)
- 🔄 **CLI Tool** - Command-line interface (basic structure exists)
- 🔄 **VS Code Extension** - IDE integration (basic structure exists)
- 🔄 **Framework Integrations** - LangChain, LlamaIndex wrappers

**Production Enhancements**
- 🔄 **CI/CD Pipeline** - Automated testing and deployment
- 🔄 **Production Docker Config** - Optimized production containers
- 🔄 **Monitoring Alerts** - Alertmanager integration
- 🔄 **Backup Strategy** - Automated database backups

**Status Legend:**
- ✅ **Completed** - Fully implemented and production-ready
- 🔄 **In Progress** - Basic implementation exists, full features in development
- 📋 **Planned** - Architecture designed, implementation scheduled

See [Planning Documentation](https://github.com/buildworksai/aistrale/tree/HEAD/planning/) for detailed implementation plans and [Architecture Documentation](https://github.com/buildworksai/aistrale/tree/HEAD/docs/architecture/) for technical designs.

---

## 🙏 Acknowledgments

Built by [BuildWorks.AI](https://buildworks.ai)

**AISTRALE** - Turn AI from a black box into an engineered system

---

<div align="center">

**[⬆ Back to Top](#aistrale)**

Built by [BuildWorks.AI](https://buildworks.ai)

</div>