aistrale

内容来源：README.md（说明文档） · 原始地址 · 查看安装指南

原始内容

<div align="center">

# 🚀 AISTRALE

**Turn AI from a black box into an engineered system**

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://www.python.org/)
[![TypeScript](https://img.shields.io/badge/TypeScript-5-blue.svg)](https://www.typescriptlang.org/)
[![FastAPI](https://img.shields.io/badge/FastAPI-Latest-teal.svg)](https://fastapi.tiangolo.com/)
[![React](https://img.shields.io/badge/React-18-blue.svg)](https://react.dev/)
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-17-blue.svg)](https://www.postgresql.org/)

**Enterprise-grade LLM engineering platform with multi-provider support, comprehensive observability, and production-ready reliability.**

[Website](https://aistrale.com) • [Documentation](#documentation) • [Quick Start](#quick-start) • [Features](#key-features)

---

</div>

## 🎯 What is AISTRALE?

AISTRALE is a **production-ready LLM engineering platform** that transforms AI model inference from an unpredictable black box into a fully observable, secure, and cost-optimized engineering system. Built for teams that need **enterprise-grade reliability** without sacrificing developer experience.

### Why AISTRALE?

- ✅ **5 LLM Providers** - HuggingFace, OpenAI, Groq, Anthropic, Google Gemini
- ✅ **Zero Vendor Lock-in** - Switch providers with one click
- ✅ **Enterprise Security** - Encryption, audit logs, RBAC, compliance-ready
- ✅ **Cost Intelligence** - Track, forecast, and optimize spending automatically
- ✅ **Production Reliability** - Circuit breakers, queuing, failover, 99.9% uptime
- ✅ **Full Observability** - Metrics, logs, traces, error tracking
- ✅ **Developer-First** - One-line integration, SDKs, CLI, VS Code extension

---

## ✨ Key Features

### 🔌 Multi-Provider LLM Inference
Unified interface for **5 major LLM providers** with automatic failover and intelligent routing:
- **HuggingFace Hub** - Open-source models
- **OpenAI** - GPT-3.5, GPT-4, and more
- **Groq** - Ultra-fast inference
- **Anthropic** - Claude models
- **Google Gemini** - Latest Gemini models

**No vendor lock-in.** Switch providers instantly, compare performance, and optimize costs.

### 📊 Comprehensive Observability
**Production-grade monitoring** out of the box:
- **Structured Logging** - JSON logs with correlation IDs (structlog)
- **Prometheus Metrics** - Custom business metrics + system metrics
- **Distributed Tracing** - OpenTelemetry with Jaeger
- **Error Tracking** - Sentry integration
- **Health Checks** - `/health` and `/ready` endpoints
- **Request/Response Logging** - Complete request lifecycle tracking

### 🔐 Enterprise Security
**Security-first architecture** for production workloads:
- **Session-Based Auth** - HTTP-only cookies, Redis-backed sessions
- **Token Encryption** - Fernet encryption with key rotation
- **Security Audit Logs** - Complete audit trail of all actions
- **Role-Based Access Control** - Admin/User roles with fine-grained permissions
- **Data Loss Prevention** - PII detection and redaction
- **Compliance Ready** - SOC 2, GDPR, HIPAA reporting capabilities
- **Workspace & Project Isolation** - Multi-tenant architecture

### 💰 Cost Intelligence
**Not just tracking—actively optimizing:**
- Real-time cost tracking per model/provider
- Cost forecasting and anomaly detection
- Budget alerts and spending limits
- Automatic cost optimization recommendations
- Provider cost comparison
- Cost analytics dashboard with trends

### 🎯 Production Reliability
**Built for 99.9% uptime:**
- **Request Queuing** - Handle traffic spikes gracefully
- **Circuit Breakers** - Prevent cascade failures
- **Intelligent Retry** - Exponential backoff with jitter
- **Automatic Failover** - Switch providers on failure
- **Load Balancing** - Distribute requests intelligently
- **Provider Health Monitoring** - Real-time health checks

### 🛠️ Developer Experience
**One-line integration, zero configuration:**
- **Python SDK** - `pip install aistrale` → `aistrale.run("prompt")`
- **TypeScript SDK** - Full type safety for web apps
- **CLI Tool** - `aistrale-cli` for local development
- **VS Code Extension** - Prompt testing, telemetry viewing
- **Framework Integrations** - LangChain, LlamaIndex ready
- **Webhooks** - Real-time event notifications
- **Model Abstraction** - Unified model interface across providers

### 📝 Prompt Management
**Version-controlled prompt templates:**
- Create, version, and manage prompt templates
- Variable substitution with validation
- Template history and rollback
- Prompt optimization recommendations
- Reusable across projects

### 📈 Telemetry & Analytics
**Complete visibility into your LLM usage:**
- Request/response logging
- Token usage tracking
- Cost analytics and forecasting
- Performance metrics (latency, throughput)
- Success/failure rates
- Provider comparison
- Evaluation framework

### 🧪 Multi-Provider Intelligence
**Advanced provider management:**
- Provider health monitoring
- Side-by-side provider comparison
- Multi-provider A/B testing
- Smart routing based on cost/performance
- Model abstraction layer
- Automatic failover

---

## 🏗️ Architecture

AISTRALE follows **C4 Model** architecture documentation and is built with production-grade patterns:

### System Architecture
```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   React     │────▶│   FastAPI    │────▶│ PostgreSQL  │
│  Frontend   │     │    Backend   │     │  + pgvector │
└─────────────┘     └──────────────┘     └─────────────┘
                            │
                    ┌───────┴───────┐
                    │               │
              ┌─────▼─────┐   ┌────▼─────┐
              │   Redis   │   │ Observ.   │
              │  Sessions │   │  Stack    │
              └───────────┘   └───────────┘
```

### Key Design Patterns
- **Factory Pattern** - Provider abstraction for easy extensibility
- **Strategy Pattern** - Unified interface across providers
- **Repository Pattern** - Clean database access
- **Middleware Pattern** - Session, observability, security

See [Architecture Documentation](https://github.com/buildworksai/aistrale/blob/HEAD/docs/c4-model/ARCHITECTURE.md) for detailed C4 Model diagrams.

---

## 🚀 Quick Start

### Prerequisites
- **Docker** and **Docker Compose**
- **Git**

### 1. Clone Repository
```bash
git clone https://github.com/buildworksai/aistrale.git
cd aistrale
```

### 2. Start Services
```bash
docker-compose up -d
```

This starts:
- **API**: http://localhost:16000
- **Frontend**: http://localhost:16500
- **PostgreSQL**: localhost:15432
- **Redis**: localhost:16379
- **Prometheus**: http://localhost:9090
- **Grafana**: http://localhost:3000 (admin/admin)
- **Jaeger**: http://localhost:16686

### 3. Access Application
1. Open http://localhost:16500
2. Login with an admin account that you seed via environment variables (see Backend env vars).

### 4. Add Your First Token
1. Navigate to **API Keys Tokens**
2. Add a token for your preferred provider (OpenAI, HuggingFace, etc.)
3. Start running inference!

---

## 📚 Documentation

### Architecture
- **[C4 Model Diagrams](https://github.com/buildworksai/aistrale/tree/HEAD/docs/c4-model/)** - Context, Container, Component, Deployment
- **[Architecture Overview](https://github.com/buildworksai/aistrale/blob/HEAD/docs/c4-model/ARCHITECTURE.md)** - System architecture principles
- **[Feature Architecture](https://github.com/buildworksai/aistrale/tree/HEAD/docs/architecture/)** - Detailed architecture for differentiator features

### API Documentation
- **Interactive API Docs**: http://localhost:16000/docs (Swagger UI)
- **API Reference**: [docs/API.md](https://github.com/buildworksai/aistrale/blob/HEAD/docs/API.md)

### Development

- See `CONTRIBUTING.md` for code standards and local tooling.
- Run the local stack via Docker Compose (see Quick Start above).

### Planning
- **[Implementation Plans](https://github.com/buildworksai/aistrale/tree/HEAD/planning/)** - Feature implementation roadmaps
- **[Engineering Gaps](https://github.com/buildworksai/aistrale/blob/HEAD/reports/engineering_gaps_analysis.md)** - Gap analysis and priorities

---

## 🛠️ Tech Stack

### Backend
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Framework** | FastAPI, Uvicorn | High-performance async API |
| **Database** | PostgreSQL 17 + pgvector | Data persistence, future vector search |
| **ORM** | SQLModel | Type-safe database models |
| **Migrations** | Alembic | Database versioning |
| **Auth** | Session-based (Redis) | Secure, scalable authentication |
| **LLM SDKs** | HuggingFace Hub, OpenAI, Groq, Anthropic, Google Generative AI | Multi-provider support |
| **Observability** | structlog, Prometheus, OpenTelemetry, Sentry | Full observability stack |

### Frontend
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Framework** | React 18 | Modern UI framework |
| **Language** | TypeScript 5 | Type safety |
| **Build Tool** | Vite | Fast development and builds |
| **Styling** | Tailwind CSS | Utility-first CSS |
| **Routing** | React Router DOM | Client-side routing |

### Infrastructure
| Component | Technology | Purpose |
|-----------|-----------|---------|
| **Containerization** | Docker, Docker Compose | Development and deployment |
| **Monitoring** | Prometheus, Grafana | Metrics and dashboards |
| **Tracing** | Jaeger | Distributed tracing |
| **Caching** | Redis | Session storage |

---

## 📦 Application Pages & Features

### Core Features
- **Dashboard** - Overview of system metrics and activity
- **Inference** - Run LLM inference with multiple providers
- **Tokens** - Manage API keys and tokens securely
- **Telemetry** - View analytics and usage metrics
- **Prompts** - Create and manage prompt templates
- **Users** - User management and administration

### Cost Optimization
- **Cost Dashboard** - Real-time cost tracking and analytics
- **Budgets** - Set and manage spending budgets
- **Cost Forecasting** - Predict future costs
- **Optimization Recommendations** - AI-powered cost savings

### Multi-Provider Intelligence
- **Provider Health** - Monitor provider status and latency
- **Provider Comparison** - Compare providers side-by-side
- **Failover** - Configure automatic failover rules
- **A/B Testing** - Test different providers and models
- **Smart Routing** - Intelligent request routing
- **Model Abstraction** - Unified model interface

### Reliability & Performance
- **Queue Management** - Manage request queues
- **Circuit Breakers** - Configure failure protection
- **Load Balancing** - Distribute load across providers
- **Reliability Dashboard** - System reliability metrics

### Security & Compliance
- **Security Audit** - View security event logs
- **Security Compliance** - Compliance dashboard
- **Compliance** - Compliance reporting and controls
- **Data Residency** - Configure data location requirements
- **DLP** - Data Loss Prevention rules
- **Permissions** - Fine-grained access control

### Workspace Management
- **Workspaces** - Multi-tenant workspace management
- **Projects** - Project organization and isolation
- **Permissions** - Role-based access control

### Developer Tools
- **SDKs** - Python and TypeScript SDK documentation
- **Webhooks** - Configure webhook endpoints
- **Evaluation** - Model evaluation framework
- **Prompt Optimization** - AI-powered prompt improvement

### Administration
- **Admin** - System administration panel
- **Developer Settings** - Developer configuration

---

## 🧪 Development

### Backend Development
```bash
cd backend
pip install -e ".[test]"
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

### Frontend Development
```bash
cd frontend
npm install
npm run dev
```

### Running Tests
```bash
# Backend tests with coverage
cd backend
pytest --cov=. --cov-report=html

# Check coverage report
open htmlcov/index.html
```

### Code Quality
```bash
# Backend
cd backend
black . && isort . && flake8 . && mypy .

# Frontend
cd frontend
npm run lint && npm run type-check
```

---

## 🚢 Deployment

### Docker Compose (Development)
```bash
docker-compose up -d
```

### Production Deployment
```bash
# Build production images
docker-compose -f docker/docker-compose.prod.yml build

# Deploy
docker-compose -f docker/docker-compose.prod.yml up -d
```

### Environment Variables
See `.env.example` for required environment variables:
- `DATABASE_URL` - PostgreSQL connection string
- `REDIS_URL` - Redis connection string
- `SECRET_KEY` - Session secret (generate strong random key)
- `SENTRY_DSN` - Sentry error tracking (optional)
- `ALLOWED_ORIGINS` - CORS allowed origins
- `ENCRYPTION_KEY` - Token encryption key (generate with Fernet)

---

## 📊 Monitoring & Observability

### Access Monitoring Tools
- **Grafana**: http://localhost:3000 (admin/admin)
  - Pre-configured dashboards for metrics
  - Log visualization
- **Jaeger**: http://localhost:16686
  - Distributed tracing UI
  - Request flow visualization
- **Prometheus**: http://localhost:9090
  - Metrics query interface
  - Alert rule management
- **API Metrics**: http://localhost:16000/metrics
  - Prometheus-compatible metrics endpoint

### Health Checks
- **Health**: http://localhost:16000/health
- **Readiness**: http://localhost:16000/ready

---

## 🔒 Security

### Authentication
- **Session-based** - HTTP-only cookies, Redis-backed
- **Password Hashing** - bcrypt with salt
- **Session Expiration** - Configurable timeout

### Authorization
- **RBAC** - Role-based access control (Admin/User)
- **Route Protection** - Deny-by-default policy
- **Fine-grained Permissions** - Per-resource access control
- **Workspace/Project Isolation** - Multi-tenant security

### Data Protection
- **Token Encryption** - Fernet encryption at rest
- **Key Rotation** - Automatic quarterly rotation
- **Security Audit Logs** - Complete audit trail
- **PII Detection** - Automatic detection and redaction
- **Data Loss Prevention** - DLP rules and policies

---

## 🤝 Contributing

We welcome contributions! Please see our contributing guidelines:

1. **Fork the repository**
2. **Create a feature branch** (`git checkout -b feature/amazing-feature`)
3. **Follow code standards** (see `CONTRIBUTING.md`)
4. **Write tests** (aim for >80% coverage)
5. **Commit changes** (`git commit -m 'Add amazing feature'`)
6. **Push to branch** (`git push origin feature/amazing-feature`)
7. **Open a Pull Request**

### Code Standards
- **Python**: Black, isort, Flake8, mypy
- **TypeScript**: ESLint, Prettier, strict mode
- **Tests**: pytest for backend, comprehensive coverage
- **Documentation**: Update README and docs for new features

---

## 📄 License

Copyright 2025 BuildWorks.AI

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

---

## 🆘 Support

- **Website**: [https://aistrale.com](https://aistrale.com)
- **Documentation**: [docs/](https://github.com/buildworksai/aistrale/tree/HEAD/docs/)
- **Issues**: [GitHub Issues](https://github.com/buildworksai/aistrale/issues)
- **Email**: info@buildworks.ai

---

## 🌟 Implementation Status

### ✅ Completed Features (Q1 2025)

**Core Platform (100% Complete)**
- ✅ **Multi-Provider LLM Support** - 5 providers (HuggingFace, OpenAI, Groq, Anthropic, Gemini)
- ✅ **Session-Based Authentication** - Redis-backed sessions with HTTP-only cookies
- ✅ **Token Management** - Encrypted token storage with key rotation
- ✅ **Telemetry Tracking** - Complete request/response logging and analytics
- ✅ **Prompt Management** - Template creation, versioning, and variable substitution

**Observability Stack (100% Complete)**
- ✅ **Structured Logging** - structlog with JSON output and correlation IDs
- ✅ **Prometheus Metrics** - Custom business metrics + system metrics
- ✅ **Distributed Tracing** - OpenTelemetry with Jaeger integration
- ✅ **Error Tracking** - Sentry integration for error monitoring
- ✅ **Health Checks** - `/health` and `/ready` endpoints

**Cost Intelligence (100% Complete)**
- ✅ **Cost Tracking** - Real-time cost tracking per model/provider
- ✅ **Cost Analytics** - Dashboard with trends and breakdowns
- ✅ **Budget Management** - Set and monitor spending budgets
- ✅ **Cost Forecasting** - Predict future costs with confidence intervals
- ✅ **Anomaly Detection** - Automatic detection of cost anomalies
- ✅ **Optimization Recommendations** - AI-powered cost savings suggestions

**Multi-Provider Intelligence (100% Complete)**
- ✅ **Provider Health Monitoring** - Real-time health checks and latency tracking
- ✅ **Provider Comparison** - Side-by-side provider performance comparison
- ✅ **Automatic Failover** - Configure and manage failover rules
- ✅ **A/B Testing** - Test different providers and models
- ✅ **Smart Routing** - Intelligent request routing based on rules
- ✅ **Model Abstraction** - Unified model interface across providers

**Reliability Features (100% Complete)**
- ✅ **Request Queuing** - Queue management for traffic spikes
- ✅ **Circuit Breakers** - Failure protection and recovery
- ✅ **Load Balancing** - Distribute requests across providers
- ✅ **Retry Logic** - Exponential backoff with jitter

**Security & Compliance (100% Complete)**
- ✅ **Security Audit Logging** - Complete audit trail of all actions
- ✅ **Role-Based Access Control** - Admin/User roles with fine-grained permissions
- ✅ **Workspace & Project Management** - Multi-tenant architecture
- ✅ **Data Loss Prevention** - PII detection and redaction rules
- ✅ **Compliance Dashboard** - Compliance reporting and controls
- ✅ **Data Residency** - Configure data location requirements

**Developer Experience (100% Complete)**
- ✅ **Webhooks** - Real-time event notifications
- ✅ **Evaluation Framework** - Model evaluation and scoring
- ✅ **Prompt Optimization** - AI-powered prompt improvement
- ✅ **SDKs Documentation** - Python and TypeScript SDK guides

**Frontend (100% Complete)**
- ✅ **37 Application Pages** - Complete UI for all features
- ✅ **Protected Routes** - Authentication and authorization
- ✅ **Responsive Design** - Mobile-friendly interface
- ✅ **Dark Mode** - Theme support

### 🔄 In Progress / Planned

**Advanced Features (Architecture Designed)**
- 🔄 **Python SDK** - Full SDK implementation (basic structure exists)
- 🔄 **TypeScript SDK** - Full SDK implementation (basic structure exists)
- 🔄 **CLI Tool** - Command-line interface (basic structure exists)
- 🔄 **VS Code Extension** - IDE integration (basic structure exists)
- 🔄 **Framework Integrations** - LangChain, LlamaIndex wrappers

**Production Enhancements**
- 🔄 **CI/CD Pipeline** - Automated testing and deployment
- 🔄 **Production Docker Config** - Optimized production containers
- 🔄 **Monitoring Alerts** - Alertmanager integration
- 🔄 **Backup Strategy** - Automated database backups

**Status Legend:**
- ✅ **Completed** - Fully implemented and production-ready
- 🔄 **In Progress** - Basic implementation exists, full features in development
- 📋 **Planned** - Architecture designed, implementation scheduled

See [Planning Documentation](https://github.com/buildworksai/aistrale/tree/HEAD/planning/) for detailed implementation plans and [Architecture Documentation](https://github.com/buildworksai/aistrale/tree/HEAD/docs/architecture/) for technical designs.

---

## 🙏 Acknowledgments

Built by [BuildWorks.AI](https://buildworks.ai)

**AISTRALE** - Turn AI from a black box into an engineered system

---

<div align="center">

**[⬆ Back to Top](#aistrale)**

Built by [BuildWorks.AI](https://buildworks.ai)

</div>

🚀 AISTRALE

Turn AI from a black box into an engineered system

Enterprise-grade LLM engineering platform with multi-provider support, comprehensive observability, and production-ready reliability.

Website • Documentation • Quick Start • Features

🎯 What is AISTRALE?

AISTRALE is a production-ready LLM engineering platform that transforms AI model inference from an unpredictable black box into a fully observable, secure, and cost-optimized engineering system. Built for teams that need enterprise-grade reliability without sacrificing developer experience.

Why AISTRALE?

✅ 5 LLM Providers - HuggingFace, OpenAI, Groq, Anthropic, Google Gemini
✅ Zero Vendor Lock-in - Switch providers with one click
✅ Enterprise Security - Encryption, audit logs, RBAC, compliance-ready
✅ Cost Intelligence - Track, forecast, and optimize spending automatically
✅ Production Reliability - Circuit breakers, queuing, failover, 99.9% uptime
✅ Full Observability - Metrics, logs, traces, error tracking
✅ Developer-First - One-line integration, SDKs, CLI, VS Code extension

✨ Key Features

🔌 Multi-Provider LLM Inference

Unified interface for 5 major LLM providers with automatic failover and intelligent routing:

HuggingFace Hub - Open-source models
OpenAI - GPT-3.5, GPT-4, and more
Groq - Ultra-fast inference
Anthropic - Claude models
Google Gemini - Latest Gemini models

No vendor lock-in. Switch providers instantly, compare performance, and optimize costs.

📊 Comprehensive Observability

Production-grade monitoring out of the box:

Structured Logging - JSON logs with correlation IDs (structlog)
Prometheus Metrics - Custom business metrics + system metrics
Distributed Tracing - OpenTelemetry with Jaeger
Error Tracking - Sentry integration
Health Checks - /health and /ready endpoints
Request/Response Logging - Complete request lifecycle tracking

🔐 Enterprise Security

Security-first architecture for production workloads:

Session-Based Auth - HTTP-only cookies, Redis-backed sessions
Token Encryption - Fernet encryption with key rotation
Security Audit Logs - Complete audit trail of all actions
Role-Based Access Control - Admin/User roles with fine-grained permissions
Data Loss Prevention - PII detection and redaction
Compliance Ready - SOC 2, GDPR, HIPAA reporting capabilities
Workspace & Project Isolation - Multi-tenant architecture

💰 Cost Intelligence

Not just tracking—actively optimizing:

Real-time cost tracking per model/provider
Cost forecasting and anomaly detection
Budget alerts and spending limits
Automatic cost optimization recommendations
Provider cost comparison
Cost analytics dashboard with trends

🎯 Production Reliability

Built for 99.9% uptime:

Request Queuing - Handle traffic spikes gracefully
Circuit Breakers - Prevent cascade failures
Intelligent Retry - Exponential backoff with jitter
Automatic Failover - Switch providers on failure
Load Balancing - Distribute requests intelligently
Provider Health Monitoring - Real-time health checks

🛠️ Developer Experience

One-line integration, zero configuration:

Python SDK - pip install aistrale → aistrale.run("prompt")
TypeScript SDK - Full type safety for web apps
CLI Tool - aistrale-cli for local development
VS Code Extension - Prompt testing, telemetry viewing
Framework Integrations - LangChain, LlamaIndex ready
Webhooks - Real-time event notifications
Model Abstraction - Unified model interface across providers

📝 Prompt Management

Version-controlled prompt templates:

Create, version, and manage prompt templates
Variable substitution with validation
Template history and rollback
Prompt optimization recommendations
Reusable across projects

📈 Telemetry & Analytics

Complete visibility into your LLM usage:

Request/response logging
Token usage tracking
Cost analytics and forecasting
Performance metrics (latency, throughput)
Success/failure rates
Provider comparison
Evaluation framework

🧪 Multi-Provider Intelligence

Advanced provider management:

Provider health monitoring
Side-by-side provider comparison
Multi-provider A/B testing
Smart routing based on cost/performance
Model abstraction layer
Automatic failover

🏗️ Architecture

AISTRALE follows C4 Model architecture documentation and is built with production-grade patterns:

System Architecture

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   React     │────▶│   FastAPI    │────▶│ PostgreSQL  │
│  Frontend   │     │    Backend   │     │  + pgvector │
└─────────────┘     └──────────────┘     └─────────────┘
                            │
                    ┌───────┴───────┐
                    │               │
              ┌─────▼─────┐   ┌────▼─────┐
              │   Redis   │   │ Observ.   │
              │  Sessions │   │  Stack    │
              └───────────┘   └───────────┘

Key Design Patterns

Factory Pattern - Provider abstraction for easy extensibility
Strategy Pattern - Unified interface across providers
Repository Pattern - Clean database access
Middleware Pattern - Session, observability, security

See Architecture Documentation for detailed C4 Model diagrams.

🚀 Quick Start

Prerequisites

Docker and Docker Compose
Git

1. Clone Repository

git clone https://github.com/buildworksai/aistrale.git
cd aistrale

2. Start Services

docker-compose up -d

This starts:

API: http://localhost:16000
Frontend: http://localhost:16500
PostgreSQL: localhost:15432
Redis: localhost:16379
Prometheus: http://localhost:9090
Grafana: http://localhost:3000 (admin/admin)
Jaeger: http://localhost:16686

3. Access Application

Open http://localhost:16500
Login with an admin account that you seed via environment variables (see Backend env vars).

4. Add Your First Token

Navigate to API Keys Tokens
Add a token for your preferred provider (OpenAI, HuggingFace, etc.)
Start running inference!

📚 Documentation

Architecture

C4 Model Diagrams - Context, Container, Component, Deployment
Architecture Overview - System architecture principles
Feature Architecture - Detailed architecture for differentiator features

API Documentation

Interactive API Docs: http://localhost:16000/docs (Swagger UI)
API Reference: docs/API.md

Development

See CONTRIBUTING.md for code standards and local tooling.
Run the local stack via Docker Compose (see Quick Start above).

Planning

Implementation Plans - Feature implementation roadmaps
Engineering Gaps - Gap analysis and priorities

🛠️ Tech Stack

Backend

Component	Technology	Purpose
Framework	FastAPI, Uvicorn	High-performance async API
Database	PostgreSQL 17 + pgvector	Data persistence, future vector search
ORM	SQLModel	Type-safe database models
Migrations	Alembic	Database versioning
Auth	Session-based (Redis)	Secure, scalable authentication
LLM SDKs	HuggingFace Hub, OpenAI, Groq, Anthropic, Google Generative AI	Multi-provider support
Observability	structlog, Prometheus, OpenTelemetry, Sentry	Full observability stack

Frontend

Component	Technology	Purpose
Framework	React 18	Modern UI framework
Language	TypeScript 5	Type safety
Build Tool	Vite	Fast development and builds
Styling	Tailwind CSS	Utility-first CSS
Routing	React Router DOM	Client-side routing

Infrastructure

Component	Technology	Purpose
Containerization	Docker, Docker Compose	Development and deployment
Monitoring	Prometheus, Grafana	Metrics and dashboards
Tracing	Jaeger	Distributed tracing
Caching	Redis	Session storage

📦 Application Pages & Features

Core Features

Dashboard - Overview of system metrics and activity
Inference - Run LLM inference with multiple providers
Tokens - Manage API keys and tokens securely
Telemetry - View analytics and usage metrics
Prompts - Create and manage prompt templates
Users - User management and administration

Cost Optimization

Cost Dashboard - Real-time cost tracking and analytics
Budgets - Set and manage spending budgets
Cost Forecasting - Predict future costs
Optimization Recommendations - AI-powered cost savings

Multi-Provider Intelligence

Provider Health - Monitor provider status and latency
Provider Comparison - Compare providers side-by-side
Failover - Configure automatic failover rules
A/B Testing - Test different providers and models
Smart Routing - Intelligent request routing
Model Abstraction - Unified model interface

Reliability & Performance

Queue Management - Manage request queues
Circuit Breakers - Configure failure protection
Load Balancing - Distribute load across providers
Reliability Dashboard - System reliability metrics

Security & Compliance

Security Audit - View security event logs
Security Compliance - Compliance dashboard
Compliance - Compliance reporting and controls
Data Residency - Configure data location requirements
DLP - Data Loss Prevention rules
Permissions - Fine-grained access control

Workspace Management

Workspaces - Multi-tenant workspace management
Projects - Project organization and isolation
Permissions - Role-based access control

Developer Tools

SDKs - Python and TypeScript SDK documentation
Webhooks - Configure webhook endpoints
Evaluation - Model evaluation framework
Prompt Optimization - AI-powered prompt improvement

Administration

Admin - System administration panel
Developer Settings - Developer configuration

🧪 Development

Backend Development

cd backend
pip install -e ".[test]"
uvicorn main:app --reload --host 0.0.0.0 --port 8000

Frontend Development

cd frontend
npm install
npm run dev

Running Tests

# Backend tests with coverage
cd backend
pytest --cov=. --cov-report=html

# Check coverage report
open htmlcov/index.html

Code Quality

# Backend
cd backend
black . && isort . && flake8 . && mypy .

# Frontend
cd frontend
npm run lint && npm run type-check

🚢 Deployment

Docker Compose (Development)

docker-compose up -d

Production Deployment

# Build production images
docker-compose -f docker/docker-compose.prod.yml build

# Deploy
docker-compose -f docker/docker-compose.prod.yml up -d

Environment Variables

See .env.example for required environment variables:

DATABASE_URL - PostgreSQL connection string
REDIS_URL - Redis connection string
SECRET_KEY - Session secret (generate strong random key)
SENTRY_DSN - Sentry error tracking (optional)
ALLOWED_ORIGINS - CORS allowed origins
ENCRYPTION_KEY - Token encryption key (generate with Fernet)

📊 Monitoring & Observability

Access Monitoring Tools

Grafana: http://localhost:3000 (admin/admin)
- Pre-configured dashboards for metrics
- Log visualization
Jaeger: http://localhost:16686
- Distributed tracing UI
- Request flow visualization
Prometheus: http://localhost:9090
- Metrics query interface
- Alert rule management
API Metrics: http://localhost:16000/metrics
- Prometheus-compatible metrics endpoint

Health Checks

Health: http://localhost:16000/health
Readiness: http://localhost:16000/ready

🔒 Security

Authentication

Session-based - HTTP-only cookies, Redis-backed
Password Hashing - bcrypt with salt
Session Expiration - Configurable timeout

Authorization

RBAC - Role-based access control (Admin/User)
Route Protection - Deny-by-default policy
Fine-grained Permissions - Per-resource access control
Workspace/Project Isolation - Multi-tenant security

Data Protection

Token Encryption - Fernet encryption at rest
Key Rotation - Automatic quarterly rotation
Security Audit Logs - Complete audit trail
PII Detection - Automatic detection and redaction
Data Loss Prevention - DLP rules and policies

🤝 Contributing

We welcome contributions! Please see our contributing guidelines:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Follow code standards (see CONTRIBUTING.md)
Write tests (aim for >80% coverage)
Commit changes (git commit -m 'Add amazing feature')
Push to branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Standards

Python: Black, isort, Flake8, mypy
TypeScript: ESLint, Prettier, strict mode
Tests: pytest for backend, comprehensive coverage
Documentation: Update README and docs for new features

📄 License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

🆘 Support

Website: https://aistrale.com
Documentation: docs/
Issues: GitHub Issues
Email: info@buildworks.ai

🌟 Implementation Status

✅ Completed Features (Q1 2025)

Core Platform (100% Complete)

✅ Multi-Provider LLM Support - 5 providers (HuggingFace, OpenAI, Groq, Anthropic, Gemini)
✅ Session-Based Authentication - Redis-backed sessions with HTTP-only cookies
✅ Token Management - Encrypted token storage with key rotation
✅ Telemetry Tracking - Complete request/response logging and analytics
✅ Prompt Management - Template creation, versioning, and variable substitution

Observability Stack (100% Complete)

✅ Structured Logging - structlog with JSON output and correlation IDs
✅ Prometheus Metrics - Custom business metrics + system metrics
✅ Distributed Tracing - OpenTelemetry with Jaeger integration
✅ Error Tracking - Sentry integration for error monitoring
✅ Health Checks - /health and /ready endpoints

Cost Intelligence (100% Complete)

✅ Cost Tracking - Real-time cost tracking per model/provider
✅ Cost Analytics - Dashboard with trends and breakdowns
✅ Budget Management - Set and monitor spending budgets
✅ Cost Forecasting - Predict future costs with confidence intervals
✅ Anomaly Detection - Automatic detection of cost anomalies
✅ Optimization Recommendations - AI-powered cost savings suggestions

Multi-Provider Intelligence (100% Complete)

✅ Provider Health Monitoring - Real-time health checks and latency tracking
✅ Provider Comparison - Side-by-side provider performance comparison
✅ Automatic Failover - Configure and manage failover rules
✅ A/B Testing - Test different providers and models
✅ Smart Routing - Intelligent request routing based on rules
✅ Model Abstraction - Unified model interface across providers

Reliability Features (100% Complete)

✅ Request Queuing - Queue management for traffic spikes
✅ Circuit Breakers - Failure protection and recovery
✅ Load Balancing - Distribute requests across providers
✅ Retry Logic - Exponential backoff with jitter

Security & Compliance (100% Complete)

✅ Security Audit Logging - Complete audit trail of all actions
✅ Role-Based Access Control - Admin/User roles with fine-grained permissions
✅ Workspace & Project Management - Multi-tenant architecture
✅ Data Loss Prevention - PII detection and redaction rules
✅ Compliance Dashboard - Compliance reporting and controls
✅ Data Residency - Configure data location requirements

Developer Experience (100% Complete)

✅ Webhooks - Real-time event notifications
✅ Evaluation Framework - Model evaluation and scoring
✅ Prompt Optimization - AI-powered prompt improvement
✅ SDKs Documentation - Python and TypeScript SDK guides

Frontend (100% Complete)

✅ 37 Application Pages - Complete UI for all features
✅ Protected Routes - Authentication and authorization
✅ Responsive Design - Mobile-friendly interface
✅ Dark Mode - Theme support

🔄 In Progress / Planned

Advanced Features (Architecture Designed)

🔄 Python SDK - Full SDK implementation (basic structure exists)
🔄 TypeScript SDK - Full SDK implementation (basic structure exists)
🔄 CLI Tool - Command-line interface (basic structure exists)
🔄 VS Code Extension - IDE integration (basic structure exists)
🔄 Framework Integrations - LangChain, LlamaIndex wrappers

Production Enhancements

🔄 CI/CD Pipeline - Automated testing and deployment
🔄 Production Docker Config - Optimized production containers
🔄 Monitoring Alerts - Alertmanager integration
🔄 Backup Strategy - Automated database backups

Status Legend:

✅ Completed - Fully implemented and production-ready
🔄 In Progress - Basic implementation exists, full features in development
📋 Planned - Architecture designed, implementation scheduled

See Planning Documentation for detailed implementation plans and Architecture Documentation for technical designs.

🙏 Acknowledgments

Built by BuildWorks.AI

AISTRALE - Turn AI from a black box into an engineered system

⬆ Back to Top

Built by BuildWorks.AI