Paper Core (paper-core)

Central LLM Inference Gateway and gRPC Service for the Paper AI Ecosystem.

paper-core is the core inference gateway in the Paper multi-repo ecosystem. It abstracts underlying LLM provider APIs (OpenAI, MiniMax, Ollama, Anthropic, Groq) behind a high-performance gRPC interface, managing request routing, token streaming, rate limiting, and circuit breaking centrally for all callers.

---

Role in the Ecosystem

Caller Services (paper-caller / paper-mcp / paper-cli)
                         |
                         | gRPC Generate() or StreamGenerate()
                         v
                    paper-core (Port 50051)
                         |
           +-------------+-------------+
           |             |             |
        MiniMax       OpenAI        Ollama

• What it owns: LLM provider adapters, gRPC server lifecycle (AIService, AIStreamService), rate limiting per API key, circuit breaker state tracking, and request scheduling.
• What it does NOT do: paper-core does not handle audio processing (owned by paper-caller), session memory (owned by paper-memory), or tool execution (owned by paper-tools).

---

Key Features

• Provider Abstraction: Unified interface for cloud and local language models (MiniMax, OpenAI, Anthropic, Groq, Ollama).
• Token-by-Token gRPC Streaming: Server-streaming gRPC interface for sub-second first-token delivery.
• Resilience Controls: Built-in circuit breakers and exponential backoff retries with jitter via paper-common.
• Token Bucket Rate Limiting: Configurable rate limiting per API key or tenant ID.
• Prometheus Telemetry: Native metrics export on port 9090 for inference latency, token counts, and error rates.

---

Repository Structure

paper-core/
├── app/
│   ├── main.py                  # Service entry point and signal handling
│   ├── config.py                # Pydantic environment configuration
│   ├── engine/                  # AIEngine orchestrator
│   ├── grpc_server/             # gRPC servicers, interceptors, and channels
│   ├── providers/               # LLM provider adapters (MiniMax, OpenAI, etc.)
│   ├── scheduler/               # Priority request queue and scheduler
│   └── throttling/              # Token bucket rate limiter
├── tests/                       # Unit and integration test suite
├── Dockerfile                   # Production container definition
├── pyproject.toml               # Python dependencies and build config
└── README.md

---

Quickstart & Installation

Prerequisites

• Python 3.12+
• Docker & Docker Compose (optional for containerized setup)

Local Setup

# Clone repository
git clone https://github.com/artificialpaper/paper-core.git
cd paper-core

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .

# Copy environment template
cp .env.example .env

Running the gRPC Server

# Start gRPC inference server on port 50051
python -m app.main

---

Configuration Reference

paper-core is configured via environment variables or a .env file:

Variable	Type	Default	Description
PAPER_PORT	int	50051	gRPC server listening port
PAPER_METRICS_PORT	int	9090	Prometheus metrics HTTP port
LLM_PROVIDER	string	minimax	Active provider key (minimax, openai, ollama)
MINIMAX_API_KEY	string	""	MiniMax API secret key
OPENAI_API_KEY	string	""	OpenAI API secret key
OLLAMA_HOST	string	http://localhost:11434	Local Ollama server address
RATE_LIMIT_TOKENS_PER_SEC	int	100	Token bucket refill rate

---

gRPC API Reference

paper-core implements contracts defined in paper-proto:

Unary Generation (AIService.Generate)

rpc Generate(GenerateRequest) returns (GenerateResponse);

Server Streaming (AIStreamService.StreamGenerate)

rpc StreamGenerate(GenerateRequest) returns (stream StreamChunk);

---

Testing & Quality

# Run pytest test suite
pytest

# Run linter and type checker
ruff check .
mypy app

---

Related Repositories & Documentation

• paper-caller - Real-time Voice AI Agent Runtime
• paper-common - Shared Platform SDK
• paper-proto - gRPC Protobuf Contracts
• paper-docs - Official Ecosystem Documentation Portal

---

License

This project is licensed under the MIT License - see the LICENSE file for details.