LLM Privacy Proxy（大语言模型隐私代理网关）
LLM Privacy Proxy is a lightweight control plane that places a privacy and governance boundary between applications and large language models. It preserves an OpenAI-compatible interface while masking sensitive data before it leaves your environment.
Why a Privacy Gateway Matters（为什么需要隐私网关）
Production AI adoption is also a data-boundary problem. Applications may send email addresses, phone numbers, national IDs, bank cards, or domain-specific identifiers to an upstream provider. This gateway makes the boundary explicit and enforceable: authenticate the client, mask sensitive values, call the model, restore the response, and record lightweight audit metadata.
Architecture（系统架构）
The gateway creates a controlled privacy boundary between the client and the upstream model. Sensitive values are masked before transmission and restored only after the response returns to the gateway.
sequenceDiagram
participant Client as OpenAI SDK / Client
participant Gateway as Privacy Gateway
participant Audit as SQLite Audit DB
participant LLM as Upstream LLM Provider
Client->>Gateway: POST /v1/chat/completions + Bearer token
Gateway->>Gateway: Validate proxy key
Gateway->>Gateway: Detect and replace PII
Gateway->>LLM: Send masked prompt
LLM-->>Gateway: Return model response
Gateway->>Gateway: Restore masked values
Gateway-->>Client: Return OpenAI-compatible response
Gateway-->>Audit: Write async request metadata
Data boundary: the upstream provider receives the masked prompt; the original sensitive values remain inside the gateway’s request-scoped masking session.
Mermaid Source（架构图源码）
sequenceDiagram
participant Client as OpenAI SDK / Client
participant Gateway as Privacy Gateway
participant Audit as SQLite Audit DB
participant LLM as Upstream LLM Provider
Client->>Gateway: POST /v1/chat/completions + Bearer token
Gateway->>Gateway: Validate proxy key
Gateway->>Gateway: Detect and replace PII
Gateway->>LLM: Send masked prompt
LLM-->>Gateway: Return model response
Gateway->>Gateway: Restore masked values
Gateway-->>Client: Return OpenAI-compatible response
Gateway-->>Audit: Write async request metadataCore Features（核心功能）
- OpenAI-compatible endpoint: existing clients can call
/v1/chat/completionswith minimal changes. - Request-scoped PII masking: email, phone, national ID, bank card, and patient identifiers are replaced with temporary placeholders.
- Response restoration: placeholders are restored only within the same request session.
- Bearer authentication: a valid proxy token is required before model invocation.
- SQLite audit trail: request time, proxy identity, resolved model, and masked-item count are recorded asynchronously.
- Private deployment: run locally, inside a controlled network, or as a containerized service.
Request Lifecycle（请求生命周期）
- The client sends a chat request with a Bearer token.
- The gateway validates the request and creates an isolated masking session.
- Supported identifiers are replaced with placeholders before transmission.
- The masked payload is forwarded to the configured provider.
- The response is restored and returned in an OpenAI-compatible shape.
- Non-content audit metadata is written asynchronously to SQLite.
Technology Stack（技术栈）
Backend: FastAPI and Python
Provider: Gemini API through a provider abstraction
Storage: SQLite
Deployment: Docker Compose
Quick Start（快速开始）
cp .env.example .env
# Set GEMINI_API_KEY and PROXY_KEY in .env
docker compose up --build -dThe service exposes /health for health checks and /docs for interactive API documentation.
API Example（接口示例）
curl http://localhost:8000/v1/chat/completions
-H "Content-Type: application/json"
-H "Authorization: Bearer sk-proxy-client-001"
-d '{
"model": "gemini-3.6-flash",
"messages": [{"role": "user", "content": "Please review this text."}]
}'Security Boundary（安全边界）
The gateway reduces accidental exposure, but it is not a complete compliance system by itself. Production deployments should replace the shared-key mechanism with an identity provider, rotate secrets through a secret manager, define audit retention policies, restrict network egress, and expand PII detection for the target regulatory and business context.