redaction-compliance-MCP

sunkencity999/redaction-compliance-MCP

3.2

If you are the rightful owner of redaction-compliance-MCP and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to dayong@mcphub.com.

The Redaction & Compliance MCP Server is a robust solution for managing sensitive data in LLM calls, ensuring compliance and security through advanced detection, policy enforcement, and audit logging.

Redaction & Compliance MCP Server

Production-Ready Edition | Version 2.0

This repository contains a production-grade implementation of a Redaction & Compliance Model Context Protocol (MCP) server. It provides a pre-flight / post-flight firewall for LLM calls with comprehensive detection, classification, policy enforcement, reversible redaction, selective detokenization, output safety, and immutable audit logging.

✨ Features

🎯 Core Capabilities

  • 🔄 Streaming Support: Real-time streaming for OpenAI, Claude, and Gemini with chunk-by-chunk detokenization
  • 🛡️ Claim Verification: Research-based hallucination detection with inline warnings (supports local models)
  • 🚀 Transparent Proxy: Zero-code integration - just change your API base URL
  • 📊 Production-Grade: NGINX, HTTPS, SIEM integration, Redis backend, systemd service

🔍 Advanced Detection

  • Multi-cloud credentials: AWS (AKID, secrets), Azure (Storage Keys, SAS tokens, Connection Strings), GCP (API keys, OAuth)
  • OAuth & Bearer tokens: JWT detection, OAuth access tokens
  • Crypto keys: PEM (RSA, DSA, EC), PKCS#12, Kubernetes config/tokens
  • PII with validation:
    • Credit cards with Luhn checksum validation
    • SSN with format validation (rejects invalid area codes 000, 666, 900-999)
    • Email addresses and phone numbers
  • Internal infrastructure: Joby Aviation domains (*.na.joby.aero, *.az.joby.aero), IP addresses, hostnames
  • Export control: Aviation keywords (eVTOL, ITAR, FAA certification, flight control systems, propulsion)

🛡️ Policy Engine

  • Geo/region constraints: US, EU, APAC, restricted regions (CN, RU, IR, KP, SY)
  • Caller-based routing: Trusted caller lists, per-caller detokenization permissions
  • Data residency: EU GDPR compliance, region-specific model routing
  • Category actions: block, redact, internal_only, allow
  • Version tracking: Policy version embedded in all decisions

🔐 Token Store

  • In-memory: Fast, stateless, for dev/test
  • Redis with AES-GCM: Production-grade with encryption at rest
    • AES-256-GCM encryption
    • PBKDF2 key derivation
    • Automatic TTL management
  • Deterministic placeholders: «token:TYPE:HASH4» stable within conversation scope

⚠️ Output Safety

  • 50+ dangerous command patterns: Filesystem destruction, system control, K8s/Docker, databases, cloud infra, network/firewall
  • External config support: JSON-based custom pattern loading
  • 3 modes: warning (annotate), block (redact), silent (pass-through)

📊 Audit & Compliance

  • Append-only JSONL: Immutable audit trail
  • Full context capture: Caller, region, categories, decisions, redaction counts
  • Query API: Search and retrieve audit records
  • SIEM integration: Real-time shipping to Splunk, Elasticsearch, Datadog, Syslog
  • Buffered shipping: <5% overhead, batch mode for production

🚀 Production Installation (5 Minutes)

Automated installer with NGINX, HTTPS, and Client SDK:

# On your Linux server (Ubuntu 20.04+ or RHEL 8+)
wget https://raw.githubusercontent.com/sunkencity999/redaction-compliance-MCP/main/install.sh
chmod +x install.sh
sudo ./install.sh

What it does:

  • ✅ Installs all dependencies (Python 3.11, Redis, NGINX)
  • ✅ Generates cryptographic secrets (you backup them)
  • ✅ Configures SIEM integration (Splunk/Elasticsearch/Datadog)
  • ✅ Sets up NGINX reverse proxy with HTTPS (Let's Encrypt or self-signed)
  • ✅ Creates systemd service (auto-start on boot)
  • ✅ Installs Python Client SDK
  • ✅ Creates integration examples
  • ✅ Runs full test suite (186+ tests)

Manual installation: See QUICKSTART.md

📦 Client SDKs

Python SDK

Seamless integration for Python/backend applications:

# Install SDK (included in automated installer)
pip install -e .

Usage:

from mcp_client import MCPClient, MCPConfig

# Configure once
mcp = MCPClient(MCPConfig(
    server_url="https://mcp.yourcompany.com",
    caller="your-app-name"
))

# Protect LLM calls automatically
user_input = "My AWS key is AKIAIOSFODNN7EXAMPLE, help me debug"

# Redact before sending to LLM
sanitized, handle = mcp.redact(user_input)
# sanitized: "My AWS key is «token:SECRET:a3f9», help me debug"

# Send sanitized version to OpenAI/Claude/etc
llm_response = your_llm_function(sanitized)

# Restore non-secret tokens
final = mcp.detokenize(llm_response, handle)

# Secrets stay tokenized, PII/ops_sensitive restored!

Or use the convenience wrapper:

from mcp_client import MCPClient, MCPConfig

mcp = MCPClient(MCPConfig.from_env())

# One-line protection
response = mcp.safe_llm_call(
    user_input,
    lambda text: openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": text}]
    ).choices[0].message.content
)

Examples: See examples/ directory after installation

JavaScript/Browser SDK

For web applications, React, Vue, Angular:

<!-- Include SDK -->
<script src="mcp_client_js/mcp-client.js"></script>

<script>
// Initialize client
const mcp = new MCPClient({
    serverUrl: 'https://mcp.yourcompany.com',
    caller: 'web-app'
});

// Protect browser-based LLM calls
async function safeChatCompletion(userInput) {
    const response = await mcp.safeLLMCall(
        userInput,
        async (sanitized) => {
            // Call OpenAI/Claude from browser
            return await callYourLLM(sanitized);
        }
    );
    return response;
}
</script>

React Example:

import { MCPClient } from './mcp-client.js';

const mcp = new MCPClient({
    serverUrl: process.env.REACT_APP_MCP_SERVER,
    caller: 'react-app'
});

function ChatComponent() {
    const handleSubmit = async (input) => {
        try {
            const response = await mcp.safeLLMCall(input, callOpenAI);
            setMessages(prev => [...prev, response]);
        } catch (error) {
            if (error instanceof MCPBlockedError) {
                alert('Request blocked: contains sensitive data');
            }
        }
    };
    // ... rest of component
}

TypeScript supported: See mcp_client_js/mcp-client.d.ts

Examples: See mcp_client_js/examples/ for browser and React demos

🔄 Transparent Proxy Mode (NEW!)

Zero-code integration for existing OpenAI/Claude/Gemini apps:

Just change your API base URL and MCP automatically protects all calls!

import openai

# Change this one line:
openai.api_base = "https://mcp.yourcompany.com/v1"

# Your existing code works unchanged!
response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "My AWS key is AKIA..."}]
)
# MCP automatically redacts before OpenAI sees it!

Supported Providers:

  • ✅ OpenAI (/v1/chat/completions) - Streaming supported
  • ✅ Claude (/v1/messages) - Streaming supported
  • ✅ Gemini (/v1/models/{model}:generateContent) - Streaming supported

Features:

  • ✅ Real-time streaming with chunk-by-chunk detokenization
  • ✅ Optional claim verification (hallucination detection)
  • ✅ Local model support (vLLM, Ollama, FastAPI)
  • ✅ Automatic redaction + detokenization
  • ✅ Full audit trail in SIEM

Setup:

# In .env file
PROXY_MODE_ENABLED=true
CLAIM_VERIFICATION_ENABLED=false  # Optional
DETOKENIZE_TRUSTED_CALLERS=openai-proxy,claude-proxy,gemini-proxy

Full Guides:

  • TRANSPARENT_PROXY.md - Proxy mode documentation
  • CLAIM_VERIFICATION.md - Hallucination detection guide

🛡️ Claim Verification (Hallucination Detection)

Optional post-processing layer to verify factual accuracy of LLM responses:

Using a research-based approach, this feature analyzes LLM responses through a 4-stage pipeline to detect and flag potential hallucinations and false claims.

# Enable in .env
CLAIM_VERIFICATION_ENABLED=true
CLAIM_VERIFICATION_MODEL=gpt-4o-mini  # Or local model

# Use any LLM normally via transparent proxy
response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What was Argentina's inflation in 2023?"}]
)

# If LLM hallucinates a wrong number, you'll see:
print(response.choices[0].message.content)
# "Argentina's inflation reached 300% in 2023."
# ⚠️ **[CLAIM FLAGGED - HIGH CONFIDENCE]**: This claim is likely false. 
# Evidence suggests Argentina's inflation was approximately 211% in 2023.

4-Stage Verification Pipeline:

  1. Sentence Splitting - Break response into sentences with context
  2. Selection - Filter to verifiable factual claims
  3. Disambiguation - Resolve or flag ambiguous statements
  4. Decomposition - Extract atomic, standalone claims
  5. Verification - Fact-check each claim with confidence scores

Output Modes:

  • Inline Warnings: 🚨 High, ⚠️ Medium, ℹ️ Low confidence flags added to text
  • Metadata: Full verification details in mcp_verification response field
  • No Blocking: Users always see full response + warnings (inform, don't censor)

Local Model Support:

# Use vLLM, Ollama, or FastAPI locally (no API fees, full privacy)
CLAIM_VERIFICATION_BASE_URL=http://localhost:8000/v1
CLAIM_VERIFICATION_MODEL=meta-llama/Meta-Llama-3.1-8B-Instruct
CLAIM_VERIFICATION_REQUIRE_AUTH=false  # No authentication needed

Use Cases:

  • ✅ Technical/Engineering - Verify calculations, formulas, specifications
  • ✅ Scientific - Fact-check research claims, data, constants
  • ✅ Financial - Validate statistics, market data, economic claims
  • ✅ Medical - Verify dosages, symptoms, treatments (strict mode)

Performance:

  • Latency: ~500-1000ms per response (cloud) or ~300ms (local)
  • Cost: ~$0.0003/response with gpt-4o-mini, $0 with local models
  • Caching: ~80% hit rate reduces both latency and cost

Full Guide: See CLAIM_VERIFICATION.md for complete setup, configuration, and examples.

🌐 API Endpoints (REST)

Core MCP Endpoints:

  • GET /health → server health check
  • POST /classify → classify payload sensitivity
  • POST /redact → sanitize payload, return token_map_handle
  • POST /detokenize → reinject allowed tokens (trusted clients only)
  • POST /route → produce an execution plan (internal/external, redaction steps)
  • POST /audit/query → simple audit search

Transparent Proxy Endpoints (when PROXY_MODE_ENABLED=true):

  • POST /v1/chat/completions → OpenAI-compatible proxy
  • POST /v1/messages → Claude-compatible proxy
  • POST /v1/models/{model}:generateContent → Gemini-compatible proxy

Full API documentation: See mcp_redaction/models.py for request/response schemas.

Policy

Edit mcp_redaction/sample_policies/default.yaml. Hot-reload on change is supported (watcher optional).

Stdio / JSON-RPC (MCP) Adapter

See mcp_redaction/stdio_adapter.py for a minimal adapter skeleton you can mount under an agent runtime.

Testing

pytest -q

Production Hardening

  • Run behind mTLS and identity-aware proxy
  • Use Redis (or KV with envelope encryption) for token maps
  • Ship audit logs to SIEM (Splunk/ELK); rotate JSONL files
  • Add OPA/Gatekeeper check on detokenize categories
  • Extend detectors (NER, export-control classifier), add OCR for attachments
  • Enforce geo-routing and model-allow lists in policy.yaml