KeMaSF/RAG_k8s

A. Local RAG → MCP Server

Took RAG code (vector.py, Local_RAG.py).

• Added mcp_server.py exposing search_reviews as an MCP tool.

• Added /healthz and /ready endpoints for probes.

• Dual-mode:

  • stdio mode (MCP_STDIO=1, default) for local MCP clients like Claude Desktop.
  • service mode (MCP_STDIO=0) for Docker/K8s.

service mode (MCP_STDIO=0) for Docker/K8s.

Run locally (stdio mode):

conda activate llm
export OLLAMA_HOST=http://localhost:11434
python mcp_server.py

# in another terminal:
curl http://localhost:8080/healthz

B. Dockerization

• Wrote a Dockerfile:
  • Installed deps.
  • Created non-root user.
  • Made /data/chroma writable.
  • Set ENV CHROMA_DIR=/data/chroma.
• Built & tagged image:

docker build -t rag-mcp:local .

Run container in service mode (with persistence & Ollama connectivity):

docker volume create chroma_data

# Linux
docker run --rm -p 8080:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e MCP_STDIO=0 \
  -e OLLAMA_HOST=http://host.docker.internal:11434 \
  -v chroma_data:/data/chroma \
  rag-mcp:local

# Mac/Windows (omit --add-host)

Test health:

curl http://localhost:8080/healthz

C. Kubernetes (local with kind)

• Created namespace, ConfigMap, Deployment, Service under k8s/.
• Deployment runs in MCP_STDIO=0 (service mode).
• Health probes check /healthz and /ready.

# 1. Create cluster
kind create cluster --name rag

# 2. Load local image into kind
kind load docker-image rag-mcp:local --name rag

# 3. Apply manifests
kubectl apply -f k8s/

# 4. Check
kubectl -n rag-dev get pods
kubectl -n rag-dev logs deploy/rag-mcp

# 5. Port-forward and test
kubectl -n rag-dev port-forward svc/rag-mcp 8080:80
curl http://localhost:8080/healthz

At this point: ✅ you have a working MCP microservice running on Kubernetes.