fileStore

bdogstudio/fileStore

3.1

If you are the rightful owner of fileStore and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to dayong@mcphub.com.

The Model Context Protocol (MCP) server is a specialized server designed to facilitate AI integration by providing a structured protocol for managing model contexts and interactions.

FileStore - Self-Hosted Storage Infrastructure

A comprehensive, scalable, open-source file storage solution with AI integration, designed for on-premises deployment with encrypted cloud backup.

Overview

FileStore is a self-hosted storage system built on open-source technologies, providing:

  • Nextcloud for file management and sharing
  • Headscale (open-source Tailscale) for secure networking
  • MCP Server for AI system integration
  • Restic for encrypted backups
  • ZFS/Btrfs for advanced filesystem features
  • Scalability from 1.5TB to 100TB+

Quick Start

Prerequisites

  • Linux server (Ubuntu 22.04+ or similar)
  • Docker and Docker Compose installed
  • At least 2TB storage available at /mnt/storage
  • Python 3.8+ for setup scripts

Installation

  1. Clone the repository
cd /home/mcrandall-admin/projects/fileStore
  1. Run the setup script
chmod +x setup.py
sudo python3 setup.py
  1. Configure environment
cp .env.template .env
# Edit .env with your settings
nano .env
  1. Start services
docker compose up -d
  1. Access Nextcloud
  • URL: http://your-server-ip:8080
  • Credentials: See admin_password.txt

Project Structure

fileStore/
├── docker-compose.yml           # Main Nextcloud stack
├── setup.py                     # Automated setup script
├── .env.template               # Environment configuration template
├── backup.sh                   # Basic backup script
├── encrypted-backup.sh         # Restic encrypted backup
├── headscale-setup.sh         # Headscale VPN setup
├── mcp-server/                # MCP server for AI integration
│   ├── mcp_filestore_server.py
│   ├── docker-compose.yml
│   └── requirements.txt
├── STORAGE_ARCHITECTURE_PLAN.md  # Overall architecture
├── SECURITY_ARCHITECTURE.md      # Security & self-hosting details
├── SCALING_STRATEGY.md          # Growth planning (1.5TB → 100TB+)
└── MIGRATION_PLAN.md            # Data migration guide

Key Features

🔒 Security First

  • All services self-hosted on premises
  • Open-source software only
  • Encrypted at rest (ZFS/LUKS)
  • Encrypted in transit (WireGuard/TLS)
  • Client-side encrypted cloud backups

🚀 Scalable Architecture

  • Phase 1: Single server (1.5TB - 10TB)
  • Phase 2: JBOD expansion (10TB - 50TB)
  • Phase 3: Distributed storage with Ceph (50TB - 100TB+)

🤖 AI Integration

  • MCP server for Model Context Protocol
  • WebDAV for ComfyUI integration
  • S3-compatible API via MinIO
  • Secure API endpoints for AI systems

💾 Backup Strategy

  • Local ZFS/Btrfs snapshots
  • Encrypted Restic backups
  • Multiple cloud provider support
  • Automated backup scheduling

Storage Layout

/mnt/storage/filestore/
├── nextcloud-data/      # Nextcloud files
├── ai-workspace/        # AI systems storage
│   ├── comfyui/        # ComfyUI projects
│   ├── mcp-data/       # MCP server data
│   └── models/         # AI models
├── personal/           # Personal files
├── notes/             # Notes and documents
└── shared/            # Shared resources

Services

Nextcloud (Port 8080)

Primary file management interface with:

  • Web interface
  • Desktop/mobile sync
  • WebDAV support
  • Collaborative editing

Headscale VPN (Port 8080)

Self-hosted Tailscale coordination server:

  • WireGuard-based mesh VPN
  • Zero-trust networking
  • No external dependencies

MCP Server (Port 8888)

AI integration server providing:

  • REST API for file operations
  • JWT authentication
  • Redis caching
  • S3-compatible interface

MinIO (Port 9000/9001)

S3-compatible object storage:

  • AI model storage
  • Large file handling
  • S3 API compatibility

Management Commands

Service Management

# Start all services
docker compose up -d

# View logs
docker compose logs -f

# Stop services
docker compose down

# Restart specific service
docker compose restart nextcloud

Backup Operations

# Run encrypted backup
./encrypted-backup.sh backup

# List snapshots
./encrypted-backup.sh list

# Restore from backup
./encrypted-backup.sh restore <snapshot-id> /restore/path

# Mount backup for browsing
./encrypted-backup.sh mount /mnt/restic

Headscale VPN

# List users
headscale-admin users

# Create new user
headscale-admin create-user <username>

# Generate auth key
headscale-admin preauthkey <username>

# List connected nodes
headscale-admin nodes

Data Migration

# Analyze source data
./analyze_source_data.sh /path/to/source

# Start migration
./migrate_data.sh

# Verify migration
python3 verify_migration.py /source /destination

AI Integration

ComfyUI WebDAV Mount

# Mount on Linux/Mac
mount -t davfs http://filestore.local:8080/remote.php/webdav/ai-workspace/comfyui /mnt/comfyui

# Windows (map network drive)
\\filestore.local@8080\remote.php\webdav\ai-workspace\comfyui

MCP API Usage

import requests

# Get token
response = requests.post('http://filestore.local:8888/auth/token', json={
    'user_id': 'ai_system',
    'password': 'your_password'
})
token = response.json()['token']

# Read file
headers = {'Authorization': f'Bearer {token}'}
response = requests.get('http://filestore.local:8888/files/models/model.pt', headers=headers)

Monitoring

Check System Health

# Storage usage
df -h /mnt/storage

# Service status
docker compose ps

# Nextcloud status
docker exec -u www-data nextcloud php occ status

# Backup status
tail -f /mnt/storage/backups/backup.log

Performance Metrics

Access Grafana dashboard at http://your-server:3000 (if monitoring stack deployed)

Troubleshooting

Nextcloud Issues

# Run maintenance repair
docker exec -u www-data nextcloud php occ maintenance:repair

# Clear cache
docker exec -u www-data nextcloud php occ files:cleanup

# Rescan files
docker exec -u www-data nextcloud php occ files:scan --all

Permission Issues

# Fix Nextcloud permissions
docker exec nextcloud chown -R www-data:www-data /var/www/html/data

# Fix storage permissions
sudo chown -R $USER:$USER /mnt/storage/filestore

Backup Recovery

# List available snapshots
./encrypted-backup.sh list

# Restore specific file
restic restore latest --path /path/to/file --target /tmp/restore

Security Considerations

  1. Change all default passwords in .env
  2. Enable firewall - only expose necessary ports
  3. Regular updates - Keep all containers updated
  4. Monitor logs - Check for suspicious activity
  5. Test backups - Regularly verify backup integrity

Support

For issues or questions:

  1. Check documentation in project files
  2. Review Docker logs: docker compose logs
  3. Verify system requirements are met
  4. Ensure storage permissions are correct

License

This project uses open-source software. Individual components are subject to their respective licenses:

  • Nextcloud: AGPLv3
  • Headscale: BSD 3-Clause
  • MinIO: AGPLv3
  • Restic: BSD 2-Clause