doctor
If you are the rightful owner of doctor and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.
Doctor is a tool designed to crawl, index, and expose websites as an MCP server for LLM agents, enhancing reasoning and code generation capabilities.
Doctor provides a comprehensive solution for crawling and indexing web pages, making them accessible to LLMs through an MCP server. It leverages a stack that includes DuckDB for data storage and vector search, Redis for task processing, and FastAPI for web service exposure. The tool is designed to facilitate the discovery and navigation of web content, offering hierarchical site maps and search functionalities. By integrating with OpenAI's API for creating embeddings, Doctor ensures that the indexed data is both relevant and up-to-date, supporting better reasoning and code generation by LLM agents. The system is built to be robust and scalable, utilizing Docker for deployment and offering a user-friendly interface through its web API.
Features
- Crawling and Indexing: Uses crawl4ai for web page crawling with hierarchy tracking.
- Text Chunking: Utilizes LangChain for efficient text chunking.
- Embeddings Creation: Integrates with OpenAI via litellm for creating embeddings.
- Data Storage: Employs DuckDB for storing document data with vector search support.
- Web Service Exposure: Provides a FastAPI service for data fetching, searching, and viewing.
Tools
MCP Server
Exposes capabilities to LLMs through an MCP server.