SimapleLLMDispatch
Lightweight LLM API intelligent scheduling agent, compatible with OpenAI API format, unifies multiple LLM vendors into one entry point.
Usage method: ###Add LLM suppliers 1. After entering the Dashboard, click ** Add Vendor ** 2. Fill in supplier information: - ** Name **: Supplier identification (such as `deepseek`,`openai`) - **Base URL**: API address (such as `api.deepseek.com`) - **API Key**: Corresponding API key 3. Click "Get Model" to automatically pull the list of available models 4. Select the model to enable, set weights and quotas 5. save the configuration ###Using Scheduling Agents Just point your LLM client's base_url to this service: ``` https://simaplellmdispatch. <boxname>.heiyu.space/v1 ``` Note: API Key can be filled in at will ##Main functions - **OpenAI compatible ** -You can use it directly by replacing 'base_url' and supporting '/v1/chat/completions' and '/v1/models' - ** Multi-vendor management ** -simultaneous access to multiple vendors such as OpenAI, Claude, DeepSeek, and Gemini - ** Intelligent scheduling ** -Supports weighted random and round-robin scheduling strategies - ** Multi-level quota control ** -RPD/RPM/TPM/total request/total Token limit at vendor level + model level - ** Powerful failover ** -Automatic retry, silent failure detection, null response protection - ** Health check and cooling ** -Automatic cooling for continuous failures, supporting custom cooling duration - ** Streaming response ** -Full support for SSE streaming output - ** Visualization Dashboard** -Glass texture dark theme management panel
