Free LLM Gateway
An OpenAI-compatible gateway that aggregates multiple free large model providers. It is suitable for uniformly managing keys, viewing available models, and connecting multiple AI clients to the same `/v1` entry in lazy cat.
##Functional characteristics - Unified OpenAI compatibility portal: Aggregates multiple free model providers into the same `/v1` Base URL to facilitate unified access by clients such as Cursor, Open WebUI, LibreChat, and OpenAI SDK. - Multi-Provider model aggregation: Supports multiple free model sources such as OpenRouter, GitHub Models, Groq, Cerebras, Google Gemini, Mistral, SiliconFlow, etc., and does rollback according to the link defined by `models.yaml`. - Intelligent fallback and polling: When a Provider fails, times out, or triggers a current limit, the gateway will automatically switch to the backup model and poll allocation requests in multi-key scenarios. - Streaming and batch invocation: Compatible with streaming SSE output, and also supports `/v1/batch` to forward multiple requests concurrently, suitable for batch testing of scripts or tools. - Console management: Built-in Dashboard that allows you to view Provider health status, cache hit rates, request logs, benchmarks, connection information and recommendation models. - Run-time key management: Supports adding, verifying, and deleting Provider API keys in the console, and encrypting and saving the new keys added during runtime to the persistent directory. - Usage and savings statistics: Automatically record the request amount, Token consumption, Provider success rate and approximate savings amount to facilitate daily observation of usage. ###First use process 1. Set a master_key that you can remember in the installation wizard 2. If you plan to use Cloudflare Workers AI, fill in additional `cloudflare_account_id` 3. Open the Keys tab of the console 4. Add at least one available Provider Key, such as openrouter, github, groq, or google_gemini 5. Click the Verify button to confirm that the corresponding Provider displays an available or at least no longer has an empty configuration status 6. Go back to the Models or Setup page and verify that you can see the list of available models ###How to obtain third-party Provider keys - `OpenRouter`: Open [openrouter.ai/keys](https://openrouter.ai/keys), log in and create a Key. The advantage is that the free model catalog is the most complete, and it is usually the most suitable for matching the first key first. - `GitHub Models`: Open [github.com/settings/tokens](https://github.com/settings/tokens) to generate Personal Access Tokens. Suitable for users who already have a GitHub account, getting the Key is relatively straightforward. - `Groq`: Open [console.groq.com/keys](https://console.groq.com/keys) to create an API Key. It is characterized by high speed and is often suitable for daily chatting and code assistants. - `Google Gemini`: Open [aistudio. google.com/apikey](https://aistudio. google.com/apikey) to create an API Key. A Google account is required, and free credit is not available in some regions. - `Cerebras`: Open [cloud.cerebras.ai](https://cloud.Cerebras.ai/) to register and enter the console to obtain the Key. Suitable for users who want to test ultra-high output speeds. - `Mistral AI`: Open [console.mistral.ai/api-keys](https://console.mistral.ai/api-keys) to create a Key. Suitable for supplementing the Mistral / Codestral series models. - `SiliconFlow`: Open [cloud.siliconflow.cn/account/ak](https://cloud.siliconflow.cn/account/ak) to create an API Key. - `Hugging Face`: Open [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) to create the `Read` permission Token. Use when supplementing the community model as appropriate. - `Cloudflare Workers AI`: Open [dash.cloudflare.com/profile/api-tokens](https://dash.cloudflare.com/profile/api-tokens) to create a Token, and also prepare the `Account ID'. If you don't fill in 'cloudflare_account_id' during installation, don't match it here for now. - `Cohere`: Open [dashboard.cohere.com/api-keys](https://dashboard.cohere.com/api-keys) to get the trial Key. Pay attention to trial limits and commercial restrictions. - `NVIDIA NIM`: Open [build.nvidia.com/explore/discover](https://build.nvidia.com/explore/discover) to register and join the NVIDIA Developer Program to generate a Key. There are many models and friendly quotas. - `LLM7`: Open [token.llm7.io](https://token.llm7.io). It supports basic access without registration, but if you want to increase the speed limit, it is still recommended to generate tokens and fill them in the console. - `Olama Cloud`: Open [ollama.com/settings/keys](https://ollama.com/settings/keys) to generate keys. Suitable for when you want to supplement the source of the Ollama Cloud model. Note: Some three-party providers need to be used in conjunction with Page Piggy, such as Gemini.
