Connect vLLM

Use the public text worker as a sidecar for a vLLM server you already operate. The common local base endpoint is http://127.0.0.1:8000; vLLM exposes its models through /v1/models.

Install or Start vLLM

Keep the vLLM deployment and launch configuration you already use. New operators should follow the official vLLM documentation for their accelerator and choose the model, tensor parallelism, context length, and API authentication before adding the Grid worker. The worker does not manage the vLLM process or its model files.

Detect the Endpoint

Start vLLM using its own reviewed deployment configuration. Confirm that its OpenAI-compatible models endpoint responds locally:

curl --fail http://127.0.0.1:8000/v1/models

Do not place a backend credential in a public command, screenshot, or issue.

Compatible Models

The worker lists the exact model identifiers returned by /v1/models and uses vLLM’s OpenAI-compatible generation endpoint. Select only the model you intend to expose. A successful inventory request is not enough: the wizard’s local generation test must produce output. Optional native request formats are advertised only after their endpoint probes succeed.

Connect and Configure

1. Download the verified text worker from aipowergrid.io/run.
2. Select detected vLLM, or enter http://127.0.0.1:8000 as the backend base URL. The wizard adds the OpenAI /v1 API prefix when it saves setup.
3. If your server requires a key, enter it only in the local worker UI.
4. Select the exact model returned by vLLM and run the local generation test.
5. Set conservative context, output, concurrency, and schedule limits.
6. Approve the worker in Console, start it, and wait for Online.

Operator Controls

Begin with one Grid job. Raise concurrency only when the backend has measured headroom and the credential path supports parallel worker slots. vLLM’s model memory allocation is not proof that its request queue is idle.

The selected backend has its own Grid model name, context/output limits, concurrency, and optional local-time schedule. A zero-concurrency window pauses new Grid claims and the worker drains active work before scaling down.

Security Boundary

Use loopback unless the backend intentionally lives on an operator-controlled LAN or HTTPS endpoint. The Grid connection is outbound. Backend credentials are stored only in the protected local worker configuration and are not returned to the browser after saving. Never provide a wallet private key. Community workers can inspect plaintext prompts and outputs.

Verify on the Grid

Wait for the dashboard’s Online state, then verify the exact worker name or ID and advertised model using the public check on aipowergrid.io/run. A running local process alone does not prove that Core accepted it.

Troubleshooting

• Not detected: verify /v1/models on the exact host and port, then enter that base URL manually if vLLM is not using port 8000.
• Unauthorized: enter the vLLM API key only in the local worker UI.
• Generation test fails: test the same model identifier directly against vLLM and inspect the worker’s local logs.
• Latency rises for local clients: reduce Grid concurrency or add pause windows. Automatic scheduler-idle detection is not implemented.