Ray LLM

LLM workloads on open-source Ray (pinned to 2.57) — OpenAI-compatible serving with ray.serve.llm (vLLM-backed LLMConfig + build_openai_app) and batch inference with ray.data.llm (build_processor). Corrects the stale defaults a model produces — the archived ray-llm repo and its YAML configs, hand-rolled vLLM engines inside plain Serve deployments, the removed build_llm_processor name, deprecated boolean stage flags, top-level LLMServer/LLMRouter imports, free-form accelerator strings, one-deployment-per-LoRA-adapter designs — with the 2.57 idioms (stage configs, placement_group_config over hand

228 stars 19 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add pproenca/dot-skills --skill ray-llm
Manual install
git clone --depth 1 https://github.com/pproenca/dot-skills.git /tmp/dot-skills
mkdir -p ~/.claude/skills
cp -r /tmp/dot-skills/skills/.experimental/ray-llm ~/.claude/skills/ray-llm

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Ray LLM

Library-reference skill for LLM workloads on open-source Ray — 13 rules across 5 categories covering `ray.serve.llm` (OpenAI-compatible, vLLM-backed serving) and `ray.data.llm` (batch inference). This surface churned faster than any other part of Ray — a standalone repo was absorbed and archived, entry points were renamed, and config shapes restructured — so the examples a model learned from mostly no longer run. Each rule names the wrong default it corrects; there is no rule for things a capable model already gets right.

Scope is the LLM-specific layer. Generic Serve/Data/cluster decisions (deployment lifecycle, autoscaling semantics, KubeRay) are the sibling [`ray`](../ray/) skill — the two compose.

Pinned to **ray 2.57.0** (`ray[llm]` extra, which pins its matching vLLM). API claims were verified against the unpacked 2.57.0 wheel and the installed package source, and every config example in the rules was constructed under CPU-only pydantic validation (including the traps, which fail exactly as described); engine/GPU runtime behavior is source-verified only — no model was actually served.

## When to Apply

- Standing up or reviewing an OpenAI-compatible LLM serving deployment on Ray
- Writing batch LLM inference over datasets — summarization, embedding, scoring at scale
- Sizing or placing multi-GPU models — tensor/pipeline parallelism, accelerator selection
- Scaling LLM deployments — replica autoscaling, ingress sizing, request routing
Read the full skill on GitHub

Details

Skill name
ray-llm
Path
skills/.experimental/ray-llm/SKILL.md
License
MIT
Repo updated
15 August 2026

FAQ

What does the Ray LLM skill do?

LLM workloads on open-source Ray (pinned to 2.57) — OpenAI-compatible serving with ray.serve.llm (vLLM-backed LLMConfig + build_openai_app) and batch inference with ray.data.llm (build_processor). Corrects the stale defaults a model produces — the archived ray-llm repo and its YAML configs, hand-rolled vLLM engines inside plain Serve deployments, the removed build_llm_processor name, deprecated boolean stage flags, top-level LLMServer/LLMRouter imports, free-form accelerator strings, one-deployment-per-LoRA-adapter designs — with the 2.57 idioms (stage configs, placement_group_config over hand

How do I install the Ray LLM skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add pproenca/dot-skills --skill ray-llm. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Ray LLM popular?

It lives in pproenca/dot-skills, which has 228 GitHub stars and 19 forks.

Related skills

12 Factor App

228

pproenca/dot-skills

The Twelve-Factor App methodology for building scalable, maintainable cloud-native applications. Use when designing backend services, APIs, microservices, or any software-as-a-service application. Triggers on deployment patterns, configuration management, process architecture, logging, and infrastructure decisions.

Software Development19

37signals Rails

228

pproenca/dot-skills

37signals Rails coding principles and conventions from DHH, Jorge Manrubia, and the Fizzy/Basecamp/HEY codebase. This skill should be used when writing, reviewing, or refactoring Ruby on Rails code following the 37signals philosophy — vanilla Rails, CRUD controllers, rich domain models, concerns, no service objects, Hotwire, Turbo, Stimulus, Solid Queue, Solid Cache, Solid Cable, multi-tenancy, Minitest, custom auth, or DHH conventions.

Software Development19

Acceptance Pipeline Catalog

228

pproenca/dot-skills

Use when implementing, reviewing, or debugging a Gherkin acceptance-test pipeline with mutation testing. Covers parser, JSON IR, generator, runtime, step handlers, test runner, mutator, value mutation rules, execution, result classification, reporting, project layout, conformance, and agent setup. Based on Uncle Bob's Acceptance Pipeline Specification. Trigger even when the user mentions Gherkin parsing, acceptance test generation, mutation testing for acceptance tests, or building a portable test pipeline.

Testing & QA19

Relational Data Modeling

228

pproenca/dot-skills

Corrects the wrong defaults a model has when designing a relational schema — the DDL decisions an experienced engineer makes differently. Use when creating or reviewing tables, migrations, ER models, or ORM schema definitions. Covers identity and keys (surrogate vs natural, identity vs serial, uuidv7, composite keys that make cross-tenant references impossible), relationships (polymorphic foreign keys the database cannot enforce, referential actions, unindexed FK columns, disjoint subtypes), invariants the engine can prove instead of application code (EXCLUDE, partial unique indexes, CHECK lim

AI, LLMs & Agents19

Provision Model Keys

201

swyxio/skills

Provision app-specific Claude, OpenAI, OpenRouter, ElevenLabs or fal.ai API keys, install secrets, rotate keys or adjust their budgets using saved authorization. Use for credential setup and lifecycle requests, not ordinary model/API implementation.

AI, LLMs & Agents12

Reserved Handle Policy

201

swyxio/skills

Design, implement, audit, or refresh protected username and handle namespaces for public products. Use whenever a product has open signup, mutable handles, profile URLs, impersonation or squatting risk, reserved route names, short usernames, common-word or common-name claims, developer/AI terminology, notable social identities, or administrator-approved handle assignment—even if the user only asks for a username denylist.

AI, LLMs & Agents12

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.