AI Inference Service Mesh

Use verified service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience.

9 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add huytran1120/vietnam-agentic-skills --skill ai-inference-service-mesh
Manual install
git clone --depth 1 https://github.com/huytran1120/vietnam-agentic-skills.git /tmp/vietnam-agentic-skills
mkdir -p ~/.claude/skills
cp -r /tmp/vietnam-agentic-skills/skills/ai-inference-service-mesh ~/.claude/skills/ai-inference-service-mesh

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# AI Inference Service Mesh

Apply Istio/Linkerd mesh controls to secure and optimize east-west AI traffic across inference microservices.

## Why Mesh for AI

- Enforce mTLS between gateway, retriever, reranker, and model services
- Apply fine-grained traffic policies without app code changes
- Run progressive delivery for model-serving backends
- Observe latency hops for retrieval + generation chains
- Route inference requests by model version, tenant, or priority tier
- Protect expensive GPU-backed services from cascading failures

## Prerequisites

Confirm the exact cluster/context, mesh version, CRD schemas, owned namespace and authorized mutation scope. Examples are alternative fragments, not a manifest to apply wholesale: multiple DestinationRules for the same host must be consolidated under the installed version's merge rules. Preview and validate in a disposable environment; protect production routes and rollback evidence.

```bash
# Illustrative mutating install; default is not a validated production configuration
istioctl install --set profile=default \
  --set meshConfig.accessLogFile=/dev/stdout \
  --set meshConfig.defaultConfig.holdApplicationUntilProxyStarts=true

# Label inference namespace for sidecar injection
kubectl create namespace ai-inference
kubectl label namespace ai-inference istio-injection=enabled

# Verify installation
istioctl verify-install
istioctl analyze -n ai-inference
```

## Core Patterns

### mTLS Strict Mode Cluster-Wide
Read the full skill on GitHub

Details

Skill name
ai-inference-service-mesh
Path
skills/ai-inference-service-mesh/SKILL.md
License
MIT
Repo updated
29 October 2026
Copies
Found in 2 GitHub repos

FAQ

What does the AI Inference Service Mesh skill do?

Use verified service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience.

How do I install the AI Inference Service Mesh skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add huytran1120/vietnam-agentic-skills --skill ai-inference-service-mesh. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is AI Inference Service Mesh popular?

It lives in huytran1120/vietnam-agentic-skills, which has 9 GitHub stars and 4 forks.

Related skills

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.