Hugging Face Free code examples in Python and JavaScript

huggingface_hub covers the registry and inference. transformers is only needed once you are running weights locally.

pip install huggingface_hub

export HF_TOKEN="hf_your-token"   # macOS / Linux
setx HF_TOKEN "hf_your-token"     # Windows, then open a new shell

# Or log in once and the client finds the token on its own:
hf auth login

THE MODEL ID IS THE HUB ID, always — not the provider's name for the same weights. provider='auto' picks the first available in your configured order.

import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="auto",                      # or "together", "groq", ...
    api_key=os.environ["HF_TOKEN"],
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Explain a database index."}],
    max_tokens=200,
)

print(response.choices[0].message.content)

hf download fetches into the shared cache, so the second consumer on the box pays nothing. PIN THE REVISION in anything reproducible — `main` moves, and a model that changed underneath you is a very confusing bug.

# Just the files you need, not the whole repository.
hf download meta-llama/Llama-3.1-8B-Instruct \
  --include "*.safetensors" "*.json" "tokenizer*" \
  --revision refs/pr/1

# Where it went, and how much of the disk it is using:
hf cache scan

# HF_HOME relocates the cache — worth setting on a box where /home
# is small and the models are not.
export HF_HOME=/data/hf

Once the weights are local, the per-token bill and the network dependency both go away. vLLM reads straight from the cache, so there is no second copy and no conversion step.

# vLLM finds the model in the HF cache by its hub id.
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --api-key "$VLLM_API_KEY" \
  --max-model-len 8192

# From here it is an OpenAI-compatible endpoint on :8000, and the
# InferenceClient can point at it too — see the next step.

InferenceClient speaks to any OpenAI-compatible endpoint, so moving from a hosted provider to your own GPU is a base URL. Note that a URL and a provider are MUTUALLY EXCLUSIVE — passing both is an error.

# No provider= here: a URL and a provider cannot both be given.
local = InferenceClient(
    base_url="http://localhost:8000/v1",
    api_key=os.environ["VLLM_API_KEY"],
)

response = local.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Explain a database index."}],
    max_tokens=200,
)

# Identical call, identical response object. The only thing that
# changed is who pays for the tokens.
print(response.choices[0].message.content)