Meta Llama Free code examples in Python and JavaScript

There is no Llama SDK and no Llama API key, because Llama is not a service. Decide where the weights run first; everything after that is the OpenAI client with a different base URL.

# Fastest path: run it on your own machine.
#   https://ollama.com/download
ollama pull llama3.3
ollama serve                      # serves an OpenAI-compatible API on :11434

# The client is just the OpenAI SDK.
pip install openai

The client requires an api_key and the local server ignores it. Name the constant rather than inlining a string — it stops the next reader wondering whether it is a real credential.

from openai import OpenAI

# Required by the client, ignored by Ollama. Not a credential.
LOCAL_KEY = "unused"

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key=LOCAL_KEY,
)

response = client.chat.completions.create(
    model="llama3.3",                      # whatever you `ollama pull`ed
    messages=[
        {"role": "system", "content": "You are a concise technical assistant."},
        {"role": "user", "content": "Explain a database index in two sentences."},
    ],
)

print(response.choices[0].message.content)

When local hardware is not enough, the code does not change — only where it points. That portability is what you buy by choosing open weights, and it is worth keeping the switch in one function so it stays true.

import os
from openai import OpenAI

LOCAL_KEY = "unused"      # required by the client, ignored by Ollama

def llama_client():
    # One place decides where the weights run. Everything downstream
    # calls chat.completions.create with identical arguments.
    where = os.environ.get("LLAMA_HOST", "local")

    if where == "groq":
        return OpenAI(api_key=os.environ["GROQ_API_KEY"],
                      base_url="https://api.groq.com/openai/v1"), \
               "meta-llama/llama-4-scout-17b-16e-instruct"

    if where == "together":
        return OpenAI(api_key=os.environ["TOGETHER_API_KEY"],
                      base_url="https://api.together.xyz/v1"), \
               "meta-llama/Llama-3.3-70B-Instruct-Turbo"

    return OpenAI(base_url="http://localhost:11434/v1",
                  api_key=LOCAL_KEY), "llama3.3"

client, model = llama_client()

response = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Explain a database index."}],
)
print(response.choices[0].message.content)

# Model ids differ per host — each one names the same weights its own
# way. That is the one thing you cannot make portable.

The honest trade. None of this applies to a hosted-only model, and all of it is yours the moment you run the weights.

# 1. QUANTISATION IS A QUALITY DIAL, NOT A FREE SPEEDUP.
#    ollama pull llama3.3:70b-instruct-q4_K_M runs on far less VRAM and
#    is measurably worse at reasoning than q8. Benchmark on YOUR task
#    before accepting a smaller quant — the loss is uneven, not uniform.

# 2. CONTEXT WINDOW IS SET BY THE SERVER, NOT THE MODEL.
#    Ollama defaults to a fraction of what the weights support and
#    SILENTLY TRUNCATES the prompt. Long inputs come back with the
#    beginning missing and no error anywhere.
#    Set num_ctx explicitly (Modelfile, or the options field).

# 3. NO VENDOR MEANS NO RATE LIMIT AND NO CAPACITY EITHER.
#    One GPU serves a handful of concurrent requests. vLLM's continuous
#    batching is the difference between a demo and a service.

# 4. THE LICENCE IS NOT MIT.
#    Llama ships under Meta's community licence, which has conditions
#    on very large deployments and on naming. Read it before shipping.