Models & Cost — จาก Nous Portal ถึง GPU ของเราเองModels & Cost — From Nous Portal to Your Own GPUs

Picture the end of the month. The provider's invoice arrives, and the number on it is several times what the agent's own dashboard had been telling you. You open /insights and see one figure; you open the provider's billing page and see another. Which one is the truth? And the question underneath that one — while the agent was working through the night, whose servers did your organization's data sleep on?

Those two questions — what does it cost and where does the data go — are more entangled than they look, because every step up or down the model ladder, from a single Nous Portal bill to GPUs in your own server room, trades cost against control at the same time. I have touched money twice already in this series: the Portal tier table and the "MIT-free, pay only for the model" framing live in #1 Hermes 101, and a brief section on cost control and BYO keys versus Portal sits in #7 Production. This post is the full treatment of both.

I will walk the five rungs of the ladder, the three layers of fallback, where the tokens actually go, the economics of the prompt cache, the cost figures Hermes reports (and what they cannot see), running open-weight models on your own GPUs, and then close with the question no Thai organization gets to skip — PDPA. Every number here comes from the official docs, the release notes, the issue tracker or the Portal pages as I fetched them on September 1, 2026. Where the sources disagree with each other, I will tell you how they disagree rather than pick the nicer figure.

The Model Ladder: Five Rungs an Organization Climbs

When I talk to a department weighing Hermes, the first question is usually "Portal, or our own keys?" — which is too narrow a question. Hermes lets you stand on any of five rungs, and each rung answers three questions differently: who holds the key, where the data travels, and who sends the bill.

1. Nous Portal — one OAuth, one bill. The README promises "300+ models — pick any of them with /model <name>", and a paid subscription unlocks the Tool Gateway (web, image, TTS, cloud browser). The fastest rung, and the only one that adds Nous as an extra hop in the data path.
2. BYO keys — an OpenRouter key or a direct provider key (Anthropic, OpenAI and so on). Bills scatter across key owners, but the data path is one hop shorter.
3. A named custom provider behind your corporate gateway — declare a provider of your own in config, pointing at an internal LLM gateway (LiteLLM, the IT department's proxy, or a Bedrock contract you already hold), authenticated with short-lived SSO tokens.
4. Self-hosted inference — vLLM, llama.cpp, SGLang, LM Studio or Ollama on your own GPUs. Token cost zero; electricity and staff cost not zero.
5. Air-gapped — the rung many government agencies want, and the one Hermes has not officially reached (more on that in the data section).

What makes the ladder usable is that every rung is switched with the same commands. The configuring-models doc draws the line clearly: hermes model is the out-of-session wizard that adds providers, while /model inside a session only switches among providers already configured — with three scopes:

# This session only — reverts when the session ends
/model <name> --provider <p>

# Persist to config.yaml
/model <name> --provider <p> --global

# One turn, then auto-restore
/model <name> --once

# A model_aliases entry lets a short name expand to provider + model
/model fav

One thing gateway operators must remember: gateway sessions pick up config changes only for new sessions unless the gateway is restarted. If you changed the model and the Telegram bot is still answering on the old one, that is the first thing to suspect.

The third rung is where mid-sized and larger organizations tend to settle, and the docs support it better than I expected. providers.<name> takes api, key_env and a transport of either chat_completions or anthropic_messages; key_cmd runs a CLI of yours to mint a short-lived token from enterprise SSO and refreshes it before expiry:

# config.yaml — a provider of your own, behind the corporate gateway
providers:
  corp-gateway:
    api: https://llm-gateway.internal.example.ac.th/v1
    transport: chat_completions        # or anthropic_messages
    key_cmd: "my-auth-cli print-token --profile prod"   # short-lived SSO token, auto-refreshed

# Switch to it in a session
/model custom:corp-gateway:<model>

And since v0.21.0 (August 31, 2026), third parties can ship providers as pip packages that Hermes discovers through entry points — which means a platform team can package the internal provider once and distribute it to every machine without editing configs one by one (a thread that continues into #10 Desktop & Fleet).

Portal as a Routing Tier, Not a Religion

In #7 I declined to say what the Free tier includes, because the docs page and the pricing page disagreed. The pricing page at portal.nousresearch.com, fetched September 1, 2026, now settles it: Free is "Free models only" with $0 in monthly credits. The four-tier table itself (Free $0 · Plus $20 → $22 credits · Super $100 → $110 · Ultra $200 → $220) is already printed in #1 Hermes 101, so I will not repeat it — what #1 does not carry is the following:

• Rollover caps — unused credits carry into the next month, but only up to $10 (Plus) / $50 (Super) / $100 (Ultra). An organization with bursty usage — heavy at the end of term, quiet over the break — quietly forfeits whatever sits above that cap in every light month.
• The paid cards add "Hosted tool usage" and "High rate limits" (Free gets "Standard rate limits").
• Top-ups — pay-as-you-go credit top-ups come in $10, $20, $50, $100 or $200.
• Copy that contradicts itself — the tier cards say "200+ Models" while the catalog copy on the same page says "over 300" and the README says "300+". I go with 300+, as the README does, and note that the cards have not caught up. The same page prices the catalog at "free to $24 per million input tokens".

The "10%" needs untangling, because there are two of them and they mean different things. The first is the credit bonus on the pricing page — "10% bonus" means you pay $20 and receive $22 in credits (and $100 → $110, $200 → $220). The second is the provider discount in the configuring-models doc — "Portal subscribers also get 10% off token-billed providers" — a lower per-token price when you route through the Portal, not extra credit. A third figure — Teknium (a Nous co-founder) wrote "20% Discount on all models" in his X post for the Herald release — exists only on X, appears on neither the Portal pages nor the docs, and I cannot verify whether it still applies or what it ever applied to. I will not print the 20% as settled — log into the Portal and read the live number before it goes into a budget.

What a paid subscription genuinely adds is the Tool Gateway. The doc says it is "included with every paid Nous Portal subscription", while Free-tier accounts "can use Portal for inference but don't include managed tools" (with the caveat that "some accounts are also entitled to a free tool pool"). Nous manages five backends: Firecrawl for web, FAL for image generation, OpenAI TTS, Browser Use for a cloud browser, and a Modal cloud terminal as an optional add-on — and, the part I like, it is "opt-in per tool, not all-or-nothing":

# config.yaml — adopt Nous-managed tools one at a time
web:
  backend: nous            # Firecrawl via the Portal
image_gen:
  provider: nous           # FAL
tts:
  provider: nous           # OpenAI TTS
stt:
  provider: nous
browser:
  cloud_provider: nous     # Browser Use — only when you want it

Since v0.19.0 (July 20, 2026) the plan can be managed from inside the TUI/CLI with /subscription and /topup — neither is in the user guide's slash-command list yet, so I am citing the release notes — and hermes portal info shows login, subscription and routing status. Under the hood Hermes mints a short-lived JWT from your stored Portal refresh token on every inference call.

Now the question people get wrong most often: "Nous makes the Hermes 4 models — why not run Hermes Agent on Hermes 4 and get the cheapest possible loop?" The answer comes from Nous itself, in the run-hermes-with-nous-portal guide:

💡 "Hermes-4-70B and Hermes-4-405B are available on the Portal at deep discounts, but they're chat/reasoning models, not tool-call-tuned. They will struggle with multi-step agent loops… For Hermes Agent itself, stick to the frontier agentic models." Same name; not the same job.

Where Hermes 4 does fit is the subscription proxy. hermes proxy start exposes a local OpenAI-compatible server (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models) so that non-agent apps such as Open WebUI can spend the same subscription on chat and research work; only nous and xAI are shipped today:

# Start the proxy on this machine, then point other apps at localhost
hermes proxy start

A closing footnote for Claude users, which affects readers of both this series and the OpenClaw series: the Anthropic OAuth path in Hermes requires a "Claude Max subscription + purchased extra usage credits" and consumes only those overage credits, while "Claude Pro subscribers cannot use the OAuth path; the supported alternative is an ANTHROPIC_API_KEY", billed per token. If you set up OpenClaw on Claude Pro earlier this year, your second rung is an API key, not a subscription.

Three Layers of Resilience: Pools, Fallbacks, Auxiliaries

In #7 I summarized the fallback chain in one line as "the way out of a 429." The full picture has three stacked layers, and the fallback-providers doc keeps them distinct: credential pools rotate several keys for the same provider, fallback_providers switch to a different provider:model, and each auxiliary task resolves its provider independently. Pools are always tried before fallbacks.

The first layer — the fallback chain, shipped in v0.6.0 (March 30, 2026) — is a top-level YAML list, with a management command since v0.12.0:

# config.yaml — the order to try when the primary fails
fallback_providers:
  - provider: openrouter
    model: <provider/model>
  - provider: nous
    model: <model>

# Or manage it from the CLI without touching the file
hermes fallback list
hermes fallback add <provider> <model>
hermes fallback remove <provider> <model>

The legacy singular fallback_model still works, but the list wins when both exist, and there is deliberately no environment variable — the doc wants this controlled by YAML and CLI only. What you need to know precisely is what triggers it:

The second layer — credential pools, from v0.7.0 (April 3, 2026, which Nous called "the resilience release") — supports OpenRouter, Anthropic, Nous Portal and custom OpenAI-compatible endpoints:

# Add a second key for the same provider to its pool
hermes auth add <provider>
hermes auth list
hermes auth reset       # clear cooldown state

# config.yaml — rotation strategy per provider
credential_pool_strategies:
  openrouter: round_robin     # fill_first (default) | round_robin | least_used | random

The rotation rules are finer-grained than you might expect: a 429 is retried once, and only a second consecutive 429 rotates the key (one-hour cooldown); a 402 rotates immediately (one-hour cooldown); a 401 tries a token refresh first, then rotates (five-minute cooldown); and only when every key in the pool is exhausted does the request fall through to the fallback chain. All of this state lives in ~/.hermes/auth.json — one more file that belongs in the backup routine from #7.

The third layer — auxiliary tasks — is the background work that is not the main agent. The configuring-models page enumerates eleven slots: title generation, vision, compression, approval scoring, web extraction, skills hub, MCP tool routing, triage specifier, Kanban decomposer, profile describer and curator (in the fallback-providers doc the same slots appear as YAML keys such as vision, compression, skills_hub, mcp, approval, title_generation, triage_specifier). The default provider: auto means the main model first, and since v0.15.0 (May 28, 2026) there is a layered ladder: main model → auxiliary.<task>.fallback_chain → fallback_providers → the built-in discovery chain (OpenRouter → Nous Portal → custom endpoint → Codex OAuth → API-key providers → give up).

# config.yaml — background work on a free/cheap model, separate from the main one (shape follows the fallback-providers doc's example)
auxiliary:
  compression:
    provider: auto                   # try the main model first
    model: ""
    fallback_chain:
      - provider: openrouter
        model: inclusionai/ring-2.6-1t:free   # the doc's own :free example
  vision:
    provider: auto                   # use the main model if it can see

This is the documented "zero-cost auxiliary" pattern: v0.8.0 (April 8, 2026) added a free Xiaomi MiMo v2 Pro on the Portal for compression, vision and summarization, and the fallback-providers doc's own example puts inclusionai/ring-2.6-1t:free on OpenRouter in the compression fallback_chain (its other example sets google/gemini-3-flash-preview as the compression model — not a :free one). Two caveats. Compression that cannot find a model will "degrade to no-summary" — quietly, with no summary produced. And subagents inherit the parent's whole chain; the cost argument for delegation lives in #2 Agent Teams.

Every layer charges the same toll, and the doc says so in plain words: a fallback or a credential rotation resets the cache keys, so "the next request re-reads the entire history at full input-token price" — and again when you switch back. I will return to that in the cache section.

Where the Tokens Go: The Fixed Cost of Every Call

Before talking about price per million tokens, you need to know what each request actually sends, because an agent like Hermes never sends just the user's message — it attaches the system prompt, the schema of every enabled tool, the skills index, memory and profile, every single time.

The number that made this concrete comes from issue #4379 (opened April 1, 2026 by a user named Bichev; still open, P2, labelled area/usage-cost), which measured at v0.6.0 a fixed per-call overhead of roughly 13,935 tokens — about 73% of a typical request — broken down as system prompt ~5,176, 31 tool definitions ~8,759, skills index ~2,200. One evening of Telegram, WhatsApp and cron gateways consumed ~3.9 million tokens across 207 API calls.

# The fixed per-message budget — system prompt, skills index, memory, profile, tool schemas
# "Runs entirely offline — no API call"  (v0.16.0, tagged 2026.6.5)
hermes prompt-size
hermes prompt-size --platform telegram --json   # per platform, because toolsets differ

# Estimated token cost per toolset
hermes tools

The diet the official FAQ prescribes is simple: measure first, then disable the toolsets and skills you do not use. Since v0.11.0 each cron job can carry its own enabled_toolsets to "cap token overhead + cost per job" — a morning news digest does not need to carry the browser-automation schemas (the scheduling side is in #8 Automation).

The bigger instrument is Tool Search: an opt-in mode that replaces the schemas of MCP tools and non-core plugin tools with three bridge tools — tool_search / tool_describe / tool_call — and lets the model look up the tool it needs when it needs it, instead of carrying every schema on every turn (progressive disclosure). The key rule: "Built-in Hermes tools never defer" — the built-ins are always present, so the benefit is largest for organizations that attach many MCP servers, as described in #5 Integrations.

The next lever the configuration doc names explicitly as a cost lever is reasoning effort, settable at the agent level and per auxiliary task:

# config.yaml — thinking depth is money
agent:
  reasoning_effort: medium        # none | minimal | low | medium | high | max  (v0.19.0 added the top tiers, max and ultra)

auxiliary:
  compression:
    reasoning_effort: low         # summarizing does not need deep thought
  vision:
    reasoning_effort: none        # reading an image needs no reasoning

Nous has also been trimming its own hidden costs release by release, which tells you where money used to leak: v0.15.0 rebuilt session_search without an LLM — the release notes say the old version "cost ~$0.30/call and took ~30 seconds to summarize three sessions, sometimes confabulating"; v0.17.0 made the skill Curator's consolidation pass opt-in (curator.consolidate: true) so routine runs incur "no aux-model spend"; and v0.18.0 routed the post-turn self-improvement fork to an auxiliary model that digests context instead of replaying the whole conversation. The rule I draw from all of it: every background job is a model call, and every model call is money — make sure you know which background jobs are on.

Prompt-Cache Economics

If the fixed overhead is the problem, the prompt cache is half of the answer — because the part that repeats every turn (system prompt, tool schemas) is exactly the part that caches best. The mechanics per the developer guide: caching is enabled automatically when the model name is an Anthropic Claude model and the provider supports cache_control; the strategy is system_and_3, which places four breakpoints — the system prompt plus the last three non-system messages — and the doc claims it "Reduces input token costs by ~75% on multi-turn conversations".

# config.yaml
prompt_caching:
  cache_ttl: "5m"     # only "5m" (default) or "1h" are accepted — configurable since v0.12.0

Three things break the cache, and two of them are things we just configured in the previous sections: mid-history edits, a change of model identity (/model, an automatic fallback, a credential rotation), and compression — where the system-prompt cache survives and the rest re-establishes within one or two turns. The official tips page condenses it into a rule I want every team to memorize:

💡 "an explicit /model switch, an automatic provider fallback, or a credential-pool rotation all force the next turn to re-read the entire conversation at full input price" — and in long sessions "it's often cheaper to start a fresh session on the other model than to bounce back and forth".

The natural follow-up is "which providers actually cache?", and this is where the docs lag the release notes. The developer guide still says Anthropic native or OpenRouter only, but configuration.md already adds Nous Portal, and the release notes add several more: xAI via the x-grok-conv-id header (v0.8.0), the Bedrock Converse API via cachePoint and DeepSeek on OpenCode gateways (v0.20.0), Claude through LiteLLM on the OpenAI wire (v0.20.2/v0.21.0), and api.meta.ai via the Responses API (v0.21.0). v0.20.0 also made tool schemas cacheable on native Anthropic "without history loss". Put differently: if you stand on rung three, behind a corporate gateway, the odds that caching engages are better than the main doc page suggests — but confirm it yourself by watching cache_read in /usage.

The other half of the equation is compression, whose mechanics I covered in #3 Memory; today I add only the cost dimension. The defaults per the developer guide:

# Compression defaults (developer guide) — worked example for a 200K model
compression:
  threshold: 0.50            # compress at 50% of the context window  → 100,000 tokens
  target_ratio: 0.20         # share of the tail that is kept          → tail budget 20,000
  tail_mode: lean            # default since v0.20.6: 2.5% of context, 10K floor / 25K cap
# the gateway keeps a safety net at 0.85
# summary budget = content × 0.20, min 2,000, max min(context × 0.05, 12,000) → 10,000

Two financial consequences. First, every compression is one call to the auxiliary compression model with nearly the whole session as input, and the doc requires that model to have "a context window at least as large as the main agent model's" — so "use a small cheap model to compress" only works when the small model also has a large enough window. Second, compression breaks the history part of the cache, so the turn after it is always more expensive than usual. If /usage spikes periodically for no visible reason, check whether the spikes line up with compression events.

Caching is also why prefix caching matters so much once you run models yourself. An independent developer's recipe on dev.to (Qwen3.6-27B + vLLM on a 24 GB RTX 3090, May 2026 — third-party throughout) measured time-to-first-token on a large prompt at 38.6 seconds cold versus 1.59 seconds with the prefix cache warm, a difference of more than twenty times — and that "large prompt" is precisely the fixed Hermes overhead we just measured.

What the Cost Numbers Actually Measure

Hermes has several cost-reporting surfaces, and each measures something different. From the nearest outward:

# In a session
/usage                 # this session's tokens + rate-limit headers and cost detail (v0.9.0),
                       # account limits (v0.11.0), per-category context breakdown (v0.18.0)
/insights --days 7     # the retrospective view

# Outside a session
hermes insights --days 30 --source telegram

# Non-interactive runs (hermes -z) — a JSON report written "even when the run fails" (v0.19.0)
hermes -z <prompt> --usage-file ./nightly-usage.json

The --usage-file report carries estimated_cost_usd; input, output, cache_read, cache_write and reasoning tokens; api_calls; model; provider; session_id; and a completed/failed status. For an organization's cron jobs, this is what should flow into the log pipeline every night — it is the most tangible per-run evidence Hermes produces.

v0.21.0 added two things I had been waiting for. The first is a status bar that shows the cache-hit ratio, latency (a rolling mean of the last ten calls) and tokens per second live in the CLI. The second is model_overrides, which lets you enter per-million-token prices for models Hermes does not know — essential on rungs three and four, because a model behind your corporate gateway or on your own GPU has no price in any public registry, and without an override every cost figure is a false zero:

# config.yaml — v0.21.0
display:
  show_cost: true
  status_bar:
    fields: [cache_hit, latency, tps]    # cache_hit resets on model switch and compression; display only

model_overrides:
  <model-id>:
    context_length: 128000
    input_price_per_mtok: 0.50          # your contracted rate (illustrative) — flows into session cost summaries
    output_price_per_mtok: 1.50
    capabilities: [tool_use, vision]

Now the figure people misread most: the Analytics page in the web dashboard (7/30/90-day tokens, cache-hit %, cost, per-model table), which is off by default for a reason spelled out in the PR that turned it off — not on the dashboard docs page:

I want everyone who owns a budget to read that sentence twice, because what it excludes — auxiliary calls, retries, fallbacks, cache writes — is everything the last four sections were about. Put differently, Hermes's numbers tell you the shape of your spend; the size can only be read off the provider's bill. Community tooling such as Bichev's hermes-dashboard (a proxy plus a SQLite dashboard that reads state.db for cache-hit rates and daily cost) exists precisely because of this gap — I cite it as evidence of the gap, not as a recommendation to install it. And hermes sessions cost, which would report cache hits per session (PR #76596, August 2, 2026), is not merged as I write.

The thing that must be said most plainly: Hermes has no spend cap in dollars, daily or monthly. Issue #26382 (opened May 15, 2026; still open, P3) asks for budget.daily_usd_cap/monthly_usd_cap and points out that /usage and /insights only look backwards. The --max-budget-usd flag that several third-party cost guides list as a "Hermes option" is a Claude Code flag — it appears in the repo only inside the bundled claude-code skill's SKILL.md, and not in cli-commands.md, configuration.md or the FAQ. What does exist are behavioural guardrails:

# config.yaml — the guardrails that actually exist (not a dollar cap)
agent:
  max_turns: 500                 # iteration budget per task (default 500)
  run_budget_seconds: 1800       # wrap-up notice at 80% of the budget
# loop_caps: cap web_search calls / subagent spawns per turn
# hard_stop_enabled: for unattended gateways

For an organization, the dollar ceiling therefore lives outside Hermes — a spend limit on the OpenRouter or provider side, a quota on the corporate gateway of rung three, or the Portal's own credit balance (a Portal that runs out of credit stops, which is a ceiling of a kind).

Bring Your Own GPUs: vLLM, llama.cpp, Ollama and the 64K Floor

The fourth rung takes the token cost to zero and adds two new problems: context and tool calling. The first is a hard number — the providers doc says Hermes "requires at least 64,000 tokens of context for agent use with tools", because the fixed overhead we measured already eats a large share. The second is that the server must parse the model's tool calls correctly, and every engine has its own flags:

# vLLM — the "hermes" parser is the tool-call format Hermes Agent speaks (others: llama3_json, mistral, deepseek_v3, xlam)
vllm serve <model> \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --max-model-len 65536

# llama.cpp — "Without --jinja, the server ignores the tools parameter entirely"
llama-server -m <model.gguf> --jinja -fa -c 64000 -ngl 99
# careful: -np divides the context across slots  →  -c 64000 -np 4 = 16K per slot, below the floor

# SGLang — add --tool-call-parser qwen at launch; the default output cap is only 128 tokens,
# raise it with --default-max-tokens or model.max_tokens

# LM Studio — a first-class provider with doctor checks since v0.12.0; endpoint :1234/v1
lms load <model> --context-length 64000

# config.yaml — point Hermes at your server
model:
  provider: custom
  base_url: http://vllm.internal:8000/v1
  api_key: none
  context_length: 65536        # tell Hermes directly instead of letting it guess

On context length, Hermes resolves the value in a documented order: your model.context_length → the persistent cache → the endpoint's /models → the Anthropic API → OpenRouter metadata → the models.dev registry → a 128K default for unknown models. For your own server, setting the first one explicitly saves a lot of debugging.

The official guide "Run Hermes Locally with Ollama — Zero API Cost" (added in v0.13.0, May 7, 2026) is the most honest page in the whole documentation set, because it says outright that most small models can chat but cannot act. In its model table, gemma4:31b (~20 GB, 24+ GB RAM) is the only entry with tool calling; gemma2:27b/9b and llama3.2:3b "can only chat; they can't take actions". The stated minimum is 8 GB RAM / 4 cores, the recommendation 32+ GB / 8+ cores / an NVIDIA GPU with 8+ GB VRAM, and CPU-only speed for the 31B is ~2–5 tokens per second — fine for an overnight cron job, not for a person waiting.

# Modelfile — raise the context to the floor Hermes needs
FROM gemma4:31b
PARAMETER num_ctx 64000

# or at the daemon level — this variable comes from Ollama's own FAQ, not the Hermes guide
OLLAMA_CONTEXT_LENGTH=64000 ollama serve

# CPU-only: allow long prefill
export HERMES_API_TIMEOUT=1800

So where does Nous's own Hermes 4 fit in this picture? The Hermes 4 family (405B, 70B, 14B) was released on August 26, 2025, and Hermes-4.3-36B (Apache-2.0, based on Seed-OSS-36B, trained with context up to 512K, with GGUFs at 4/5/6/8-bit) on December 3, 2025 — but in the Ollama library on September 1, 2026 the newest official entry is hermes3, updated a year ago. There is no official Hermes 4 or 4.3 entry, only community uploads, on top of Nous's own warning, quoted in the Portal section, that the 70B/405B are not tool-call-tuned. My conclusion: running Hermes Agent on Hermes 4 weights at home is possible via a GGUF with llama.cpp or LM Studio, but it is not an officially endorsed path, and you must test tool calling yourself.

Worse, the "Recommended Local Models" list on the providers page (Qwen 2.5 Coder, Llama 3.1, "Hermes 2/3", DeepSeek 67B, Mistral) contradicts the Ollama guide on the same site (gemma4:31b) and Red Hat's recipe (Qwen2.5-7B-Instruct) — the official docs do not agree with themselves, so the only safe rule is test tool calling on your own workload before buying a GPU.

For organizations already running Kubernetes, the Red Hat Developer recipe (June 2, 2026, by Gerald Trotman) places Hermes on OpenShift AI with a KServe vLLM InferenceService, default model Qwen/Qwen2.5-7B-Instruct, one GPU per replica, and points the agent at OPENAI_BASE_URL=http://hermes-llm-predictor.hermes.svc.cluster.local:8080/v1 — with a sentence worth taping to the server-room door: "Inference on CPU is 10-100x slower". AMD has a technical article on Hermes plus vLLM on an Instinct MI300X in the AMD Developer Cloud as well, but the page timed out when I fetched it, so I quote no flags from it.

Two closing notes. First, on the Nemotron question people keep asking: Nous was not a founding member of the Nemotron Coalition (NVIDIA's March 16, 2026 announcement names eight labs, Nous not among them) but joined around early June and served nvidia/nemotron-3-ultra:free on the Portal through Nebius only from June 4 to June 18, 2026. That guide is still live past its expiry (issue #53729 open), and I cannot verify whether any Nemotron SKU is free today. Second, Hermes keeps adapting to local inference: Ollama got a default max_tokens of 65536 in v0.17.0, v0.19.0 skips the Ollama probe for providers known not to be Ollama (part of the ~80% TTFT cut, from ~4.3 s to ~0.9 s — a figure from third-party release trackers, not the official release notes), and configuration.md says local endpoints "auto-raise socket and stale timeouts" to accommodate slow prefill.

Where the Data Goes: Telemetry, the Privacy Policy and the Air Gap That Isn't

Now the other half of the opening question. The Hermes runtime gives a clean answer — the official FAQ: "API calls go only to the LLM provider you configure… Hermes Agent does not collect telemetry, usage data, or analytics", with conversations, memory and skills all under ~/.hermes/. But "no telemetry" is not "no network egress at all": from the same doc and the release notes, four known outbound calls remain — the update check, models.dev registry lookups (cache-first), the plugin index fetch (24-hour cache, falling back to a bundled seed offline), and an originator: hermes-agent header on Codex requests. Your network team should know all four before writing firewall rules.

The bigger question, then, is not the agent but the Portal, which is rung one — and this is the page I want every DPO to read for themselves. The Nous Research Privacy Policy (last updated June 11, 2026) for Portal and Cloud states that inference payloads are stored by default; that prompts and outputs "may be shared with such providers as necessary to provide the Services"; that Nous may use data for "training, fine-tuning" and may generate or license aggregated or derivative data. The way out is Privacy Mode: once enabled, Nous will not store inference payloads and will not use them for training, product improvement or support, except to the limited extent necessary for security and legal purposes — and it is a prospective opt-out, not a retroactive one.

💡 What this policy does not state matters as much as what it does: no retention period, no data location, no zero-data-retention tier, no sub-processor list, and no DPA to sign. I will not infer any of those on Nous's behalf — and neither should you.

On the software side there are two helpers. The first is the data-training-tier guard in v0.21.0 — "a unified selection-guard registry warns you across every picker surface when a model trains on your data", and non-interactive workloads fail closed unless you set security.allow_data_training_tiers_noninteractive: true (secrets and vault integration are covered in #4 Security). The second is OpenRouter's routing knob:

# config.yaml — refuse routes to data-collecting providers (OpenRouter)
provider_routing:
  data_collection: deny
  sort: price                 # price | throughput | latency
  # only: [...] / ignore: [...] to allow-list or block providers by name

# v0.21.0 — non-interactive runs stop themselves if the chosen model trains on your data
security:
  allow_data_training_tiers_noninteractive: false

But honesty about the limits: issue #32757 "[Bug]: Data privacy" (May 26, 2026) reported that data_collection: deny did not prevent a data-logging model when routed via Nous Portal. It was closed as not planned with a label saying implemented on main, and no maintainer reply is visible — I file it as an open question, not a fixed bug.

And Hermes Cloud? The portal.nousresearch.com/cloud page says it is in preview, requires $10 in credits or an active subscription, and "scales to zero when idle — you only pay while it works" — which is the whole of what the page says about billing (the word "hourly" does not appear on it); its own FAQ asks "Where are the servers located?" and the content I could fetch gives no answer. No machine sizes, prices or regions are published — the figures on other blogs are all third-party. For a Thai organization this one decides itself: until someone can say which country the servers are in, Cloud does not clear the first PDPA gate.

As for the fifth rung — air-gapped — it is still a request, not a feature. Issue #17696 (opened April 30, 2026, P3, still open) notes that the installer needs GitHub, source and public registries. What can be done today: the PyPI wheel (PR #26593, merged May 15, 2026 — though the latest PyPI release, 0.19.0, lags GitHub's v0.21.0, and the platform matrix lists pip installs as unsupported), and the official nousresearch/hermes-agent image (5M+ pulls, roughly 900 MB), which can be mirrored into an internal registry. And do not confuse any of this with terminal.docker_network: false:

# config.yaml — a sandbox with no network (--network=none)
terminal:
  docker_network: false
# cuts egress only for agent commands (terminal, execute_code, file tools)
# NOT for the agent process's own model calls — and flipping it removes the existing networked container

PDPA and a Decision Tree for Thai Organizations

In #4 I told the story of a Hermes instance in YOLO mode running inside the Ministry of Finance network. That incident was about who controls the agent. This section is the other side of the same coin — when we are the ones in control, where does the data the agent reads end up, and the answer has to pass through PDPA.

The directly relevant law is the pair of sub-regulations under section 28 (transfers to countries with adequate protection) and section 29 (binding corporate rules), gazetted on December 25, 2023 and in force since March 24, 2024. Several law-firm summaries agree that no adequacy list has been published — I have not read the full notification text on pdpc.or.th myself, so I flag that as a secondary-source observation. The practical effect is that sending prompts containing personal data abroad must rest on one of the other section 28 exceptions (which your legal team, not this post, should identify), and that conversation with your DPO is far easier when inference stays in the country.

The good news is that in-country options now exist: AWS Asia Pacific (Thailand) ap-southeast-7 launched on January 8, 2025 (3 AZs); Google Cloud's Bangkok region launched on January 21, 2026 (3 zones); Microsoft has announced a Thailand region, but I found no GA date. The first two let you place rung three (an in-country LLM gateway) or rung four (vLLM on GPUs in a Thai region) without Hermes noticing any difference.

Another option worth watching for Thai-language work is Typhoon from SCB 10X: the endpoint https://api.opentyphoon.ai/v1 is OpenAI-compatible, Typhoon's docs include tool-calling examples (e.g. typhoon-v2.1-12b-instruct), and typhoon2.5-qwen3-4b is on Ollama with function calling and a 256K context. On paper, that makes it a Hermes custom provider:

# config.yaml — Typhoon as a named custom provider (untested with Hermes)
providers:
  typhoon:
    api: https://api.opentyphoon.ai/v1
    transport: chat_completions
    key_env: TYPHOON_API_KEY

/model custom:typhoon:typhoon-v2.1-12b-instruct

To be clear: nobody has tested Typhoon with Hermes officially, its hosting location and pricing were not on the pages I fetched, and the 64K floor still applies — this is an experiment I intend to run myself, not a recommendation.

The context on the horizon is Thailand's draft AI Act, which ETDA opened for consultation on July 9, 2026 through August 14, 2026 — a risk-based framework that Baker McKenzie's analysis expects to take two to three years to enact. Not binding yet; but choosing a rung that keeps the data under your control today is cheap insurance for the day it is.

Everything above folds into one decision table — not a ready-made answer, but the questions to ask, in order:

My position as of September 1, 2026: for a research or development team whose data carries no names, the Portal is still the best-value rung — turn on Privacy Mode, put the auxiliaries on a free model, and make /usage a habit. For any unit that touches personal data, the durable answer is rung three — a corporate LLM gateway in a Thai region that Hermes sees as just another provider — because it moves the entire PDPA question into a single contract your legal team can read, and leaves Hermes doing what it is good at: switching models at will, with no lock-in.