Hermes Agent's Learning Loop Is a Counter, a Critic and a Garbage Collector

Hermes Agent, Nous Research’s open-source personal agent, describes itself as “the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge.”¹ It has 244,071 GitHub stars and is on version 0.21.1.

“Learning” is a loaded word in this field. It can mean gradient updates, or optimisation against a metric, or something much more modest. So instead of reading the marketing, this post reads the code — every claim below links to a line in the repository, pinned to the commit I read.

The short version: the loop is real, it is well engineered, and it is not learning in the sense the word usually implies. It is three mechanisms stacked on top of each other — a counter that decides when to reflect, a critic that decides what to write, and a garbage collector that decides what to throw away. Once you see which part does what, the design’s strengths and its blind spots are both obvious.

The whole loop on one page

flowchart LR
T["Tool call<br/>counter +1"] --> E{"Turn ends:<br/>count ≥ 15?"}
E -- "no" --> T
E -- "yes · reset" --> F["Review fork<br/>writes via guards"]
F --> L[("Skill<br/>library")]
L --> Q["Curator<br/>stale 30d · archive 90d"]

Part one: the counter

The foreground agent — the one talking to you — gets exactly one sentence about skills in its system prompt: “When you work out a non-trivial workflow, record it with skill_manage for future reuse.”² It can act on that whenever it likes. But the loop does not rely on the agent remembering to. It relies on a counter.

Every tool-calling iteration increments _iters_since_skill, as long as the skill_manage tool is available; the source comment notes that it “resets whenever skill_manage is used.”³ At the end of each turn, the finalizer checks it:

# agent/turn_finalizer.py
_should_review_skills = (
    agent._skill_nudge_interval > 0
    and agent._iters_since_skill >= agent._skill_nudge_interval
    and "skill_manage" in agent.valid_tool_names
)
if _should_review_skills:
    agent._iters_since_skill = 0

When it fires, a background review is spawned — but only after the response has been delivered, only if the turn was not interrupted, and never for cron jobs, where the source explains the fork “costs ~30K tokens / event with no human-in-the-loop benefit.”⁴

The interval itself has a quirk worth knowing. The code falls back to 10 iterations if nothing is configured.⁵ The shipped example config sets 15,⁶ and the installers copy that file verbatim into ~/.hermes/config.yaml.⁷ So a normally installed agent reflects every 15 tool calls, and the number in the code is the one almost nobody runs.

Part two: the critic

The review is a forked copy of the agent with a narrowed toolset — just skills, plus memory when memory is also due for review.⁸ It receives the conversation snapshot and a long instruction. The opening lines set the tone:

“Review the conversation above and update the skill library. Be ACTIVE — most sessions produce at least one skill update, even if small. A pass that does nothing is a missed learning opportunity, not a neutral outcome.”⁹

That is a deliberate thumb on the scale. The prompt lists what counts as evidence — the user corrected your style or tone, the user corrected your workflow, a non-trivial technique or workaround emerged, or a skill consulted this session turned out to be wrong or outdated — and then gives an order of preference: patch a skill that was loaded this session, then patch an existing umbrella skill, then add a support file, and only then create a new class-level skill.⁹ “Nothing to save.” is allowed, but the prompt says it “should NOT be the default.”⁹

It also enforces taste on naming. A new skill “MUST NOT be a specific PR number, error string, feature codename … or ‘fix-X / debug-Y / audit-Z-today’ session artifact.”⁹ That rule exists because the obvious failure mode of an agent that writes its own notes is a library of a thousand one-off post-mortems.

What the critic is not allowed to touch

The fork runs with no user present, so its writes pass through guards that refuse anything it does not own.¹⁰ Ownership is decided at creation time, and the rule is stricter than you might expect: only a skill created by the background review is marked as agent-created. A skill the foreground agent creates — even unprompted — “belongs to the user.”¹¹

The read-before-write guard is the subtle one. Before patching an existing skill, the fork must call skill_view on it inside the review; content quoted earlier in the transcript does not count.¹⁰ That closes a real hole: without it, a review could “update” a skill from a stale copy in the conversation and silently revert someone’s newer edit.

Part three: the garbage collector

An agent told to be active will write a lot. The curator exists, in the docs’ words, so that skills created by the loop “don’t pile up forever … Without maintenance, you end up with dozens of narrow near-duplicates that pollute the catalog and waste tokens.”¹²

It is not a cron job. It runs when two conditions hold: at least interval_hours since the last run (default 168, one week) and at least min_idle_hours of inactivity (default 2).¹² A fresh install does not run it at all at first — the first observation seeds the timestamp and defers a full interval.¹³

A run has two phases, and only the first is on by default:

• Deterministic transitions. A skill with no activity for 30 days becomes stale; after 90 days it is moved to an archive directory. Pinned skills and skills referenced by any cron job are skipped, and a never-used skill is not archived until it is at least 30 days old — “zero uses is absence of evidence, not proof the skill is disposable.”¹²
• LLM consolidation, which merges overlapping skills into umbrellas. Off by default, because it “costs aux-model tokens on every run and makes broad structural changes”; a full sweep “typically takes 50–100 API calls.”¹²

The curator never deletes. The worst outcome is an archive, which hermes curator restore reverses.¹²

Read that carefully, because it is the most important sentence in this post: the curator removes skills that are unused, not skills that are wrong. Activity is the sum of views, uses and patches.¹⁴ A skill that is loaded constantly and quietly steers the agent in the wrong direction is, by the curator’s only measure, the healthiest skill in the library.

The 60-character bottleneck

Skills follow the Agent Skills open standard: a directory with a SKILL.md whose frontmatter carries a name and a description. The standard allows descriptions up to 1,024 characters and explains the design as progressive disclosure — about 100 tokens of metadata per skill load at startup, and the full instructions load only when a task matches.¹⁵

Hermes is stricter. The skill index in its system prompt truncates every description to 60 characters — 57 plus an ellipsis.¹⁶ Hermes’ own authoring prompt spells out the consequence: “anything past char 60 is silently cut and never routes.”¹⁷

This is the real ceiling on how much the loop can “learn”. Every skill the critic writes competes for recall on the strength of 60 characters of routing text. Write a hundred skills and the agent is choosing among a hundred short phrases each session — which is exactly the pressure the curator’s consolidation pass exists to relieve, and exactly the pass that is off by default.

Memory has the same shape. MEMORY.md is capped at 2,200 characters and USER.md at 1,375, both injected as a frozen snapshot at session start, and neither auto-compacts — a write that would exceed the limit returns an error.¹⁸

Where the actual optimisation lives

If “improves them during use” suggested measured improvement, the measured version does exist — in a separate repository. Hermes Agent Self-Evolution uses DSPy and GEPA to “evolve and optimize Hermes Agent’s skills, tool descriptions, system prompts, and code,” generating an eval dataset, evaluating candidate variants against constraint gates, and opening a pull request against the main repository.¹⁹

That is optimisation against a metric. It runs offline, on demand, and its output goes through code review. The runtime loop inside every installed agent is the other thing: an LLM reading one transcript and editing text.

What this design costs

There is no outcome signal. Nothing in the runtime loop checks whether a patched skill made the next task succeed. The critic’s evidence is the conversation: corrections, frustration, a technique that appeared. That works well for preferences — “stop being so verbose” is unambiguous — and badly for correctness, where a confident wrong approach produces no complaint in the transcript at all.

The bias toward writing is a bias toward growth. Telling the reviewer that doing nothing is a missed opportunity is defensible; a timid reviewer never learns anything. But the counterweight is time-based archival that ignores correctness, and a consolidation pass that ships disabled. By default, the library grows by judgement and shrinks only by neglect.

One correction generalises. The prompt asks that a user’s complaint be embedded in the skill that “governs that class of task.”⁹ A preference expressed once, in one context, becomes a rule for every future session of that class. That is the feature. It is also how a bad day becomes permanent behaviour.

The loop’s writes outlive the session that caused them. A skill body is text the agent wrote after reading a transcript — and transcripts contain tool output, web pages and messages from gateways like Telegram and Discord. The ownership guards stop the review from editing skills it did not create; they do not vet what goes into the ones it did. An optional security scan of agent-written skills exists, but skills.guard_agent_created defaults to off.¹¹ If you want a human between the critic and the library, skills.write_approval: true stages every write for review under ~/.hermes/pending/skills/.²⁰

Reflection is not free. Each review event is a forked run on the full conversation snapshot — around 30K tokens by the source’s own estimate.⁴ At an interval of 15 tool calls, a tool-heavy session pays that repeatedly.

Takeaways

• “Self-improving” in Hermes means: a counter triggers an LLM review that edits skill files. No weights change, and no metric is optimised at runtime.
• The trigger is 15 tool calls in a normal install, checked at the end of a turn — the code’s default of 10 is overridden by the config the installer copies.
• The critic is told that doing nothing is a missed opportunity, so the library is designed to grow, and preferences expressed once become rules for a whole class of task.
• Ownership is strict and well guarded. The unattended review can only edit skills it created itself, and must re-read a file before changing it.
• The curator removes skills that go unused, not skills that are wrong. A frequently loaded bad skill is the one it protects most.
• 60 characters of description decide whether a skill is ever found. That, not the number of skills, is the practical limit on what the loop can accumulate.
• Turn on write_approval if the agent reads untrusted content. Whatever the review writes is loaded into future sessions.

References