Memory & Knowledge ManagementMemory & Knowledge Management

🧠 Why Is an AI That Remembers Nothing Useless?

Imagine you have a brilliant assistant, but every morning he arrives at work having forgotten everything that happened the day before. He does not remember what the two of you discussed, does not know what you like or dislike, and the mistake he made yesterday he will simply make again today.

This is the problem with stateless AI — a system with no memory, unable to learn from its past, forced to start over every single time. For AI agents that have to work inside a real organization, the problem is severe:

• The same questions every day: "Which ERP system do you use?" or "What is the payroll approval process?" Asked yesterday, asked again today.
• Forgotten preferences: No recollection of the report format you prefer, or the hours when you should not be disturbed.
• No learning from mistakes: The same error over and over, because the lesson was never kept.
• No continuity: Every conversation feels like talking to a stranger.

🏗️ OpenClaw's Memory Architecture

OpenClaw answers this with a three-layer memory system, designed so the layers work together the way a human brain does:

🔄 Short-term Memory (Context Window)

This is the temporary store holding the current conversation and whatever data is in active use. It runs to roughly 200K tokens (about 150,000 words). As it approaches capacity, the system performs an automatic "memory flush", preserving what matters before it clears the space.

📅 Medium-term Memory (Daily Logs)

This writes each day's activity into a memory/YYYY-MM-DD.md file. It acts as a "daily journal" recording what happened, which commands were run, which decisions were taken, and which lessons the day produced.

🎯 Long-term Memory (Curated Knowledge)

This is the filtered knowledge held in MEMORY.md — the material that must be remembered permanently, such as security passphrases, user information, important lessons, and the knowledge the work depends on.

📁 The Memory Files That Matter

OpenClaw's memory system is built from a handful of core files, each with its own role and its own format:

🧠 MEMORY.md - Curated Long-term Memory

This is the heart of the system, holding the filtered information the agent needs over the long run:

# MEMORY.md - Core Agent Memory

## Security
- Security passphrase: "rainbow-elephant-7295"
- GUEST_MODE: Activated for unknown users
- Sensitive data protection protocols active

## User Preferences
- Report format: Professional DOCX with page numbers
- Communication style: Direct, concise, no fluff
- Timezone: Bangkok (UTC+7)
- Language: English for technical work, Thai for documentation

## Key Facts
- Primary database: PostgreSQL on port 5432
- ERP system: SAP with custom modules
- Backup schedule: Daily at 02:00 AM
- Critical servers: srv-app-01, srv-db-01, srv-backup-01

## Lessons Learned
- 2026-03-08: Never run raw psql commands - always use ha_query.py wrapper
- 2026-03-09: DOCX reports must include page X of Y footer
- 2026-03-10: Long tasks should spawn sub-agents to keep chat responsive

## Important Connections
- Admin contact: [email protected]
- Emergency escalation: +66-2-xxx-xxxx
- Vendor support: Weekdays 9-17 Bangkok time

📅 memory/YYYY-MM-DD.md - Daily Logs

The daily record of activities, decisions and observations — the "session history" you can go back and read:

# 2026-03-10.md - Daily Memory Log

## Session Summary
- Started: 07:45 Bangkok time
- Task: Writing OpenClaw blog post #3 about memory management
- Spawned sub-agent: openclaw-blog-3 for content creation

## Key Activities
1. **Blog Writing** (07:45-08:30)
   - Created comprehensive Thai-language post about memory systems
   - Used proper CSS structure with purple theme (#8b5cf6)
   - Included all required components: code-block, diagram-box, info-card

2. **File Management** (08:20)
   - Saved to tmp/openclaw-memory.html and tmp/openclaw-memory.md
   - Followed series navigation structure

## Observations
- User prefers comprehensive explanations over code dumps
- Purple theme (#8b5cf6) is standard for OpenClaw blog series
- Series format: Post #3, links to previous and upcoming posts

## Decisions Made
- Used Thai language with English technical terms
- Included substantial explanations between code examples
- Created both HTML (styled) and Markdown (clean) versions

💓 HEARTBEAT.md - Proactive Memory System

The system that gives the agent "proactive awareness", automatically checking and reporting on the situations that matter:

# HEARTBEAT.md - Proactive Monitoring

## Check Rotation Schedule
- **Morning** (08:00): Email, calendar, system health
- **Afternoon** (14:00): Sub-agent status, pending tasks
- **Evening** (18:00): Daily summary, memory distillation
- **Night** (22:00): Backup verification, security scan

## Heartbeat State Tracking
```json
{
  "last_email_check": "2026-03-10T08:00:00+07:00",
  "last_calendar_sync": "2026-03-10T08:00:00+07:00",
  "active_subagents": ["openclaw-blog-3"],
  "pending_tasks": [],
  "memory_last_distilled": "2026-03-09T18:00:00+07:00"
}
```

## Memory Distillation Workflow
1. Review daily logs from past 3 days
2. Extract key facts and lessons learned
3. Update MEMORY.md with important information
4. Archive old daily logs (keep last 30 days)
5. Update knowledge graph connections

✅ Memory Management Best Practices

Managing memory well takes clear principles and genuine discipline:

📝 Write It Down Now, Don't Rely on Recall

"Mental notes don't survive restarts" — this is the first and most important rule. The moment you decide "I'll just keep that in mind" and the system restarts, that information is gone. Write it to a file the instant it occurs to you.

// ❌ Wrong - relying on "mental notes"
"I'll remember to check the database backup tomorrow"

// ✅ Right - written to a file immediately
echo "TODO: Check database backup status" >> memory/$(date +%Y-%m-%d).md

🎯 Be Specific, Not Vague

A vague note cannot be put to any use later. Record detail that is specific and genuinely actionable:

⚖️ Separate Facts from Opinions

Keeping fact apart from opinion makes future decisions considerably more accurate:

🧹 Regular Review and Pruning

Memory needs regular maintenance. You cannot simply keep piling material into it:

• Weekly Review: go through the daily logs and move what matters into MEMORY.md
• Monthly Cleanup: delete what has gone stale or is no longer relevant
• Quarterly Audit: check accuracy and update whatever has changed
• Annual Archiving: move old material somewhere safe, but out of the way of current work

🕸️ Knowledge Graph & Obsidian Vault

Linear memory is not enough for complex work. OpenClaw uses a Knowledge Graph through an Obsidian Vault to build a network of connected knowledge:

🔗 Wikilinks & Bidirectional Connections

Using [[wikilinks]] lets connections between pieces of information form naturally:

# Project Status Report

## Database Migration
- Status: In Progress
- Related: [[Database Schema v2.1]], [[Migration Scripts]], [[Production Deployment]]
- Dependencies: [[User Authentication System]], [[API Endpoints]]
- Timeline: Complete by [[2026-03-15]]

## Risk Assessment
- Main concern: [[Data Integrity]] during migration
- Mitigation: [[Backup Strategy]], [[Rollback Procedures]]
- Contact: [[Database Team]], [[DevOps Lead]]

📊 KnowledgeGraph/ Structure

The Obsidian Vault is laid out systematically, so that anything in it is easy to find and easy to reuse:

🔄 Git Sync & Version Control

The Obsidian Vault uses Git as its version control, which gives it systematic backups and a tracked history of every change:

# Daily sync workflow
cd /home/clawdbot/obsidian-vault

# Pull the latest from remote
git pull origin main

# Stage the new changes
git add .
git commit -m "Daily update: $(date +%Y-%m-%d) - Added project status"

# Push to the remote repository
git push origin main

🔍 Semantic Search — ChromaDB

Keyword search is not enough when what you want is related or merely nearby. OpenClaw uses Semantic Search through ChromaDB to search by meaning and context:

🧠 Embeddings Concept

An embedding turns text into a "vector" — a set of numbers that reflects meaning — so that a computer can tell which pieces of text sit close to one another in meaning:

⚙️ ChromaDB Implementation

OpenClaw pairs Gemini embeddings (gemini-embedding-001) with ChromaDB to build search that understands meaning:

# Using ChromaDB through chroma_helper.py

# Activate virtual environment
source .venv/bin/activate

# Search for anything about database issues
python3 tools/chroma_helper.py search "database connection timeout"

# What comes back
Results:
1. memory/2026-03-08.md (similarity: 0.89)
   "MySQL connection timeout at 14:30, resolved by increasing wait_timeout parameter"

2. MEMORY.md (similarity: 0.85)
   "Database issues: Always check connection pool settings first"

3. memory/2026-03-07.md (similarity: 0.82)
   "PostgreSQL slow queries fixed by adding index on user_id column"

🆚 Semantic Search vs Grep

Which tool to reach for depends entirely on the kind of thing you are looking for:

# Examples of picking the right tool

# 🔍 Semantic Search - find related concepts
python3 tools/chroma_helper.py search "performance optimization"

# 🔍 Grep - find an exact error code
grep -r "ERROR 1045" memory/

# 🔍 Semantic Search - find similar fixes
python3 tools/chroma_helper.py search "slow database queries"

# 🔍 Grep - find the command that was used
grep -r "mysql -u" memory/

💓 Heartbeat System — Proactive Memory

The Heartbeat System is the mechanism that gives OpenClaw "situational awareness": it checks and reports on the situations that matter without waiting for the user to ask, working rather like a "background consciousness":

📊 heartbeat-state.json Tracking

The system uses a JSON file to track the state of each check and to rotate through them methodically:

{
  "last_checks": {
    "email": "2026-03-10T08:00:00+07:00",
    "calendar": "2026-03-10T08:00:00+07:00",
    "subagents": "2026-03-10T07:45:00+07:00",
    "system_health": "2026-03-10T06:00:00+07:00",
    "weather": "2026-03-10T07:00:00+07:00"
  },
  "rotation_schedule": {
    "morning": ["email", "calendar", "system_health"],
    "afternoon": ["subagents", "pending_tasks", "weather"],
    "evening": ["daily_summary", "memory_distillation"],
    "night": ["backup_verify", "security_scan"]
  },
  "active_monitoring": {
    "subagents": ["openclaw-blog-3"],
    "pending_tasks": [],
    "alerts": []
  },
  "memory_stats": {
    "last_distillation": "2026-03-09T18:00:00+07:00",
    "daily_logs_count": 3,
    "memory_size_kb": 15.2
  }
}

🔄 Rotation of Checks

Rather than checking everything every time, the system rotates its checks by time of day and by importance — which is efficient, and does not get in the way of the work:

✅ HEARTBEAT_OK Protocol

When everything is normal the system reports only HEARTBEAT_OK, so that notification fatigue never sets in — but when something needs attention, it says so plainly:

# ✅ Normal heartbeat
HEARTBEAT_OK - 08:00 Bangkok
- Email: No urgent messages
- Calendar: Meeting in 2 hours
- System: All services running
- Sub-agents: 1 active (openclaw-blog-3)

# ⚠️ Alert heartbeat
HEARTBEAT_ALERT - 14:30 Bangkok
- Database: Connection timeout detected
- Action needed: Check MySQL service
- Sub-agent: blog-writer completed task
- Memory: Daily log approaching 50KB limit

🧹 Memory Distillation Workflow

Distilling memory is the process that keeps knowledge from being lost while keeping it compact and usable:

1. Review Daily Logs: go back through the memory/YYYY-MM-DD.md files for the past 3-7 days
2. Extract Key Information: pick out what matters — decisions, lessons learned, new facts
3. Update MEMORY.md: add the new material to long-term memory
4. Archive Old Logs: move old files into the archive folder (keep the last 30 days)
5. Update Knowledge Graph: add or update connections in the Obsidian Vault

🔐 Memory Security

An AI agent's memory holds material that is sensitive and important to the organization, which makes securing it essential rather than optional:

🔑 Passphrase Protection

MEMORY.md is protected by a passphrase system that requires the user to confirm before sensitive information can be reached:

# Security Protocol in MEMORY.md

## Access Control
- Security passphrase required for sensitive operations
- Current passphrase: "rainbow-elephant-7295" (rotate monthly)
- Last rotation: 2026-03-01
- Next rotation due: 2026-04-01

## Protected Information
1. Database credentials and connection strings
2. API keys and authentication tokens
3. Personal information of users/employees
4. Financial data and sensitive business metrics
5. Security procedures and vulnerability information

👤 GUEST MODE Protection

When an unknown user is detected, the system enters GUEST MODE automatically, restricting access to private data and to sensitive operations:

📊 Data Sensitivity Levels

Information is graded by how sensitive it is, and protected with measures to match:

⚠️ What NOT to Store

Some categories of information should never go into the memory system at all, however good the protection around it:

• Raw Passwords: keep only hints, or references to a password manager
• Credit Card Numbers: never store financial details directly
• Personal Conversations: do not keep private conversations unrelated to the work
• Temporary Tokens: short-lived API tokens should not be retained for long
• Legal Documents: store references instead, since the documents themselves may change

🔄 Pre-Compaction Memory Flush

The context window is finite. As it approaches full, the system performs a "memory flush" to preserve what matters before it clears space for what is coming:

⚡ Memory Flush Triggers

The system begins the flush process when:

• Context Usage > 85%: preparation for a flush begins
• Context Usage > 95%: an emergency flush runs immediately
• Long Conversation: the conversation has run beyond two hours
• Manual Request: the user asks for a flush

💾 Memory Flush Protocol

The flush is carried out carefully, so that nothing important is lost along the way:

# Memory Flush Protocol

## Phase 1: Assessment (10 seconds)
- Analyze current conversation for key information
- Identify important decisions made
- Mark critical facts and lessons learned
- Calculate optimal flush points

## Phase 2: Preservation (30 seconds)
- Extract key information to memory/YYYY-MM-DD.md
- Update MEMORY.md with critical facts
- Save important file references
- Record conversation summary

## Phase 3: Notification (5 seconds)
- Notify user about memory flush
- Provide summary of what was preserved
- Confirm critical information retention

## Phase 4: Compaction (automatic)
- Clear context window
- Reload essential memory files
- Resume conversation with preserved context

🎯 What Gets Preserved vs Lost

Not everything survives a memory flush. Knowing what does will help you plan the work considerably better:

🌅 A Real Example: One Day in Agent's Life

To see the memory system clearly at work, follow a single agent through an entire day — from the moment it wakes to the evening wrap-up:

🌅 06:00 - Morning Heartbeat

HEARTBEAT_OK - 06:00 Bangkok
- System: All services running normally
- Memory: Loading MEMORY.md, USER.md, yesterday's daily log
- Weather: Clear skies, 28°C
- Calendar: Team meeting at 09:00, project review at 14:00
- Email: 3 new messages, 1 marked urgent

Memory Activity: the system reads the core memory files and syncs with the Obsidian Vault, preparing itself for the new day.

🔧 07:30 - First User Request

User: "Can you check the database backup status from last night?"

Action taken:
- Searched memory for backup-related information
- Found: "Backup schedule: Daily at 02:00 AM" from MEMORY.md
- Ran: ha_query.py "SELECT * FROM backup_log WHERE date >= '2026-03-09'"
- Result: Backup completed successfully at 02:15 AM

Recorded to memory/2026-03-10.md:
- Database backup verified for 2026-03-09
- Backup size: 2.3GB (normal range)
- No errors detected in backup log

📊 09:00 - Project Meeting Follow-up

User: "Please summarize yesterday's meeting notes and send the team update"

Process:
1. Semantic search: "project meeting 2026-03-09"
2. Found relevant notes in memory/2026-03-09.md
3. Generated summary with action items
4. Updated [[Project Status Report]] in Obsidian
5. Sent email update to team members

New memory entry:
- Meeting follow-up completed
- 3 action items assigned, deadlines set
- Team acknowledged receipt of summary

🚨 11:45 - Problem Detection

System Alert: Database connection timeout detected

Immediate actions:
1. Checked memory for similar issues: Found "MySQL timeout at 14:30" from 2026-03-08
2. Applied previous solution: Increased wait_timeout parameter
3. Verified fix: Connection restored
4. Updated troubleshooting guide in Knowledge Graph

Lessons recorded:
- MySQL timeout pattern: Occurs during high load periods
- Solution: wait_timeout = 300 seconds works consistently
- Prevention: Monitor connection pool during peak hours

📝 14:00 - Knowledge Graph Update

Proactive memory maintenance:
- Added connection between [[Database Issues]] and [[Performance Monitoring]]
- Updated [[Troubleshooting Guide]] with new timeout solution
- Created link from [[Project Timeline]] to [[Technical Challenges]]
- Synced changes to Git repository

Knowledge network growth:
- New connections: 4
- Updated pages: 3
- Bi-directional links created: 6

🧹 16:00 - Memory Distillation

Afternoon memory review:
1. Analyzed daily log entries from past 3 days
2. Identified patterns: Database issues occur 11:00-12:00 daily
3. Extracted key lesson: Preemptive connection pool scaling needed
4. Updated MEMORY.md with new operational insight
5. Scheduled proactive monitoring for tomorrow 10:45

Distillation results:
- 3 new facts added to MEMORY.md
- 2 outdated entries removed
- 1 process improvement identified

🌆 18:00 - Daily Summary & Planning

End-of-day heartbeat:
- Tasks completed: 12/12
- Issues resolved: 1 (database timeout)
- Memory entries: 8 new, 2 updated
- Knowledge graph: 4 new connections
- Tomorrow's priorities: Implement connection monitoring, team check-in

Final memory state:
- Daily log size: 3.2KB
- MEMORY.md updated: Yes
- Obsidian synced: Yes
- Backup verified: Yes

HEARTBEAT_OK - Day completed successfully

🎯 Key Takeaways

For building a memory and knowledge-management system that actually works, a few principles are worth holding on to:

• 🧠 Memory is not storage — memory is not merely somewhere to put data; it is what gives an AI agent context and continuity
• 📝 Write immediately, don't rely on "mental notes" — anything not written to a file disappears the moment the system restarts
• 🏗️ Three-layer architecture works — splitting memory into short-term, medium-term and long-term is what makes it manageable
• 🔗 Knowledge graphs beat flat files — a networked structure makes information easier to find, and its context easier to understand, than separate files ever will
• 🔍 Semantic search complements keyword search — use semantic search for concepts, use grep for exact text
• 💓 Proactive memory beats reactive memory — the heartbeat system lets an agent notice the events that matter without waiting to be told
• 🔐 Security by design, not afterthought — protecting the data has to be part of the architecture, not something added at the end
• 🧹 Memory hygiene prevents information overload — cleaning and distilling regularly is what keeps performance up
• ⚡ Prepare for memory flushes — the context window is finite, so you need a mechanism for preserving what matters
• 🎯 Quality over quantity in memory management — good memory is not measured by volume, but by how usefully what is stored can be put to work