Skip to content

Persistence Options

KBS offers multiple storage backends for persistent facts. This guide helps you choose and configure the right storage for your use case.

Storage Backends

1. No Persistence (Default)

Transient in-memory facts using KBS::Engine:

engine = KBS::Engine.new

# Facts exist only in memory
engine.add_fact(:sensor, { temp: 28 })

# Lost on exit

When to use: - Prototyping - Short-lived processes - Pure computation (no state retention needed) - Testing

Pros: - ✅ Fastest (no I/O) - ✅ Zero configuration - ✅ Simple

Cons: - ❌ No persistence - ❌ Lost on crash - ❌ No audit trail

2. SQLite (Default Persistent)

Embedded database using KBS::Blackboard::Engine:

engine = KBS::Blackboard::Engine.new(db_path: 'kb.db')

engine.add_fact(:sensor, { temp: 28 })
engine.close

# Next run
engine = KBS::Blackboard::Engine.new(db_path: 'kb.db')
puts engine.facts.size  # => 1 (persisted)

When to use: - Single-process applications - Moderate fact volumes (< 1M facts) - ACID transaction requirements - Complete audit trails - No external dependencies

Pros: - ✅ Embedded (no server) - ✅ ACID guarantees - ✅ Durable - ✅ Full audit trail - ✅ SQL queries available

Cons: - ❌ Slower than Redis - ❌ Single writer - ❌ Not distributed

Configuration:

engine = KBS::Blackboard::Engine.new(
  db_path: 'kb.db',          # Database file path
  journal_mode: 'WAL'        # WAL mode for better concurrency
)

3. Redis (High Performance)

In-memory data structure store:

require 'kbs/blackboard/persistence/redis_store'

store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0'
)

engine = KBS::Blackboard::Engine.new(store: store)

When to use: - High-frequency updates (> 1000 writes/sec) - Real-time systems (trading, IoT) - Distributed systems (multiple engines) - Large fact volumes - Speed is critical

Pros: - ✅ 100x faster than SQLite - ✅ Distributed (multiple engines share data) - ✅ Perfect for real-time - ✅ Scalable

Cons: - ❌ Requires Redis server - ❌ Volatile by default (enable RDB/AOF for persistence) - ❌ No ACID across keys - ❌ Less audit trail

Configuration:

store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0',
  namespace: 'kbs',              # Key prefix
  ttl: 86400                     # Expire facts after 24h (optional)
)

Redis Persistence Options:

# In redis.conf:

# RDB: Point-in-time snapshots
save 900 1      # Save after 900s if 1 key changed
save 300 10     # Save after 300s if 10 keys changed

# AOF: Append-only file (durability)
appendonly yes
appendfsync everysec  # Sync to disk every second

4. Hybrid (Best of Both)

Combines Redis (speed) with SQLite (durability):

require 'kbs/blackboard/persistence/hybrid_store'

store = KBS::Blackboard::Persistence::HybridStore.new(
  redis_url: 'redis://localhost:6379/0',
  db_path: 'audit.db'
)

engine = KBS::Blackboard::Engine.new(store: store)

How it works: - Facts: Stored in Redis (fast access) - Audit log: Written to SQLite (durable history) - Messages: Redis sorted sets (fast priority queue)

When to use: - Production systems requiring both speed and auditing - Regulatory compliance (need audit trail) - High-frequency updates with history requirements - Distributed systems needing accountability

Pros: - ✅ Fast fact access (Redis) - ✅ Durable audit trail (SQLite) - ✅ Best of both worlds - ✅ Can reconstruct from audit log

Cons: - ❌ Requires both Redis and SQLite - ❌ More complex setup - ❌ Slightly slower writes (dual write)

Configuration:

store = KBS::Blackboard::Persistence::HybridStore.new(
  redis_url: 'redis://localhost:6379/0',
  db_path: 'audit.db',
  audit_facts: true,        # Log fact changes to SQLite
  audit_rules: true         # Log rule firings to SQLite
)

Choosing a Backend

Decision Tree

Need persistence?
├─ No → KBS::Engine (default)
└─ Yes
   │
   ├─ Need speed > 1000 ops/sec?
   │  ├─ Yes
   │  │  ├─ Need audit trail?
   │  │  │  ├─ Yes → Hybrid Store
   │  │  │  └─ No → Redis Store
   │  │  └─ No → Redis Store
   │  │
   │  └─ No
   │     ├─ Need distributed access?
   │     │  └─ Yes → Redis Store
   │     └─ No → SQLite Store
   │
   └─ Single machine, moderate load?
      └─ SQLite Store

By Use Case

IoT / Real-Time Sensors: 

# High frequency, need speed
store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0'
)

Trading Systems: 

# Speed + audit trail for compliance
store = KBS::Blackboard::Persistence::HybridStore.new(
  redis_url: 'redis://localhost:6379/0',
  db_path: 'trading_audit.db'
)

Expert Systems: 

# Moderate load, need durability
engine = KBS::Blackboard::Engine.new(db_path: 'expert.db')

Development / Testing: 

# No persistence needed
engine = KBS::Engine.new

Migration Between Backends

SQLite → Redis

# 1. Load from SQLite
sqlite_engine = KBS::Blackboard::Engine.new(db_path: 'old.db')
facts = sqlite_engine.facts

# 2. Save to Redis
redis_store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0'
)
redis_engine = KBS::Blackboard::Engine.new(store: redis_store)

facts.each do |fact|
  redis_engine.add_fact(fact.type, fact.attributes)
end

Redis → SQLite

# 1. Load from Redis
redis_store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0'
)
redis_engine = KBS::Blackboard::Engine.new(store: redis_store)
facts = redis_engine.facts

# 2. Save to SQLite
sqlite_engine = KBS::Blackboard::Engine.new(db_path: 'new.db')

facts.each do |fact|
  sqlite_engine.add_fact(fact.type, fact.attributes)
end

sqlite_engine.close

Performance Comparison

Write Performance

require 'benchmark'

# SQLite
sqlite_engine = KBS::Blackboard::Engine.new(db_path: 'perf.db')
Benchmark.bm do |x|
  x.report("SQLite writes:") do
    10_000.times { |i| sqlite_engine.add_fact(:test, { value: i }) }
  end
end
# ~5,000 ops/sec

# Redis
redis_store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0'
)
redis_engine = KBS::Blackboard::Engine.new(store: redis_store)
Benchmark.bm do |x|
  x.report("Redis writes:") do
    10_000.times { |i| redis_engine.add_fact(:test, { value: i }) }
  end
end
# ~50,000 ops/sec (10x faster)

Read Performance

# SQLite
Benchmark.bm do |x|
  x.report("SQLite reads:") do
    10_000.times { sqlite_engine.facts }
  end
end
# ~10,000 ops/sec

# Redis
Benchmark.bm do |x|
  x.report("Redis reads:") do
    10_000.times { redis_engine.facts }
  end
end
# ~100,000 ops/sec (10x faster)

Custom Persistence

Implement your own backend by subclassing KBS::Blackboard::Persistence::Store:

class PostgresStore < KBS::Blackboard::Persistence::Store
  def save_fact(fact)
    # Insert into PostgreSQL
  end

  def load_facts(type = nil)
    # Query from PostgreSQL
  end

  def delete_fact(id)
    # Delete from PostgreSQL
  end

  def save_message(topic, message, priority)
    # Store message
  end

  def pop_message(topic)
    # Retrieve highest priority message
  end

  def log_fact_change(operation, fact)
    # Audit logging
  end

  def fact_history(fact_id)
    # Get change history
  end
end

# Use custom store
store = PostgresStore.new(connection_string: "...")
engine = KBS::Blackboard::Engine.new(store: store)

See Custom Persistence for details.

Configuration Best Practices

SQLite

# Enable WAL mode for better concurrency
engine = KBS::Blackboard::Engine.new(
  db_path: 'kb.db',
  journal_mode: 'WAL',
  synchronous: 'NORMAL',  # Trade some durability for speed
  cache_size: -64000      # 64MB cache
)

Redis

# Connection pooling for high concurrency
store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0',
  pool_size: 10,          # Connection pool size
  pool_timeout: 5         # Timeout in seconds
)

Hybrid

# Balance between speed and durability
store = KBS::Blackboard::Persistence::HybridStore.new(
  redis_url: 'redis://localhost:6379/0',
  db_path: 'audit.db',
  batch_audit_writes: true,   # Batch SQLite writes
  audit_batch_size: 100       # Flush every 100 changes
)

Troubleshooting

SQLite Database Locked

# Increase busy timeout
engine = KBS::Blackboard::Engine.new(
  db_path: 'kb.db',
  busy_timeout: 5000  # Wait up to 5 seconds
)

Redis Connection Issues

# Enable retry logic
store = KBS::Blackboard::Persistence::RedisStore.new(
  url: 'redis://localhost:6379/0',
  reconnect_attempts: 3,
  reconnect_delay: 1.0
)

Hybrid Sync Issues

# Force synchronous audit writes
store = KBS::Blackboard::Persistence::HybridStore.new(
  redis_url: 'redis://localhost:6379/0',
  db_path: 'audit.db',
  sync_audit_writes: true  # Don't batch, write immediately
)

Next Steps

Choose your backend based on speed, durability, and distribution requirements.