Testing Guide¶

HTM follows a comprehensive testing philosophy to ensure reliability, correctness, and maintainability. This guide covers everything you need to know about testing HTM.

Testing Philosophy¶

Core Principles¶

HTM's testing approach is guided by these principles:

Test Everything: Every feature must have corresponding tests
Test in Isolation: Methods should be testable independently
Test Real Scenarios: Integration tests with actual database
Test Edge Cases: Don't just test the happy path
Keep Tests Fast: Optimize for quick feedback loops
Keep Tests Clear: Tests are documentation

Test-Driven Development¶

We encourage (but don't strictly require) test-driven development:

Write the test first - Define expected behavior
Watch it fail - Ensure the test actually tests something
Implement the feature - Make the test pass
Refactor - Clean up while tests keep you safe

Test Suite Overview¶

HTM uses Minitest as its testing framework. The test suite is organized into three categories:

Unit Tests¶

Purpose: Test individual methods and classes in isolation

Location: test/*_test.rb (excluding integration_test.rb)

Characteristics:

Fast execution (milliseconds)
No database required
Mock external dependencies
Test logic and behavior

Example: test/htm_test.rb, test/embedding_service_test.rb

Integration Tests¶

Purpose: Test full workflows with real dependencies

Location: test/integration_test.rb

Characteristics:

Slower execution (seconds)
Requires PostgreSQL
Requires Ollama for embeddings
Tests real-world scenarios
Tests database interactions

Performance Tests¶

Purpose: Ensure performance characteristics

Status: Planned for future implementation

Focus areas:

Query performance
Memory usage
Token counting accuracy
Embedding generation speed

Running Tests¶

Run All Tests¶

# Using Rake (recommended)
rake test

# Using Ruby directly
ruby -Ilib:test test/**/*_test.rb

Expected output:

HTMTest
  test_version_exists                             PASS (0.00s)
  test_version_format                             PASS (0.00s)
  test_htm_class_exists                           PASS (0.00s)
  ...

IntegrationTest
  test_htm_initializes_with_ollama                PASS (0.15s)
  test_add_node_with_embedding                    PASS (0.32s)
  ...

Finished in 2.47s
28 tests, 0 failures, 0 errors, 0 skips

Run Specific Test File¶

# Run unit tests only
ruby test/htm_test.rb

# Run embedding service tests
ruby test/embedding_service_test.rb

# Run integration tests
ruby test/integration_test.rb

Run Specific Test Method¶

# Run a single test method
ruby test/htm_test.rb -n test_version_exists

# Run tests matching a pattern
ruby test/integration_test.rb -n /embedding/

Run Tests with Verbose Output¶

# Verbose output
rake test TESTOPTS="-v"

# Show test names as they run
ruby test/htm_test.rb -v

Run Tests with Debugging¶

# Run with debug output
DEBUG=1 rake test

# Run with Ruby debugger
ruby -r debug test/htm_test.rb

Test Structure and Organization¶

Test File Layout¶

Test Directory Structure

Test File Template¶

Every test file follows this structure:

# frozen_string_literal: true

require "test_helper"

class MyFeatureTest < Minitest::Test
  def setup
    # Runs before each test
    # Initialize test data, mocks, etc.
  end

  def teardown
    # Runs after each test
    # Clean up test data
  end

  def test_something_works
    # Arrange: Set up test data
    input = "test value"

    # Act: Execute the code being tested
    result = MyClass.some_method(input)

    # Assert: Verify the results
    assert_equal "expected", result
  end

  def test_handles_edge_case
    # Test edge cases and error conditions
    assert_raises(ArgumentError) do
      MyClass.some_method(nil)
    end
  end
end

Test Helper Configuration¶

test/test_helper.rb provides shared configuration:

# frozen_string_literal: true

$LOAD_PATH.unshift File.expand_path("../lib", __dir__)
require "htm"

require "minitest/autorun"
require "minitest/reporters"

Minitest::Reporters.use! [Minitest::Reporters::SpecReporter.new]

Writing Tests¶

Unit Test Example¶

Testing a method in isolation:

class WorkingMemoryTest < Minitest::Test
  def setup
    @memory = HTM::WorkingMemory.new(max_tokens: 1000)
  end

  def test_calculates_token_count
    node = { value: "Hello, world!" }

    result = @memory.calculate_tokens(node)

    assert_instance_of Integer, result
    assert result > 0
  end

  def test_rejects_nodes_exceeding_capacity
    @memory = HTM::WorkingMemory.new(max_tokens: 10)
    large_node = { value: "x" * 1000 }

    assert_raises(HTM::WorkingMemoryFullError) do
      @memory.add_node(large_node)
    end
  end
end

Integration Test Example¶

Testing with real database:

class DatabaseIntegrationTest < Minitest::Test
  def setup
    skip "Database not configured" unless ENV['HTM_DBURL']

    @htm = HTM.new(
      robot_name: "Test Robot",
      working_memory_size: 128_000,
      embedding_service: :ollama
    )
  end

  def teardown
    return unless @htm

    # Clean up test data
    @htm.forget("test_node_001", confirm: :confirmed) rescue nil
  end

  def test_adds_and_retrieves_node
    # Add a node
    node_id = @htm.add_node(
      "test_node_001",
      "Test memory content",
      type: :fact,
      importance: 5.0
    )

    assert_instance_of Integer, node_id

    # Retrieve it
    node = @htm.retrieve("test_node_001")

    refute_nil node
    assert_equal "test_node_001", node['key']
    assert_includes node['value'], "Test memory"
  end
end

Testing with Mocks and Stubs¶

For testing without external dependencies:

class EmbeddingServiceTest < Minitest::Test
  def test_generates_embedding_vector
    service = HTM::EmbeddingService.new(:ollama, model: 'gpt-oss')

    # Skip if Ollama is not available
    skip "Ollama not running" unless ollama_available?

    embedding = service.generate_embedding("test text")

    assert_instance_of Array, embedding
    assert_equal 1536, embedding.length
    assert embedding.all? { |v| v.is_a?(Float) }
  end

  private

  def ollama_available?
    require 'net/http'
    uri = URI('http://localhost:11434/api/version')
    response = Net::HTTP.get_response(uri)
    response.is_a?(Net::HTTPSuccess)
  rescue
    false
  end
end

Test Fixtures¶

What are Fixtures?¶

Fixtures are pre-defined test data that can be reused across tests. HTM will use fixtures for complex test scenarios.

Future Fixture Structure¶

# test/fixtures/memories.rb
module Fixtures
  MEMORIES = {
    fact: {
      key: "user_preference_001",
      value: "User prefers debug_me over puts for debugging",
      type: :fact,
      importance: 7.0
    },
    decision: {
      key: "decision_001",
      value: "We decided to use TimescaleDB for time-series optimization",
      type: :decision,
      importance: 9.0,
      tags: ["database", "architecture"]
    }
  }
end

Using Fixtures¶

require_relative 'fixtures/memories'

class MemoryTest < Minitest::Test
  def test_stores_fact
    htm = HTM.new(robot_name: "Test")
    fact = Fixtures::MEMORIES[:fact]

    node_id = htm.add_node(
      fact[:key],
      fact[:value],
      type: fact[:type],
      importance: fact[:importance]
    )

    assert node_id > 0
  end
end

Assertions Reference¶

Common Assertions¶

Minitest provides many assertion methods:

# Equality
assert_equal expected, actual
refute_equal unexpected, actual

# Truth/falsy
assert actual
refute actual
assert_nil value
refute_nil value

# Type checking
assert_instance_of String, value
assert_kind_of Numeric, value

# Collections
assert_includes collection, item
assert_empty collection
refute_empty collection

# Exceptions
assert_raises(ErrorClass) { code }
assert_silent { code }

# Matching
assert_match /pattern/, string
refute_match /pattern/, string

# Comparison
assert_operator 5, :>, 3
assert_in_delta 3.14, Math::PI, 0.01

Custom Assertions¶

You can create custom assertions for HTM-specific checks:

module HTMAssertions
  def assert_valid_embedding(embedding)
    assert_instance_of Array, embedding
    assert_equal 1536, embedding.length
    assert embedding.all? { |v| v.is_a?(Float) }
  end

  def assert_valid_node(node)
    assert_instance_of Hash, node
    assert node.key?('id')
    assert node.key?('key')
    assert node.key?('value')
    assert node.key?('type')
  end
end

class MyTest < Minitest::Test
  include HTMAssertions

  def test_node_structure
    node = create_test_node
    assert_valid_node(node)
  end
end

Mocking and Stubbing¶

When to Mock¶

Mock external dependencies to:

Speed up tests (avoid slow API calls)
Test error conditions
Isolate the code under test
Test without required services

Minitest Mocking¶

Minitest includes built-in mocking:

require 'minitest/mock'

class ServiceTest < Minitest::Test
  def test_calls_external_api
    mock_client = Minitest::Mock.new
    mock_client.expect :call, "response", ["arg"]

    service = MyService.new(client: mock_client)
    result = service.process

    assert_equal "response", result
    mock_client.verify  # Ensures expectations were met
  end
end

Stubbing Methods¶

Temporarily replace method implementations:

class NetworkTest < Minitest::Test
  def test_handles_network_failure
    # Stub a method to simulate failure
    HTM::Database.stub :connected?, false do
      assert_raises(HTM::DatabaseError) do
        htm = HTM.new(robot_name: "Test")
      end
    end
  end
end

Test Coverage¶

Coverage Goals¶

HTM aims for high test coverage:

Unit tests: 90%+ line coverage
Integration tests: Cover all critical paths
Edge cases: Test error conditions
Documentation: Tests serve as usage examples

Measuring Coverage (Future)¶

We plan to add SimpleCov for coverage reporting:

# test/test_helper.rb (future)
require 'simplecov'
SimpleCov.start do
  add_filter '/test/'
  minimum_coverage 90
end

Coverage Report¶

# Generate coverage report
rake test:coverage

# View report
open coverage/index.html

Continuous Integration¶

GitHub Actions (Future)¶

HTM will use GitHub Actions for CI/CD:

# .github/workflows/test.yml (future)
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: timescale/timescaledb-ha:pg17
        env:
          POSTGRES_PASSWORD: testpass
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.3
          bundler-cache: true
      - name: Run tests
        run: bundle exec rake test

CI Requirements¶

All pull requests must:

Pass all tests (100%)
Maintain or improve coverage
Pass style checks (future)
Pass integration tests

Testing Best Practices¶

DO: Write Clear Tests¶

# Good: Clear test name and assertions
def test_working_memory_evicts_least_important_nodes_when_full
  memory = HTM::WorkingMemory.new(max_tokens: 100)
  memory.add_node(key: "important", importance: 9.0, tokens: 50)
  memory.add_node(key: "unimportant", importance: 1.0, tokens: 51)

  assert memory.contains?("important")
  refute memory.contains?("unimportant")
end

# Bad: Unclear test
def test_eviction
  memory = HTM::WorkingMemory.new(max_tokens: 100)
  memory.add_node(key: "a", importance: 9.0, tokens: 50)
  memory.add_node(key: "b", importance: 1.0, tokens: 51)
  assert memory.contains?("a")
end

DO: Test One Thing at a Time¶

# Good: Each test focuses on one behavior
def test_calculates_token_count
  result = calculate_tokens("hello")
  assert result > 0
end

def test_handles_empty_string
  result = calculate_tokens("")
  assert_equal 0, result
end

# Bad: Testing multiple things
def test_token_stuff
  assert calculate_tokens("hello") > 0
  assert_equal 0, calculate_tokens("")
  assert_raises(ArgumentError) { calculate_tokens(nil) }
end

DO: Use Descriptive Test Names¶

# Good: Describes what is being tested
def test_recall_returns_memories_from_specified_timeframe
def test_forget_requires_confirmation_parameter
def test_add_node_generates_embedding_automatically

# Bad: Vague or unclear
def test_recall
def test_forget
def test_add

DO: Clean Up After Tests¶

def setup
  @htm = HTM.new(robot_name: "Test")
  @test_keys = []
end

def teardown
  # Clean up any created nodes
  @test_keys.each do |key|
    @htm.forget(key, confirm: :confirmed) rescue nil
  end
end

def test_adds_node
  key = "test_#{Time.now.to_i}"
  @test_keys << key

  @htm.add_node(key, "content", type: :fact)
  # Test continues...
end

DON'T: Rely on Test Order¶

# Bad: Tests depend on each other
def test_1_creates_node
  @htm.add_node("shared", "value", type: :fact)
end

def test_2_retrieves_node  # Fails if test_1 doesn't run first
  node = @htm.retrieve("shared")
  assert node
end

# Good: Each test is independent
def test_creates_node
  @htm.add_node("test_create", "value", type: :fact)
  node = @htm.retrieve("test_create")
  assert node
end

def test_retrieves_node
  @htm.add_node("test_retrieve", "value", type: :fact)
  node = @htm.retrieve("test_retrieve")
  assert node
end

DON'T: Use Sleep for Timing¶

# Bad: Flaky test with arbitrary sleep
def test_async_operation
  start_operation
  sleep 2  # Hope it finishes in 2 seconds
  assert operation_complete?
end

# Good: Poll with timeout
def test_async_operation
  start_operation
  wait_until(timeout: 5) { operation_complete? }
  assert operation_complete?
end

def wait_until(timeout: 5)
  start = Time.now
  loop do
    return if yield
    raise "Timeout" if Time.now - start > timeout
    sleep 0.1
  end
end

DON'T: Test Implementation Details¶

# Bad: Testing internal implementation
def test_uses_specific_sql_query
  assert_match /SELECT \* FROM/, @htm.instance_variable_get(:@last_query)
end

# Good: Testing behavior/outcome
def test_retrieves_all_node_fields
  @htm.add_node("key", "value", type: :fact)
  node = @htm.retrieve("key")

  assert node.key?('id')
  assert node.key?('key')
  assert node.key?('value')
  assert node.key?('type')
end

Debugging Test Failures¶

Run Single Test with Verbose Output¶

ruby test/htm_test.rb -v -n test_specific_test

Use debug_me in Tests¶

require 'debug_me'

def test_something
  debug_me { [ :input, :expected ] }

  result = method_under_test(input)

  debug_me { [ :result ] }

  assert_equal expected, result
end

Check Test Data¶

def test_database_state
  # Add debugging to inspect state
  pp @htm.memory_stats
  pp @htm.working_memory.inspect

  # Your test assertions
  assert something
end

Use Ruby Debugger¶

# Install debugger
gem install debug

# Run test with debugger
ruby -r debug test/htm_test.rb

# Set breakpoints in test
def test_something
  debugger  # Execution will stop here
  result = method_under_test
  assert result
end

Testing Checklist¶

Before submitting a pull request, ensure:

All existing tests pass
New features have tests
Edge cases are tested
Error conditions are tested
Tests are clear and well-named
Tests are independent (no order dependency)
Integration tests clean up test data
No skipped tests (unless explicitly documented)
Tests run in reasonable time (<5s for unit, <30s for integration)

Resources¶

Minitest Documentation¶

Official docs: https://docs.seattlerb.org/minitest/
Minitest assertions: https://docs.seattlerb.org/minitest/Minitest/Assertions.html
Minitest mocking: https://docs.seattlerb.org/minitest/Minitest/Mock.html

Testing Guides¶

Ruby Testing Guide: https://guides.rubyonrails.org/testing.html
Better Specs: https://www.betterspecs.org/

Next Steps¶

Contributing Guide: Learn how to submit your tests
Database Schema: Understand what you're testing
Setup Guide: Get your test environment running

Happy testing! Remember: Good tests make better code.