Skip to content

Development Setup Guide

This guide will walk you through setting up a complete HTM development environment from scratch.

Overview

Setting up HTM for development involves:

  1. Cloning the repository
  2. Installing Ruby and system dependencies
  3. Installing Ruby gem dependencies
  4. Setting up TimescaleDB database
  5. Configuring Ollama for embeddings
  6. Verifying your setup
  7. Running tests and examples

Let's get started!

Prerequisites

Before you begin, ensure you have:

  • macOS, Linux, or WSL2 (Windows Subsystem for Linux)
  • Git installed (git --version)
  • Ruby 3.0 or higher (we'll install this)
  • Internet connection (for downloading dependencies)

Step 1: Clone the Repository

If you plan to submit pull requests, fork the repository first:

  1. Visit https://github.com/madbomber/htm
  2. Click the "Fork" button in the upper right
  3. Clone your fork:
git clone https://github.com/YOUR_USERNAME/htm.git
cd htm

Or Clone Directly (For Read-Only Access)

git clone https://github.com/madbomber/htm.git
cd htm

Add Upstream Remote (For Forked Repos)

If you forked, add the original repository as upstream:

git remote add upstream https://github.com/madbomber/htm.git
git fetch upstream

Step 2: Install Ruby

HTM requires Ruby 3.0 or higher. We recommend using rbenv for managing Ruby versions.

Check Current Ruby Version

ruby --version
# Example output: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23]

If you already have Ruby 3.0+, you can skip to Step 3.

Install rbenv (macOS)

# Install rbenv and ruby-build
brew install rbenv ruby-build

# Initialize rbenv in your shell
echo 'eval "$(rbenv init - bash)"' >> ~/.bashrc
source ~/.bashrc

# Verify installation
rbenv --version

Install rbenv (Linux)

# Clone rbenv
git clone https://github.com/rbenv/rbenv.git ~/.rbenv

# Add to PATH
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init - bash)"' >> ~/.bashrc
source ~/.bashrc

# Install ruby-build
git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

Install Ruby 3.3 (Latest Stable)

# List available Ruby versions
rbenv install --list

# Install Ruby 3.3.0 (or latest 3.x version)
rbenv install 3.3.0

# Set as global default
rbenv global 3.3.0

# Verify installation
ruby --version
# Should show: ruby 3.3.0

Step 3: Install Ruby Dependencies

HTM uses Bundler to manage Ruby gem dependencies.

Install Bundler

gem install bundler

# Verify installation
bundle --version

Install Project Dependencies

# From the htm directory
bundle install

This will install:

  • pg: PostgreSQL adapter
  • pgvector: Vector similarity search support
  • connection_pool: Database connection pooling
  • tiktoken_ruby: Token counting for working memory
  • ruby_llm: LLM client for embeddings
  • rake: Task automation
  • minitest: Testing framework
  • minitest-reporters: Test output formatting
  • debug_me: Debugging utility

Verify Installation

bundle exec ruby -e "require 'htm'; puts HTM::VERSION"
# Should output: 0.1.0 (or current version)

Step 4: Set Up TimescaleDB Database

HTM requires PostgreSQL with TimescaleDB and pgvector extensions. You have two options:

This is the fastest way to get a working database:

1. Create Account

Visit https://www.timescale.com/ and sign up for a free account.

2. Create Service

  • Click "Create Service"
  • Select your region (choose closest to you)
  • Choose the Free Tier (or your preferred plan)
  • Click "Create Service"
  • Wait 2-3 minutes for provisioning

3. Get Connection Details

  • Click on your new service
  • Click "Connection Info"
  • Copy the full connection string (looks like postgres://username:password@host:port/database?sslmode=require)

4. Configure Environment Variables

Create or edit ~/.bashrc__tiger:

# TimescaleDB Connection Configuration
export HTM_SERVICE_NAME="db-67977"  # Your service name
export HTM_DBNAME="tsdb"
export HTM_DBUSER="tsdbadmin"
export HTM_DBPASS="your_password_here"
export HTM_DBPORT="37807"  # Your port number
export HTM_DBURL="postgres://tsdbadmin:your_password@host:port/tsdb?sslmode=require"

Replace the placeholders with your actual connection details.

5. Load Environment Variables

# Load configuration
source ~/.bashrc__tiger

# Optionally, add to your ~/.bashrc for automatic loading
echo 'source ~/.bashrc__tiger' >> ~/.bashrc

Option B: Local PostgreSQL with Docker (Advanced)

For local development with Docker:

# Create docker-compose.yml
cat > docker-compose.yml <<'EOF'
version: '3.8'
services:
  timescaledb:
    image: timescale/timescaledb-ha:pg17
    environment:
      POSTGRES_USER: tsdbadmin
      POSTGRES_PASSWORD: devpassword
      POSTGRES_DB: tsdb
    ports:
      - "5432:5432"
    volumes:
      - timescale_data:/var/lib/postgresql/data

volumes:
  timescale_data:
EOF

# Start TimescaleDB
docker-compose up -d

# Configure environment variables
cat > ~/.bashrc__tiger <<'EOF'
export HTM_SERVICE_NAME="local-dev"
export HTM_DBNAME="tsdb"
export HTM_DBUSER="tsdbadmin"
export HTM_DBPASS="devpassword"
export HTM_DBPORT="5432"
export HTM_DBURL="postgres://tsdbadmin:devpassword@localhost:5432/tsdb?sslmode=disable"
EOF

source ~/.bashrc__tiger

Verify Database Connection

Test your database connection:

# From the htm directory
ruby test_connection.rb

Expected output:

Connected successfully!
TimescaleDB Extension: Version 2.22.1
pgvector Extension: Version 0.8.1
pg_trgm Extension: Version 1.6

Enable Required Extensions

Run the extension setup script:

ruby enable_extensions.rb

This ensures that TimescaleDB, pgvector, and pg_trgm extensions are enabled.

Step 5: Set Up Ollama for Embeddings

HTM uses Ollama (via RubyLLM) to generate vector embeddings for semantic search.

Install Ollama

macOS

# Download and install from official site
curl https://ollama.ai/install.sh | sh

# Or using Homebrew
brew install ollama

Linux

curl https://ollama.ai/install.sh | sh

Start Ollama Service

Ollama typically starts automatically after installation. Verify it's running:

# Check if Ollama is running
curl http://localhost:11434/api/version

# Expected output:
# {"version":"0.1.x"}

If not running, start it manually:

# macOS - Ollama runs as a background service
# Check Activity Monitor or start from Applications

# Linux
ollama serve &

Pull the gpt-oss Model

HTM uses the gpt-oss model by default:

# Pull the model (downloads ~4GB)
ollama pull gpt-oss

# Verify the model is available
ollama list
# Should show gpt-oss in the list

Test Embedding Generation

# Test that embeddings work
ollama run gpt-oss "Hello, HTM!"

Optional: Configure Custom Ollama URL

If Ollama is running on a different host or port:

# Add to ~/.bashrc__tiger
export OLLAMA_URL="http://custom-host:11434"

Step 6: Initialize Database Schema

Now that everything is set up, initialize the HTM database schema:

# Run database setup
rake db_setup

This creates all required tables, indexes, views, and triggers. See the Schema Documentation for details.

Alternative: Manual Setup

You can also run the schema SQL directly:

# Using psql
psql $HTM_DBURL -f sql/schema.sql

# Or using Ruby
ruby -r ./lib/htm -e "HTM::Database.setup"

Step 7: Verify Your Setup

Let's make sure everything is working correctly.

Run the Test Suite

# Run all tests
rake test

Expected output:

HTMTest
  test_version_exists                             PASS (0.00s)
  test_version_format                             PASS (0.00s)
  test_htm_class_exists                           PASS (0.00s)
  ...

Finished in 0.05s
12 tests, 0 failures, 0 errors, 0 skips

Run Integration Tests

Integration tests require a working database:

ruby test/integration_test.rb

Run the Example

Test the full workflow with a real example:

rake example

# Or directly
ruby examples/basic_usage.rb

Expected output:

HTM Basic Usage Example
============================================================

1. Initializing HTM for 'Code Helper' robot...
   Using RubyLLM with Ollama provider and gpt-oss model for embeddings
✓ HTM initialized
  Robot ID: robot-abc123
  Robot Name: Code Helper
  Embedding Service: Ollama (gpt-oss via RubyLLM)

2. Adding memory nodes...
✓ Added decision about database choice
✓ Added decision about RAG approach
✓ Added fact about user preferences
...

Development Tools

HTM includes several Rake tasks to streamline development:

Available Rake Tasks

# Show all available tasks
rake --tasks

# Output:
# rake db_setup    # Run database setup
# rake db_test     # Test database connection
# rake example     # Run example
# rake stats       # Show gem statistics
# rake test        # Run tests

Common Development Commands

# Run all tests
rake test

# Test database connection
rake db_test

# Run example
rake example

# Show code statistics
rake stats

# Setup database schema
rake db_setup

Environment Configuration

HTM uses environment variables for configuration. Here's a complete reference:

Database Variables

Variable Description Example
HTM_DBURL Full PostgreSQL connection URL (preferred) postgres://user:pass@host:port/db?sslmode=require
HTM_DBNAME Database name tsdb
HTM_DBUSER Database username tsdbadmin
HTM_DBPASS Database password your_password
HTM_DBPORT Database port 37807
HTM_SERVICE_NAME Service identifier (informational) db-67977

Ollama Variables

Variable Description Example
OLLAMA_URL Ollama API URL (optional) http://localhost:11434

Managing Environment Files

You can organize your environment variables using multiple files:

# ~/.bashrc__tiger - Database configuration
# ~/.bashrc__ollama - Ollama configuration (if needed)

# Load all configuration files in ~/.bashrc
source ~/.bashrc__tiger

Troubleshooting

Common Setup Issues

"Cannot connect to database"

Symptoms: Connection refused or timeout errors

Solutions:

# Verify environment variables are set
echo $HTM_DBURL

# Test connection directly with psql
psql $HTM_DBURL

# Check if service is running (TimescaleDB Cloud)
# Visit your Timescale Cloud dashboard

# For Docker, check if container is running
docker ps | grep timescale

"Ollama connection refused"

Symptoms: Embedding generation fails

Solutions:

# Verify Ollama is running
curl http://localhost:11434/api/version

# Start Ollama service
# macOS: Start from Applications or Activity Monitor
# Linux: ollama serve &

# Check if model is downloaded
ollama list | grep gpt-oss

# Pull model if missing
ollama pull gpt-oss

"Extension not available"

Symptoms: Errors about missing TimescaleDB or pgvector

Solutions:

# Re-run extension setup
ruby enable_extensions.rb

# Check extension status
psql $HTM_DBURL -c "SELECT extname, extversion FROM pg_extension ORDER BY extname"

# For TimescaleDB Cloud, extensions should be pre-installed
# For local PostgreSQL, ensure you're using timescale/timescaledb-ha image

"Bundle install fails"

Symptoms: Gem installation errors

Solutions:

# Ensure you have development tools
# macOS:
xcode-select --install

# Linux (Ubuntu/Debian):
sudo apt-get install build-essential libpq-dev

# Update RubyGems and Bundler
gem update --system
gem install bundler

# Clear bundle cache and retry
bundle clean --force
bundle install

"Test failures"

Symptoms: Tests fail with database or connection errors

Solutions:

# Ensure database is set up
rake db_setup

# Verify environment variables
source ~/.bashrc__tiger
env | grep TIGER

# Check Ollama is running
curl http://localhost:11434/api/version

# Run tests with verbose output
rake test TESTOPTS="-v"

SSL/TLS Issues

If you see SSL certificate errors:

# Ensure sslmode is set in connection URL
echo $HTM_DBURL | grep sslmode
# Should show: sslmode=require

# For local development without SSL
export HTM_DBURL="postgres://user:pass@localhost:5432/tsdb?sslmode=disable"

Ruby Version Issues

If you see Ruby version errors:

# Check Ruby version
ruby --version

# Update to Ruby 3.3
rbenv install 3.3.0
rbenv global 3.3.0

# Reinstall gems
bundle install

Development Best Practices

Keep Your Fork Updated

If you forked the repository, regularly sync with upstream:

# Fetch upstream changes
git fetch upstream

# Merge into your main branch
git checkout main
git merge upstream/main

# Push to your fork
git push origin main

Use Feature Branches

Always create a branch for your changes:

# Create and switch to feature branch
git checkout -b feature/my-new-feature

# Make changes, commit, push
git add .
git commit -m "Add my new feature"
git push origin feature/my-new-feature

Run Tests Before Committing

Always run the test suite before committing:

# Run all tests
rake test

# If tests pass, commit
git commit -m "Your commit message"

Use debug_me for Debugging

HTM uses the debug_me gem for debugging. Don't use puts statements:

require 'debug_me'

def some_method(param)
  debug_me { [ :param ] }  # Outputs: param = "value"

  # For simple strings
  debug_me "Reached this point in execution"

  # Your code here
end

Next Steps

Now that your development environment is set up:

  1. Learn about Testing: Understand HTM's test suite and write your own tests
  2. Read Contributing Guide: Learn our workflow and submit your first PR
  3. Explore the Schema: Understand the database architecture
  4. Check the Roadmap: See what features are planned

Getting Help

If you encounter issues not covered here:

Happy developing! We look forward to your contributions.