Skip to content

Class HTM::LongTermMemory

Inherits Object
Includes HTM::LongTermMemory::FulltextSearch, HTM::LongTermMemory::HybridSearch, HTM::LongTermMemory::NodeOperations, HTM::LongTermMemory::RelevanceScorer, HTM::LongTermMemory::RobotOperations, HTM::LongTermMemory::TagOperations, HTM::LongTermMemory::VectorSearch
Defined in lib/htm/long_term_memory.rb, lib/htm/long_term_memory/hybrid_search.rb, lib/htm/long_term_memory/vector_search.rb, lib/htm/long_term_memory/tag_operations.rb, lib/htm/long_term_memory/fulltext_search.rb, lib/htm/long_term_memory/node_operations.rb, lib/htm/long_term_memory/relevance_scorer.rb, lib/htm/long_term_memory/robot_operations.rb

Long-term Memory - PostgreSQL-backed permanent storage

LongTermMemory provides durable storage for all memory nodes with: * Vector similarity search (RAG) * Full-text search * Time-range queries * Relationship graphs * Tag system * Sequel ORM for data access * Query result caching for efficiency

This class uses standalone utility classes and modules:

Standalone classes (used via class methods or instances): * HTM::SqlBuilder: SQL condition building helpers (class methods) * HTM::QueryCache: Query result caching (instantiated as @cache)

Included modules: * RelevanceScorer: Dynamic relevance scoring * NodeOperations: Node CRUD operations * RobotOperations: Robot registration and activity * TagOperations: Tag management * VectorSearch: Vector similarity search * FulltextSearch: Full-text search * HybridSearch: Combined search strategies

Constants

DEFAULT_CACHE_SIZE

Number of queries to cache

DEFAULT_CACHE_TTL

Cache lifetime in seconds (5 minutes)

DEFAULT_QUERY_TIMEOUT

milliseconds (30 seconds)

MAX_VECTOR_DIMENSION

Maximum supported dimension with HNSW index (pgvector limitation)

Attributes

query_timeout [R]

Returns the value of attribute query_timeout.

Public Instance Methods

add(content:, robot_id:, token_count: 0, embedding: nil, metadata: {})

Add a node to long-term memory (with deduplication)

If content already exists (by content_hash), links the robot to the existing node and updates timestamps. Otherwise creates a new node. - @param content [String] Conversation message/utterance - @param token_count [Integer] Token count - @param robot_id [Integer] Robot identifier - @param embedding [Array, nil] Pre-generated embedding vector - @param metadata [Hash] Flexible metadata for the node (default: {}) - @raise [ArgumentError] If metadata is not a Hash - @return [Hash] { node_id:, is_new:, robot_node: }

add_tag(node_id:, tag:)

Add a tag to a node (creates tag and all parent tags)

When adding a hierarchical tag like "database:postgresql:extensions", this also creates and associates the parent tags "database" and "database:postgresql" with the node. - @param node_id [Integer] Node database ID - @param tag [String] Tag name - @return [void]

@example 

add_tag(node_id: 123, tag: "database:postgresql:extensions")
# Creates tags: "database", "database:postgresql", "database:postgresql:extensions"
# Associates all three with node 123

batch_load_node_tags(node_ids)

Batch load tags for multiple nodes (avoids N+1 queries) - @param node_ids [Array] Node database IDs - @return [Hash<Integer, Array>] Map of node_id to array of tag names

calculate_relevance(node:, query_tags: [], vector_similarity: nil, node_tags: nil)

Calculate dynamic relevance score for a node given query context - @param node [Hash] Node data with similarity, tags, created_at, access_count - @param query_tags [Array] Tags associated with the query - @param vector_similarity [Float, nil] Pre-computed vector similarity (0-1) - @param node_tags [Array, nil] Pre-loaded tags for this node (avoids N+1 query) - @return [Float] Composite relevance score (RELEVANCE_MIN to RELEVANCE_MAX)

clear_cache!()

Clear the query result cache - @return [void]

delete(node_id)

Delete a node - @param node_id [Integer] Node database ID - @return [void]

exists?(node_id)

Check if a node exists - @param node_id [Integer] Node database ID - @return [Boolean] True if node exists

fetch_candidates_by_timeframe(timeframe:, metadata:, limit:)

Fetch candidates by timeframe using raw SQL (avoids ORM overhead) - @param timeframe [nil, Range, Array] Time range(s) to search - @param metadata [Hash] Filter by metadata fields - @param limit [Integer] Maximum results - @return [Array] Candidate nodes as hashes

find_query_matching_tags(query, include_extracted: false)

Find tags that match terms in the query

Searches the tags table for tags where any hierarchy level matches query words. Uses semantic extraction via LLM to find relevant tags.

Performance: Uses a single UNION query instead of multiple sequential queries. - @param query [String] Search query - @param include_extracted [Boolean] If true, returns hash with :extracted and :matched keys - @return [Array] Matching tag names (default) - @return [Hash] If include_extracted: { extracted: [...], matched: [...] }

get_node_tags(node_id)

Get tags for a specific node - @param node_id [Integer] Node database ID - @return [Array] Tag names

initialize(config, pool_size: nil, query_timeout: DEFAULT_QUERY_TIMEOUT, cache_size: DEFAULT_CACHE_SIZE, cache_ttl: DEFAULT_CACHE_TTL)

Initialize long-term memory storage - @param config [Hash] Database configuration (host, port, dbname, user, password) - @param pool_size [Integer, nil] Connection pool size (uses Sequel default if nil) - @param query_timeout [Integer] Query timeout in milliseconds (default: 30000) - @param cache_size [Integer] Number of query results to cache (default: 1000, use 0 to disable) - @param cache_ttl [Integer] Cache time-to-live in seconds (default: 300) - @return [LongTermMemory] a new instance of LongTermMemory

@example Initialize with defaults 

ltm = LongTermMemory.new(HTM::Database.default_config)

@example Initialize with custom cache settings 

ltm = LongTermMemory.new(config, cache_size: 500, cache_ttl: 600)

@example Disable caching 

ltm = LongTermMemory.new(config, cache_size: 0)

Link a robot to a node (create or update robot_node record) - @param robot_id [Integer] Robot ID - @param node [HTM::Models::Node] Node to link - @param working_memory [Boolean] Whether node is in working memory (default: false) - @return [HTM::Models::RobotNode] The robot_node link record

mark_evicted(robot_id:, node_ids:)

Mark nodes as evicted from working memory

Sets working_memory = false on the robot_nodes join table for the specified robot and node IDs. - @param robot_id [Integer] Robot ID whose working memory is being evicted - @param node_ids [Array] Node IDs to mark as evicted - @return [void]

node_topics(node_id)

Get topics for a specific node - @param node_id [Integer] Node database ID - @return [Array] Topic paths

nodes_by_topic(topic_path, exact: false, fuzzy: false, min_similarity: DEFAULT_TAG_SIMILARITY_THRESHOLD, limit: 50)

Retrieve nodes by ontological topic

Matching modes (in order of precedence): * exact: true - Only exact tag name match * fuzzy: true - Trigram similarity search (typo-tolerant) * default - LIKE prefix match (e.g., "database" matches "database:postgresql") - @param topic_path [String] Topic hierarchy path - @param exact [Boolean] Exact match only (highest priority) - @param fuzzy [Boolean] Use trigram similarity for typo-tolerant search - @param min_similarity [Float] Minimum similarity for fuzzy mode (0.0-1.0) - @param limit [Integer] Maximum results (capped at MAX_TAG_QUERY_LIMIT) - @return [Array] Matching nodes

ontology_structure()

Get ontology structure view - @return [Array] Ontology structure

pool_size()

For backwards compatibility with tests/code that expect pool_size

Get most popular tags - @param limit [Integer] Number of tags to return (capped at MAX_TAG_QUERY_LIMIT) - @param timeframe [Range, nil] Optional time range filter - @return [Array] Tags with usage counts

recency_half_life_hours()

Not documented.

register_robot(robot_name)

Register a robot - @param robot_name [String] Robot name - @return [Integer] Robot ID

retrieve(node_id)

Retrieve a node by ID

Automatically tracks access by incrementing access_count and updating last_accessed. Uses a single UPDATE query instead of separate increment! and touch calls. - @param node_id [Integer] Node database ID - @return [Hash, nil] Node data or nil

search(timeframe:, query:, limit:, embedding_service:, metadata: {})

Vector similarity search - @param timeframe [nil, Range, Array] Time range(s) to search (nil = no filter) - @param query [String] Search query - @param limit [Integer] Maximum results (capped at MAX_VECTOR_LIMIT) - @param embedding_service [Object] Service to generate embeddings - @param metadata [Hash] Filter by metadata fields (default: {}) - @return [Array] Matching nodes

search_by_tags(tags:, match_all: false, timeframe: nil, limit: 20)

Search nodes by tags - @param tags [Array] Tags to search for - @param match_all [Boolean] If true, match ALL tags; if false, match ANY tag - @param timeframe [Range, nil] Optional time range filter - @param limit [Integer] Maximum results - @return [Array] Matching nodes with relevance scores

search_fulltext(timeframe:, query:, limit:, metadata: {})

Full-text search - @param timeframe [Range] Time range to search - @param query [String] Search query - @param limit [Integer] Maximum results (capped at MAX_FULLTEXT_LIMIT) - @param metadata [Hash] Filter by metadata fields (default: {}) - @return [Array] Matching nodes

search_hybrid(timeframe:, query:, limit:, embedding_service:, prefilter_limit: 100, metadata: {})

Hybrid search using Reciprocal Rank Fusion - @param timeframe [Range] Time range to search - @param query [String] Search query - @param limit [Integer] Maximum results (capped at MAX_HYBRID_LIMIT) - @param embedding_service [Object] Service to generate embeddings - @param prefilter_limit [Integer] Candidates per search (default: 100) - @param metadata [Hash] Filter by metadata fields (default: {}) - @return [Array] Matching nodes

search_tags(query, limit: 20, min_similarity: DEFAULT_TAG_SIMILARITY_THRESHOLD)

Fuzzy search for tags using trigram similarity

Uses PostgreSQL pg_trgm extension to find tags that are similar to the query string, tolerating typos and partial matches. - @param query [String] Search query (tag name or partial) - @param limit [Integer] Maximum results (capped at MAX_TAG_QUERY_LIMIT) - @param min_similarity [Float] Minimum similarity threshold (0.0-1.0) - @return [Array] Matching tags with similarity scores Each hash contains: { name: String, similarity: Float }

search_with_relevance(timeframe:, query: nil, query_tags: [], limit: 20, embedding_service: nil, metadata: {})

Search with dynamic relevance scoring

Returns nodes with calculated relevance scores based on query context - @param timeframe [nil, Range, Array] Time range(s) to search (nil = no filter) - @param query [String, nil] Search query - @param query_tags [Array] Tags to match - @param limit [Integer] Maximum results - @param embedding_service [Object, nil] Service to generate embeddings - @param metadata [Hash] Filter by metadata fields (default: {}) - @return [Array] Nodes with relevance scores

shutdown()

Shutdown - no-op with Sequel (connection pool managed by Sequel)

stats()

Get memory statistics - @return [Hash] Statistics

topic_relationships(min_shared_nodes: 2, limit: 50)

Get topic relationships (co-occurrence) - @param min_shared_nodes [Integer] Minimum shared nodes - @param limit [Integer] Maximum relationships (capped at MAX_TAG_QUERY_LIMIT) - @return [Array] Topic relationships

track_access(node_ids)

Track access for multiple nodes (bulk operation)

Updates access_count and last_accessed for all nodes in the array - @param node_ids [Array] Node IDs that were accessed - @return [void]

update_last_accessed(node_id)

Update last_accessed timestamp - @param node_id [Integer] Node database ID - @return [void]

update_robot_activity(robot_id)

Update robot activity timestamp - @param robot_id [Integer] Robot identifier - @return [void]

weight_access()

Not documented.

weight_recency()

Not documented.

weight_semantic()

Configurable scoring weights (via HTM.configuration)

weight_tag()

Not documented.