# Textrawl

> Personal knowledge server — your documents, searchable and AI-ready

Textrawl is a personal knowledge server with persistent memory, searchable documents, and proactive insights. Access it through the web dashboard, MCP for AI assistants, or the REST API. Backed by Neon PostgreSQL with vector embeddings.

## Server Interfaces

- MCP: Model Context Protocol via StreamableHTTPServerTransport (stateless HTTP)
- REST API: Standard HTTP endpoints for search, documents, and upload
- WebSocket: Real-time event streaming
- Endpoint: POST /mcp
- Authentication: Bearer token (API_BEARER_TOKEN)
- Rate Limits: API 100/min, Upload 10/min
- Response Format: Compact by default (COMPACT_RESPONSES=true, 40-60% fewer tokens)

## Tools

Tools with `outputSchema` return `structuredContent` alongside text content: `search`, `get_document`, `list_documents`, `query_memory`, `query_conversations`, `get_stats`, `health_check`.

### search

Hybrid semantic + full-text search with optional cross-source memory and conversation results.

- query (string, REQUIRED): Natural language search query
- limit (number): Max results 1-50, default 10
- fullTextWeight (number): Keyword weight 0-2, default 1.0
- semanticWeight (number): Semantic weight 0-2, default 1.0
- tags (string[]): Filter to docs with ALL tags
- sourceType (enum): 'note' | 'file' | 'url'
- contentType (enum): Content type filter
- minScore (number): Minimum relevance 0-1
- includeMemories (boolean): Include memory results, default false
- includeConversations (boolean): Include conversation results, default false
- memoryWeight (number): Weight for memory results 0-2
- conversationWeight (number): Weight for conversation results 0-2

### get_document

Retrieve full document by ID.

- documentId (UUID, REQUIRED): The document UUID
- includeChunks (boolean): Include chunks, default false

### list_documents

List documents with pagination.

- limit (number): 1-100, default 20
- offset (number): Pagination offset
- sourceType (enum): 'note' | 'file' | 'url'
- tags (string[]): Filter by tags

### update_document

Update document metadata.

- documentId (UUID, REQUIRED): Document to update
- title (string): New title
- tags (string[]): New tags (replaces existing)

MUST provide title or tags.

### add_note

Create markdown notes with automatic embedding.

- title (string, REQUIRED): Note title (max 500 chars)
- content (string, REQUIRED): Markdown content (max 1MB)
- tags (string[]): Tags for organization

### capture

Workflow tool (compact `normal` surface). Save content to the knowledge base; consolidates add_note + save_url.

- kind (enum, REQUIRED): "note" or "url"
- title (string): note title (required for kind=note) or url title override
- content (string): markdown body (required for kind=note)
- url (string): web page URL (required for kind=url)
- tags (string[]): tags for organization
- audience (enum): private_jeff | family_shared | public_safe

### remember

Workflow tool (compact `normal` surface). Write structured knowledge to the memory graph; consolidates remember_fact + build_knowledge + relate_entities. Requires at least one of facts/relations.

- facts (array): [{ entityName, entityType, observation, source? }] (a single fact is a one-element array)
- relations (array): [{ fromEntity, relation, toEntity }]

## Memory Tools (Persistent Memory)

Enable with ENABLE_MEMORY=true (default). Requires setup-db-memory.sql schema.

### remember_fact

Store facts about entities with semantic embeddings.

- entityName (string, REQUIRED): Entity name (e.g., "Ada", "Project Alpha")
- entityType (enum, REQUIRED): person | concept | project | preference | fact | location | organization
- observation (string, REQUIRED): Single fact to remember (max 2000 chars)
- source (enum): conversation | note | document | manual
- validUntil (string): ISO date for expiring facts

### build_knowledge

Store multiple facts and relations in a single batch call.

- facts (array): Facts to store, max 50. Each: entityName, entityType, observation, source?
- relations (array): Relations to create, max 50. Each: fromEntity, relation, toEntity, fromEntityType?, toEntityType?

### query_memory

Unified memory query tool. Dispatches based on mode.

- mode (enum, REQUIRED): 'search' | 'entity' | 'list'
- query (string): Search query, required when mode is 'search' (max 1000 chars)
- entityTypes (enum[]): Filter by entity types (search and list modes)
- searchMode (enum): hybrid | semantic, default hybrid (search mode)
- entityName (string): Entity to look up, required when mode is 'entity'
- includeRelated (boolean): Include relations in entity mode, default true
- maxObs (number): Max observations per entity
- limit (number): Max results (search: 1-50, default 10; list: 1-100, default 50)
- offset (number): Pagination offset (list mode)

### relate_entities

Create relationships between entities.

- fromEntity (string, REQUIRED): Source entity name
- relation (string, REQUIRED): Relation type (works_at, knows, prefers, etc.)
- toEntity (string, REQUIRED): Target entity name

### forget_entity

Delete entity and all associated memories.

- entityName (string, REQUIRED): Entity to delete
- confirm (boolean, REQUIRED): Must be true

### extract_memories

Extract entities and facts from text using LLM. Requires ENABLE_MEMORY_EXTRACTION=true and ANTHROPIC_API_KEY.

- text (string, REQUIRED): Text to extract from (10-100000 chars)
- source (enum): conversation | note | document | manual, default 'document'
- storeResults (boolean): Store in database, default false (preview only)

## Conversation Tools (Conversation Memory)

Enable with ENABLE_CONVERSATIONS=true (default). Requires setup-db-conversation.sql schema.

### save_conversation_context

Save conversation summary and turns for later recall.

- summary (string, REQUIRED): Conversation summary (1-10000 chars)
- sessionKey (string): Key to identify conversation (1-200 chars)
- title (string): Conversation title (max 500 chars)
- recentTurns (array): Turns with role and content (max 50)
- embedTurns (boolean): Generate turn embeddings, default false

### query_conversations

Unified conversation query tool. Dispatches based on mode.

- mode (enum, REQUIRED): 'search' | 'get' | 'list'
- query (string): Search query, required when mode is 'search' (max 1000 chars)
- searchMode (enum): summary | turns | both, default summary (search mode)
- includeTranscript (boolean): Include turns in results, default false (search mode)
- maxTurnsPerConversation (number): Max turns per conversation (search mode), default 10
- sessionId (string): Session ID, for get mode
- sessionKey (string): Session key, for get mode
- maxTurns (number): Max turns to include (get mode), default 50
- limit (number): Max results (search: 1-20, default 5; list: 1-50, default 20)
- offset (number): Pagination offset (list mode)

### delete_conversation

Delete a conversation session.

- sessionId (string): Session ID
- sessionKey (string): Session key
- confirm (boolean, REQUIRED): Must be true

## Stats

### get_stats

Get statistics across the knowledge base. Dispatches based on scope.

- scope (enum): 'all' | 'knowledge' | 'memory' | 'conversations' | 'insights', default 'all'

### health_check

Check the health of all subsystems. Returns pass/fail per component with overall status.

- verbose (boolean): Include latency and row counts, default false

## Insight Tools (Proactive Discovery)

Enable with ENABLE_INSIGHTS=true (default).

### get_insights

Get discovered cross-source connections and patterns.

- status (enum): new | seen | dismissed, default new
- insightType (enum): cross_source | theme_cluster | entity_bridge | temporal_pattern | outlier
- query (string): Semantic search for relevant insights
- limit (number): Max insights 1-50, default 5

### discover_connections

Trigger insight scan across knowledge base.

- fullScan (boolean): Scan all content, default false
- maxChunks (number): Max chunks to analyze 10-1000, default 200

### dismiss_insight

Dismiss an insight.

- insightId (string, REQUIRED): Insight ID to dismiss

## Unified Tools

### ask

Unified RAG search across all knowledge sources (documents, memory, conversations, insights).

- query (string, REQUIRED): Natural language search query
- scope (enum): auto | documents | memory | conversations | insights, default auto
- limit (number): Max results 1-50, default 10

### daily_briefing

Generate a daily briefing with recent additions, new insights, and resurfaced "on this day" knowledge. No required parameters.

### save_url

Fetch a URL, convert HTML to markdown, extract metadata (title, author, date), and save as a document.

- url (string, REQUIRED): URL to fetch and save
- tags (string[]): Tags for organization
- extract_memories (boolean): Extract memories from content

### timeline

Browse knowledge chronologically within a date range, optionally filtered by topic.

- start_date (string, REQUIRED): Start of date range (ISO date)
- end_date (string, REQUIRED): End of date range (ISO date)
- topic (string): Filter by topic
- limit (number): Max results to return

## Postgres Analysis Tools

Enabled when DATABASE_URL is configured. Connects directly to Postgres (independent of the main database client).

### pg_analyze

Run comprehensive Postgres health analysis. Returns table stats, index health, vacuum status, connection info, bloat estimates, Textrawl-specific checks, and actionable recommendations.

- save (boolean): Save report to history for future comparison, default false

### pg_recommendations

Get actionable optimization recommendations for Postgres.

- severity (enum): 'all' | 'critical' | 'warning' | 'info', default 'all'

### pg_report_history

View past Postgres analysis reports and compare trends.

- count (number): Number of past reports to load 1-50, default 5
- diff (boolean): Compare latest two reports and show changes, default true

## Documentation

- /AGENTS.md - Agent conventions and tool selection guide
- /CLAUDE.md - Development conventions for Claude Code
- /docs/cli/ - CLI conversion tools (MBOX, HTML, PDF)
- /docs/guides/security-hardening.mdx - Row Level Security setup
- /.well-known/mcp.json - MCP capability advertisement

## Quick Facts

- MCP tool count varies by surface (set via MCP_TOOLSET): `normal` (default) advertises the compact workflow tools (`ask`, `search`, `get_document`, `capture`, `remember`, `daily_briefing`, `timeline`); `full` and `legacy` advertise the full granular set. See README.md and src/tools/ for the canonical list.
- Embedding: OpenAI (1536 dim), Google AI (3072 dim), or Ollama (1024 dim / 768 dim)
- Database: Neon PostgreSQL with pgvector
- Chunking: 512 tokens, 50 token overlap, paragraph-aware
- Node.js >= 22.0.0 required

## Source Types

- note: Created via add_note tool
- file: Uploaded via CLI or Web UI (PDF, DOCX, MBOX, EML, HTML, TXT, MD)
- url: Web content saved via save_url tool

## Optional

- llms-full.txt: /llms-full.txt (complete documentation)