Configuration

textrawl is configured via environment variables in a .env file.

Environment Variables

Database Connection

Variable	Required	Description
`DATABASE_URL`	Yes	Postgres connection string (Neon pooled URL recommended; Supabase / RDS / self-hosted also work)

DATABASE_URL=postgresql://user:pass@ep-xxx-pooler.neon.tech/textrawl?sslmode=require

Warning: Never commit your connection string to version control — it contains credentials. Use .env files or secret managers.

For Supabase users: copy the Connection pooling URL from Settings → Database, not the direct connection.

Embedding Provider

Choose between OpenAI (cloud), Google AI (cloud), or Ollama (local):

Variable	Required	Description
`EMBEDDING_PROVIDER`	No	`openai` (default), `ollama`, or `google`
`OPENAI_API_KEY`	If OpenAI	Your OpenAI API key
`GOOGLE_AI_API_KEY`	If Google	Your Google AI API key
`GOOGLE_EMBEDDING_MODEL`	If Google	Model name (default: `gemini-embedding-2`)
`OLLAMA_BASE_URL`	If Ollama	Ollama server URL
`OLLAMA_MODEL`	If Ollama	Model name (default: `nomic-embed-text`)

OpenAI Configuration:

EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-proj-...

Model: text-embedding-3-small
Dimensions: 1536
Schema: scripts/setup-db.sql

Google AI Configuration:

EMBEDDING_PROVIDER=google
GOOGLE_AI_API_KEY=your-google-ai-api-key
GOOGLE_EMBEDDING_MODEL=gemini-embedding-2

Model: gemini-embedding-2
Dimensions: 1536 (requested via outputDimensionality; the model is natively 3072d)
Schema: scripts/setup-db-google.sql

Ollama Configuration:

EMBEDDING_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=nomic-embed-text

Dimensions: 1024 (nomic-embed-text) or 768 (nomic-embed-text-v2-moe)
Schema: scripts/setup-db-ollama.sql (1024d) or scripts/setup-db-ollama-v2.sql (768d)

Important: Each provider uses different embedding dimensions. You cannot mix providers without re-embedding all documents. Use the matching SQL schema for your chosen provider.

Server Configuration

Variable	Required	Default	Description
`PORT`	No	`3000`	Server port
`LOG_LEVEL`	No	`info`	`debug`, `info`, `warn`, `error`
`ALLOWED_ORIGINS`	No	`*`	CORS allowed origins (comma-separated)

PORT=3000
LOG_LEVEL=info
ALLOWED_ORIGINS=http://localhost:3000,https://myapp.com

Authentication

Variable	Required	Description
`API_BEARER_TOKEN`	Production	Auth token (min 32 characters)

API_BEARER_TOKEN=your-very-secure-token-with-at-least-32-characters

When set, all API endpoints require an Authorization: Bearer <your-token> header.

Unprotected endpoints (for health checks):

/health
/health/live
/health/ready

Web UI

Variable	Required	Default	Description
`UI_PORT`	No	`3001`	Web UI port for file conversion

UI_PORT=3001

Feature Flags

Variable	Required	Default	Description
`ENABLE_MEMORY`	No	`true`	Enable/disable memory tools
`ENABLE_CONVERSATIONS`	No	`true`	Enable/disable conversation memory tools
`ENABLE_INSIGHTS`	No	`true`	Enable/disable proactive insight tools
`ENABLE_MEMORY_EXTRACTION`	No	`false`	Enable LLM-based memory extraction
`COMPACT_RESPONSES`	No	`true`	Token-efficient response format

ENABLE_MEMORY=true
ENABLE_CONVERSATIONS=true
ENABLE_INSIGHTS=true
ENABLE_MEMORY_EXTRACTION=false
COMPACT_RESPONSES=true

Memory Extraction

Required only when ENABLE_MEMORY_EXTRACTION=true:

Variable	Required	Default	Description
`ANTHROPIC_API_KEY`	If extraction	-	Anthropic API key for Claude
`EXTRACTION_MODEL`	No	`claude-haiku-4-5`	Model for entity extraction

ENABLE_MEMORY_EXTRACTION=true
ANTHROPIC_API_KEY=sk-ant-...
EXTRACTION_MODEL=claude-haiku-4-5

Insight Configuration

Controls the LLM model used for proactive insight synthesis (discover_connections and the background insight scheduler). Independent of EXTRACTION_MODEL.

Variable	Required	Default	Description
`INSIGHT_MODEL`	No	`claude-sonnet-5`	Model for insight synthesis
`INSIGHT_BATCH_THRESHOLD`	No	`50`	Minimum new chunks before triggering an insight scan
`INSIGHT_DEBOUNCE_SECONDS`	No	`300`	Cooldown between automatic insight scans (seconds)

INSIGHT_MODEL=claude-sonnet-5
INSIGHT_BATCH_THRESHOLD=50
INSIGHT_DEBOUNCE_SECONDS=300

Chunking

Variable	Required	Default	Description
`CHUNKING_MODE`	No	`fixed`	`fixed` or `semantic` (embedding-based topic splitting)
`SEMANTIC_SIMILARITY_THRESHOLD`	No	`0.5`	Threshold for semantic chunking (0-1)

CHUNKING_MODE=semantic
SEMANTIC_SIMILARITY_THRESHOLD=0.5

When COMPACT_RESPONSES=true (default), memory tools return token-efficient responses that reduce LLM context usage by 40-60%. This uses short keys like n, t, c instead of name, type, content.

Set COMPACT_RESPONSES=false for human-readable verbose responses during development or debugging.

See Response Format for the complete key mapping.

Postgres Analysis

DATABASE_URL (defined in the Database Connection section above) enables three Postgres analysis tools: pg_analyze, pg_recommendations, and pg_report_history.

Variable	Required	Default	Description
`PG_REPORT_DIR`	No	`./reports/pg-analysis`	Directory for analysis report history

PG_REPORT_DIR=./reports/pg-analysis

Rate Limiting

Variable	Required	Default	Description
`REDIS_URL`	No	-	Redis URL for shared rate limiting across instances (e.g. `redis://localhost:6379`)

REDIS_URL=redis://localhost:6379

When REDIS_URL is set, rate limit counters are shared across all server instances via Redis. Without it, each instance tracks limits independently in memory (fine for single-instance deployments).

Rate Limits

Built-in rate limiting:

Endpoint	Limit
API (`/mcp`, `/api/*`)	100 requests/minute
Upload (`/api/upload`)	10 requests/minute
Health (`/health/*`)	300 requests/minute

Example Configurations

Development (Local)

# .env
DATABASE_URL=postgresql://user:pass@ep-xxx-pooler.neon.tech/textrawl?sslmode=require
OPENAI_API_KEY=sk-your-key
PORT=3000
LOG_LEVEL=debug

Production (Cloud Run)

# Set via Secret Manager
DATABASE_URL=postgresql://user:pass@ep-xxx-pooler.neon.tech/textrawl?sslmode=require
OPENAI_API_KEY=sk-your-key
API_BEARER_TOKEN=your-secure-production-token
PORT=8080
LOG_LEVEL=info
ALLOWED_ORIGINS=https://your-frontend.com

Self-Hosted (Ollama)

# .env
DATABASE_URL=postgresql://textrawl:textrawl@localhost:5432/textrawl
EMBEDDING_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=nomic-embed-text
PORT=3000
LOG_LEVEL=info

Generating Secure Tokens

The setup script generates a secure token automatically:

pnpm setup

Or generate manually:

# macOS/Linux
openssl rand -base64 32

# Node.js
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"

Next Steps

Quick Start - Get running in 5 minutes
Security Hardening - Production security
Cloud Run Deployment - Deploy to GCP

Environment Variables

Database Connection

Embedding Provider

Server Configuration

Authentication

Web UI

Feature Flags

Memory Extraction

Insight Configuration

Chunking

Compact Responses

Postgres Analysis

Rate Limiting

Rate Limits

Example Configurations

Development (Local)

Production (Cloud Run)

Self-Hosted (Ollama)

Generating Secure Tokens

Next Steps

On this page

Configuration

Environment VariablesCopy Anchor Link

Database ConnectionCopy Anchor Link

Embedding ProviderCopy Anchor Link

Server ConfigurationCopy Anchor Link

AuthenticationCopy Anchor Link

Web UICopy Anchor Link

Feature FlagsCopy Anchor Link

Memory ExtractionCopy Anchor Link

Insight ConfigurationCopy Anchor Link

ChunkingCopy Anchor Link

Compact ResponsesCopy Anchor Link

Postgres AnalysisCopy Anchor Link

Rate LimitingCopy Anchor Link

Rate LimitsCopy Anchor Link

Example ConfigurationsCopy Anchor Link

Development (Local)Copy Anchor Link

Production (Cloud Run)Copy Anchor Link

Self-Hosted (Ollama)Copy Anchor Link

Generating Secure TokensCopy Anchor Link

Next StepsCopy Anchor Link

On this page

Environment Variables

Database Connection

Embedding Provider

Server Configuration

Authentication

Web UI

Feature Flags

Memory Extraction

Insight Configuration

Chunking

Compact Responses

Postgres Analysis

Rate Limiting

Rate Limits

Example Configurations

Development (Local)

Production (Cloud Run)

Self-Hosted (Ollama)

Generating Secure Tokens

Next Steps