textrawl
byJeff Green
Getting Started

Installation

Detailed installation guide for textrawl

System Requirements

RequirementMinimumRecommended
Node.js22.0.022.x LTS
pnpm9.0.09.x
RAM512 MB1 GB+
Disk100 MB1 GB+

Installation Methods

# Clone the repository
git clone https://github.com/jeffgreendesign/textrawl.git
cd textrawl

# Install dependencies
pnpm install

# Run interactive setup
pnpm run setup

# Start development server
pnpm run dev

Docker

# Clone the repository
git clone https://github.com/jeffgreendesign/textrawl.git
cd textrawl

# Create .env file
cp .env.example .env
# Edit .env with your credentials

# Build and run
docker compose up -d

Docker (Local Stack)

For a fully self-hosted setup with local PostgreSQL and Ollama:

# Start PostgreSQL + pgvector
docker compose -f docker-compose.local.yml up -d

# Optionally start Ollama for local embeddings
docker compose -f docker-compose.local.yml --profile ollama up -d

# Optionally start pgAdmin for database management
docker compose -f docker-compose.local.yml --profile admin up -d

Database Setup

textrawl requires PostgreSQL with the pgvector extension. Use a managed Postgres provider with pgvector (Neon recommended — pgvector is included on all Neon tiers, including Free), or self-host.

pgvector 0.8 or newer is recommended. Filtered hybrid searches (by source_type, content_type, or tags) rely on pgvector's iterative index scans to keep results complete under selective filters. textrawl enables them automatically when available and degrades gracefully on older versions; Neon ships pgvector ≥ 0.8.

For compute tier recommendations and storage estimates, see the Database Requirements guide.

  1. Create a project at neon.tech. From the project dashboard, copy the pooled connection string.
  2. Set DATABASE_URL so the following psql commands (and the textrawl server) can pick it up — either export it in your shell or persist it in .env:
# Option A: export for the current shell session
export DATABASE_URL="postgresql://user:pass@ep-xxx-pooler.neon.tech/textrawl?sslmode=require"

# Option B: persist in .env (loaded by the server and by pnpm setup)
echo 'DATABASE_URL="postgresql://user:pass@ep-xxx-pooler.neon.tech/textrawl?sslmode=require"' >> .env
# then export it for psql in this shell:
set -a; source .env; set +a
  1. Run the schema setup that matches your chosen embedding provider:
# For OpenAI embeddings (1536 dimensions)
psql "$DATABASE_URL" -f scripts/setup-db.sql

# For Google AI embeddings (1536 dimensions)
psql "$DATABASE_URL" -f scripts/setup-db-google.sql

# For Ollama embeddings (1024 dimensions, nomic-embed-text)
psql "$DATABASE_URL" -f scripts/setup-db-ollama.sql

# For Ollama v2 embeddings (768 dimensions, nomic-embed-text-v2-moe)
psql "$DATABASE_URL" -f scripts/setup-db-ollama-v2.sql

⚠️ You MUST match the schema to your OLLAMA_MODEL:

  • nomic-embed-text → setup-db-ollama.sql (1024d)
  • nomic-embed-text-v2-moe → setup-db-ollama-v2.sql (768d)

Using the wrong schema causes dimension mismatch errors.

  1. Run security hardening:
psql "$DATABASE_URL" -f scripts/security-rls.sql

Option 2: Self-Hosted PostgreSQL

# Using docker-compose.local.yml
docker compose -f docker-compose.local.yml up -d

# Set DATABASE_URL for local Postgres
export DATABASE_URL=postgresql://textrawl:textrawl@localhost:5432/textrawl

# Enable pgvector (if not already enabled by the image)
psql "$DATABASE_URL" -c "CREATE EXTENSION IF NOT EXISTS vector;"

# Run setup SQL files
psql "$DATABASE_URL" -f scripts/setup-db.sql
psql "$DATABASE_URL" -f scripts/security-rls.sql

Option 3: Other Providers

Any Postgres 15+ with the pgvector extension works. Set DATABASE_URL to a pooled connection string and run the same psql … -f scripts/setup-db*.sql commands shown above.

  • Supabase — copy the Connection pooling URL from Settings → Database into DATABASE_URL. Run the schema via psql or paste it into the SQL Editor.
  • AWS RDS / Azure Database for PostgreSQL / GCP Cloud SQL — enable the vector extension on the instance, then copy the connection string.

Environment Configuration

Copy the example and configure:

cp .env.example .env

Required Variables

# Postgres connection (Neon pooled URL recommended)
DATABASE_URL=postgresql://user:pass@ep-xxx.neon.tech/textrawl?sslmode=require

# OpenAI for embeddings
OPENAI_API_KEY=sk-...

Optional Variables

# Server configuration
PORT=3000
LOG_LEVEL=info

# Authentication (required for production)
API_BEARER_TOKEN=your-secure-token-min-32-chars

# CORS (comma-separated origins)
ALLOWED_ORIGINS=http://localhost:3000

# Alternative: Use Google AI instead of OpenAI
# EMBEDDING_PROVIDER=google
# GOOGLE_AI_API_KEY=your-google-ai-api-key

# Alternative: Use Ollama instead of OpenAI
# EMBEDDING_PROVIDER=ollama
# OLLAMA_BASE_URL=http://localhost:11434
# OLLAMA_MODEL=nomic-embed-text

Verify Installation

# Check health endpoint
curl http://localhost:3000/health

# Expected response:
# {"status":"ok","timestamp":"2025-01-01T00:00:00.000Z"}

# Check readiness (includes database)
curl http://localhost:3000/health/ready

# Test MCP Inspector
pnpm run inspector

Next Steps

Quick Start

Get textrawl running in 5 minutes

Configuration

Environment variables and configuration options

On this page

System RequirementsInstallation Methodspnpm (Recommended)DockerDocker (Local Stack)Database SetupOption 1: Neon (Recommended)Option 2: Self-Hosted PostgreSQLOption 3: Other ProvidersEnvironment ConfigurationRequired VariablesOptional VariablesVerify InstallationNext Steps