CLI Tools Overview

textrawl includes CLI tools for converting various file formats to Markdown, then uploading them to your knowledge base.

Quick Start

# Convert files
pnpm convert -- mbox ~/Mail/archive.mbox
pnpm convert -- html ./saved-pages/
 
# Upload to knowledge base
pnpm upload -- ./converted/emails

Available Commands

mbox

Convert MBOX email archives (Gmail exports, Thunderbird backups).

MBOX conversion →

html

Convert HTML files and web pages to searchable markdown.

HTML conversion →

scan

Analyze converted markdown files for upload readiness and flag oversized files.

Scan →

split

Break large markdown files into uploadable parts at heading boundaries.

Split →

upload

Upload converted files to your knowledge base with automatic embedding.

Batch upload →

Unified Converter

The main entry point for all conversions:

pnpm convert -- <command> <path> [options]

Command	Description
`mbox <file>`	Convert MBOX email archive
`eml <path>`	Convert EML file(s) or directory
`html <path>`	Convert HTML file(s) or directory
`takeout <path>`	Convert Google Takeout archive
`spotify <path>`	Convert Spotify data export
`reddit <path>`	Convert Reddit data export
`facebook <path>`	Convert Facebook data export
`instagram <path>`	Convert Instagram data export
`auto <path>`	Auto-detect format and convert

Common Options

All conversion commands support these options:

Option	Default	Description
`-o, --output <dir>`	`./converted`	Output directory
`-v, --verbose`	`false`	Enable verbose logging
`--dry-run`	`false`	Preview without writing files
`-t, --tags <tags...>`	`[]`	Additional tags to add

Workflow

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Source    │     │  Converted  │     │  Knowledge  │
│   Files     │ ──► │  Markdown   │ ──► │    Base     │
│  (MBOX,HTML)│     │   + YAML    │     │ (Postgres)  │
└─────────────┘     └─────────────┘     └─────────────┘
      │                    │                    │
      │                    │                    │
 pnpm convert     Frontmatter         pnpm upload
                     with metadata

Output Format

Converted files are saved as Markdown with YAML frontmatter:

---
title: "Email Subject or Page Title"
source_type: email
source_hash: "abc123..."
tags:
  - imported
  - email
created_at: "2024-01-15T10:30:00Z"
converted_at: "2024-03-20T14:22:00Z"
metadata:
  from: "sender@example.com"
  to: "recipient@example.com"
---
 
# Email Subject
 
Email or document content here...

Frontmatter Fields

Field	Description
`title`	Document title
`source_type`	`email`, `web`, `youtube`, `calendar`, `contact`
`source_hash`	Hash for deduplication
`tags`	Array of tags
`created_at`	Original creation date
`converted_at`	Conversion timestamp
`metadata`	Format-specific metadata

The source_hash prevents duplicate uploads when re-running the upload command.

Web UI

A drag-and-drop web interface for file conversion:

pnpm ui
# Opens at http://localhost:3001

Features:

Drag-and-drop file upload
Real-time conversion progress
Auto-upload option after conversion
Supports MBOX, EML, ZIP (Takeout), HTML, PDF, DOCX, TXT, MD

Next Steps

MBOX Conversion - Import email archives
HTML Conversion - Convert web pages
Scan - Check files for upload readiness
Split - Break large files into parts
Batch Upload - Upload to knowledge base

CLI Tools Overview

On this page