textrawl
byJeff Green

Scan

Analyze converted markdown files for upload readiness

scan inspects converted markdown files before upload. It reports file sizes, estimated chunk counts, and heading structure, and flags files that exceed the upload limits so you can split them first.

It is read-only — it never modifies or uploads anything.

Usage

pnpm scan -- <directory-or-file> [options]
# Scan a directory, showing only problematic files
pnpm scan -- ./converted/
 
# Show every file, not just oversized ones
pnpm scan -- ./converted/ --all
 
# Detailed analysis of a single file (heading tree + split suggestions)
pnpm scan -- ./converted/large-file.md

Options

OptionDefaultDescription
-r, --recursivetrueScan subdirectories
--pattern <glob>**/*.mdGlob pattern for files to scan
--max-file-size <mb>20File size threshold (MB) before a file is flagged
--max-chunks <n>500Estimated chunk-count threshold before a file is flagged
--allfalseShow all files, not just ones that exceed limits
--format <type>tableOutput format: table or json

Example Output

=== Upload Readiness Scan ===
File size limit: 20 MB | Chunk limit: 500

  STATUS              SIZE   CHUNKS  FILE
  ✓ OK             1.2 MB     ~180  emails/2024-01.md
  ✗ CHUNKS         4.8 MB     ~720  archive/all-threads.md

=== Summary ===
Total: 2 | Uploadable: 1 | Need splitting: 1
Estimated total chunks: ~900

=== Suggested Splits ===
archive/all-threads.md: 6 h2 headings -> split at h2 would create ~7 parts
  Run: pnpm split -- archive/all-threads.md

A file is flagged when it exceeds either --max-file-size (✗ SIZE) or --max-chunks (✗ CHUNKS). For a single file, the scan additionally prints the full heading tree and suggested split points.

Use Case

Run scan after pnpm convert and before pnpm upload to find files that would fail the upload chunk limit. For any flagged file, run split to break it into uploadable parts.

Next Steps

  • Split - Break large files into uploadable parts
  • Batch Upload - Upload converted files

On this page