TWITTER_ARTICLE

Nielsen777Brian says an openclawd workspace had grown to about 180,000 tokens…

2026-02-10 · 19:13 UTC ·Nielsen777Brian ·3 min read

Brief

Claw-compactor is a token-compression utility built for openclawd and Claude Code-style agent workspaces that accumulate large memory files such as session logs, CLAUDE.md, and observation notes. Nielsen777Brian frames the tool as a deterministic alternative to paying for larger context windows, claiming that a mid-size codebase had ballooned to 180,000 tokens, much of it redundant. The system applies five layers of compression, mixing reversible transformations like deduplication, formatting cleanup, dictionary encoding, and shorthand for repeated patterns with a partially lossy abbreviation layer that preserves facts while removing filler. Its headline capability is compressing JSONL session transcripts into structured summaries, reportedly shrinking 50,000-token logs to about 1,500 tokens. The post reports 50-70% savings on first-time cleanup, 10-20% on weekly maintenance, and 3-12% on already-optimized workspaces, and says the tool installs in about 10 minutes on Python 3.9+ with no required dependencies.

Why it matters

Nielsen777Brian says an openclawd workspace had grown to about 180,000 tokens, with at least half attributed to redundant formatting, duplicate context, and verbose session transcripts, prompting the creation of the claw-compactor tool.

Key details

  • claw-compactor uses five stacked compression layers: a rule engine for deduplication and markdown cleanup (4-8% savings), dictionary encoding with reversible `$XX` tokens (4-5%), observation compression for JSONL session logs (~97%), RLE-style shorthand for repeated patterns like file paths and IPs (1-2%), and a partially lossy Compressed Context Protocol (20-60%).
  • The biggest gain comes from Layer 3: the author claims a 50,000-token JSONL session log can be reduced to roughly 1,500 tokens by converting transcripts into structured summaries of facts and decisions.
  • Reported outcomes vary by workspace state: first-time verbose workspaces save 50-70%, weekly maintenance runs save 10-20%, and already-optimized workspaces save 3-12%; setup is described as a 10-minute process requiring Python 3.9+ with no dependencies, with optional tiktoken support.
  • The author argues that combining a 50% context reduction from claw-compactor with prompt caching using `cacheRetention: "long"` and a 90% cached-token discount yields an effective cost of about 5% of the original token spend, or roughly 95% total cost reduction.
Cleaned source text

title: @Nielsen777Brian: my openclawd workspace was 70% waste. so I built claw-compactor, 70% token savin...

author: Nielsen777Brian

content_type: twitter_article

published: 2026-02-10T19:13:21+00:00

source_url: https://x.com/Nielsen777Brian/status/2021301480079389144

word_count: 579

my openclawd workspace was 70% waste. so I built claw-compactor, 70% token savings, 10 minutes to se

my openclawd workspace was 70% waste. so I built claw-compactor, 70% token savings, 10 minutes to set up, zero LLM calls.

you're burning tokens you don't need to burn. I was too.

redundant formatting. duplicate context. bloated session logs. all eating tokens, all costing money, all completely unnecessary.

I didn't need a smarter model. I needed a compressor. so I built one.

here's exactly how it works and why I designed each of the 5 compression layers.

why I built it

I was running openclawd on a mid-size codebase. my workspace memory files kept growing — session logs, CLAUDE.md, observation notes. one day I checked: 180K tokens of context, and at least half of it was redundant formatting, duplicate content, and verbose session transcripts.

I didn't need a smarter model. I needed a compressor.

so I built one. 5 compression layers, each stacking on the last:

Layer 1 — Rule Engine (4-8% savings)

Deduplicates content, cleans up markdown formatting, merges redundant sections. the kind of stuff you'd do manually if you had the patience.

Layer 2 — Dictionary Encoding (4-5% savings)

Auto-learns a codebook from your workspace. repeated phrases get replaced with short `$XX` tokens. fully reversible.

Layer 3 — Observation Compression (~97% savings on session files)

this is the one I'm most proud of. your JSONL session transcripts — which are massive — get compressed into structured summaries. a 50,000 token session log becomes ~1,500 tokens of facts and decisions.

Layer 4 — RLE Patterns (1-2% savings)

file paths, IP addresses, repeated enums get shorthand notation. small but it compounds.

Layer 5 — Compressed Context Protocol (20-60% savings)

abbreviation levels that trade verbosity for density. partial lossy — facts stay, filler goes.

10-minute setup

step 1: clone it

step 2: benchmark (non-destructive — just shows what you'd save)

step 3: look at the numbers. if you like them, run full compression:

that's it. Python 3.9+, no dependencies required. optional tiktoken for precise token counts.

what the numbers actually look like

First-time verbose workspace 50-70% savings . unoptimized CLAUDE.md, raw logs. this is where most people start.

Session transcripts (JSONL) ~97% savings . this is not a typo. a 50K token log becomes ~1.5K tokens.

Regular maintenance (weekly) 10-20% savings . diminishing returns, still worth running.

Already-optimized workspace 3-12% savings . you've already done the easy wins.

the session transcript number is the headline. if you're running openclawd or claude code agents that accumulate session logs, Layer 3 alone justifies the install.

the stacking trick I keep telling people

claw-compactor + prompt caching = ~95% effective cost reduction.

here's the math:

claw-compactor compresses your context by 50%

prompt caching (`cacheRetention: "long"`) gives 90% off cached tokens

50% compression x 90% cache discount = you're paying 5% of original cost

that's not theoretical. that's what I see on my own workspaces when I combine deterministic compression with model-level caching.

why I'm sharing this now

the openclawd ecosystem is exploding. people are building iOS apps in 5 days with Claude Code, running full agent teams, creating SaaS replacements.

but nobody's optimizing what goes into the context window. they're building bigger and bigger workspaces, accumulating more session history, and wondering why their token costs keep climbing.

I built claw-compactor to solve my own problem. turns out a lot of people have the same one.

bookmark this

you'll need it when your agent workspace hits 200K tokens and you're wondering where the money went.

Posted: 2026-02-10T19:13:21.000Z

Engagement: 215 likes, 46 retweets, 25 replies