fix(cli): quote install hints so pip install soup-cli[extra] works on cmd.exe (v0.71.37)

Every printed and documented `pip install 'soup-cli[extra]'` was bash / zsh /
PowerShell syntax and failed on Windows cmd.exe:

    ERROR: Invalid requirement: "'soup-cli[train]'": Expected package name at
    the start of dependency specifier

cmd.exe has no single-quote quoting, so it passes the quotes to pip verbatim
and pip rejects the requirement. Nothing in Soup can fix that once the command
is typed -- pip and the shell own it, and Soup is not installed yet when the
README line runs -- so the fix is the spelling we print.

Migrated 147 sites across 67 files to `pip install "soup-cli[extra]"`:
  - 64 in src/  (Rich console hints + plain ImportError text)
  - 57 in README.md + docs/
  - 22 in src/soup_cli/templates/*.yaml + examples/configs/*.yaml
  -  3 in examples/README.md

Double quotes are the only spelling valid in every shell (cmd, PowerShell,
bash, zsh), which is why the repo already used `pip install -e ".[dev]"`.
Measured on Windows: single quotes fail ONLY on cmd; double quotes pass
everywhere; bare passes on Windows but zsh globs `[extra]` and fails.

Method note (the PR MakazhanAlpamys#247 class): the hints sit INSIDE double-quoted Python
string literals, so a blind ' -> " sed produces SyntaxError. A tokenize-based
rewriter escaped `\"` in DQUOTE tokens and left bare `"` in TRIPLE / COMMENT
tokens; every touched .py was compile-checked. The full suite (not ruff, not
compile-check) caught two rewriter blind spots: the real YAML templates under
src/soup_cli/templates/ (byte-identical drift test) and examples/README.md.

A regression test (tests/test_v07137.py) scans the package and every docs code
block for the single-quoted form; prose may still name it so a reader from an
older tutorial recognises the error.

Also bundles MakazhanAlpamys#315 (@Sanjays2402): eval-gate benchmark tasks now run via
ForgettingDetector instead of a helper that never existed. Closes MakazhanAlpamys#310.

Test count: 16283 -> 16288 (+4 in tests/test_v07137.py).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: MakazhanAlpamys

CHANGELOG.md

@@ -12,8 +12,35 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.37] - 2026-07-17
+
+**Every `pip install soup-cli[extra]` command now works on Windows `cmd.exe`**, and
+eval-gate benchmark tasks run instead of always failing.
+
 ### Fixed
 
+- **Install hints are now quoted so they work in every shell.** Soup printed
+  `pip install 'soup-cli[ui]'` — bash / zsh / PowerShell syntax. `cmd.exe` has no
+  single-quote quoting, so it hands the quotes to pip verbatim and pip refuses:
+
+  ```
+  ERROR: Invalid requirement: "'soup-cli[train]'": Expected package name at the start of dependency specifier
+  ```
+
+  Every hint, README command, and docs example now uses `pip install
+  "soup-cli[extra]"`, which works in cmd, PowerShell, bash, and zsh alike — the
+  same spelling the repo already used for `pip install -e ".[dev]"`. Measured on
+  Windows: single quotes fail only on `cmd.exe`; double quotes pass everywhere;
+  dropping the quotes passes on Windows but breaks zsh, which globs the bracket.
+
+  Nothing in Soup can rescue the command after it is typed — pip and the shell
+  own it, and Soup is not installed yet when the README command runs — so the
+  fix is the spelling we print. A regression test now scans the package and every
+  docs code block for the single-quoted form.
+
+  If you followed an older tutorial and hit `Invalid requirement`, swap the `'`
+  for `"`; nothing is wrong with the package.
+
 - **Eval-gate `type: benchmark` tasks now actually run.** `eval/gate.py` probed
   for a `forgetting.run_mini_benchmark` helper that never existed, so every
   `type: benchmark` task in a gate suite failed 100% of the time — while

CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 21 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0, +4 compliance v0.71.35)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (314 files, 16283 tests)
+tests/                - Test suite (315 files, 16288 tests)
 examples/             - Real-world config examples and datasets
 ```
 

README.md

@@ -32,7 +32,7 @@
 Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.
 
 ```bash
-pip install 'soup-cli[train]'   # add [train] to fine-tune; bare `soup-cli` is the light CLI
+pip install "soup-cli[train]"   # add [train] to fine-tune; bare `soup-cli` is the light CLI
 soup init --template chat
 soup train
 ```
@@ -68,7 +68,7 @@ infrastructure instead of improving models. Soup fixes that.
   separates them. The default is deliberately conservative; see the CHANGELOG.
 - **Two blocking bugs fixed:** the hardware-fit gate **refused to train any model you merged
   yourself** (a local checkpoint's name has no size marker, so it guessed 7B and predicted
-  16 GB), and every `pip install 'soup-cli[extra]'` hint printed **without the extra** —
+  16 GB), and every `pip install "soup-cli[extra]"` hint printed **without the extra** —
   17 sites where the suggested command succeeds and leaves the feature still broken.
 
 ```bash
@@ -104,26 +104,37 @@ Full history: [CHANGELOG.md](CHANGELOG.md) · [GitHub Releases](https://gi
 ### 1. Install
 
 ```bash
-pip install soup-cli            # light: CLI + config + data tools (no PyTorch)
-pip install 'soup-cli[train]'   # add the training stack (torch, transformers, peft, trl, …)
-pip install git+https://github.com/MakazhanAlpamys/Soup.git   # latest dev
+# Light core: CLI + config + data tools, no PyTorch
+pip install soup-cli
+
+# Add the training stack (torch, transformers, peft, trl, datasets, …)
+pip install "soup-cli[train]"
+
+# Everything (train + serve + ui + data) in one shot
+pip install "soup-cli[all]"
+
+# Or from GitHub (latest dev)
+pip install git+https://github.com/MakazhanAlpamys/Soup.git
 ```
 
-> **On Windows `cmd.exe`, swap the single quotes for double quotes:**
-> `pip install "soup-cli[train]"`
+The full extras table (`fast`, `mlx`, `serve`, `eval`, `ui`, `vision`, `audio`, …) lives in
+[`docs/models.md`](docs/models.md#optional-extras).
+
+> **Use double quotes around the extra.** They are the only spelling that works in
+> every shell — `cmd.exe`, PowerShell, bash, and zsh.
 >
-> The commands above are written for bash / zsh / PowerShell, where `'…'` quotes a
-> string. `cmd.exe` has no single-quote quoting — it passes the characters straight
-> to pip, which then rejects them:
+> Older tutorials and videos (including some of ours) show the single-quoted
+> `pip install 'soup-cli[train]'`. That is bash / zsh / PowerShell syntax, and it
+> fails on Windows `cmd.exe`, which has no single-quote quoting and hands the
+> quotes straight to pip:
 >
 > ```
 > ERROR: Invalid requirement: "'soup-cli[train]'": Expected package name at the start of dependency specifier
 > ```
 >
-> That is pip parsing a literal `'`, not a problem with the package. Double quotes
-> work in **every** shell — cmd, PowerShell, bash, and zsh — so `pip install
-> "soup-cli[train]"` is always safe. (Unquoted `pip install soup-cli[train]` also
-> works in cmd and PowerShell, but zsh reads `[train]` as a glob and fails.)
+> If you hit that, swap the `'` for `"` — pip is rejecting a literal quote
+> character, nothing is wrong with the package. (Dropping the quotes entirely
+> works on Windows too, but zsh then reads `[train]` as a glob and fails.)
 
 `soup init`, `soup data …`, and the other data/inspection commands work on the light install.
 Fine-tuning (`soup train`) needs the `[train]` extra.
@@ -159,7 +170,7 @@ A complete `soup.yaml`:
 ```yaml
 base: meta-llama/Llama-3.1-8B-Instruct
 task: sft
-# backend: unsloth  # 2-5x faster, pip install 'soup-cli[fast]'
+# backend: unsloth  # 2-5x faster, pip install "soup-cli[fast]"
 
 data:
   train: ./data/train.jsonl

docs/backends-and-ops.md

@@ -79,7 +79,7 @@ Fine-tune on M1-M4 Macs via Apple's [MLX](https://github.com/ml-explore/mlx) fra
 
 ```bash
 # Install MLX support
-pip install 'soup-cli[mlx]'
+pip install "soup-cli[mlx]"
 ```
 
 ```yaml
@@ -108,7 +108,7 @@ Use the [Unsloth](https://github.com/unslothai/unsloth) backend for significantl
 
 ```bash
 # Install unsloth support
-pip install 'soup-cli[fast]'
+pip install "soup-cli[fast]"
 ```
 
 Then add one line to your config:
@@ -143,7 +143,7 @@ app from your `soup.yaml` for serverless, per-second-billed GPU training. The co
 base64-embedded as **data** — no code interpolation, no secrets in the generated stub.
 
 ```bash
-pip install 'soup-cli[modal]'   # only needed for live submit
+pip install "soup-cli[modal]"   # only needed for live submit
 
 # Plan-only (default): write the stub + print the `modal run` command.
 soup train --config soup.yaml --cloud modal --gpu a100
@@ -585,7 +585,7 @@ Full-screen Textual dashboard. Two-pane: run list (left) + selected-run detail
 (right). `r` refreshes, `q` quits.
 
 ```bash
-pip install 'soup-cli[tui]'
+pip install "soup-cli[tui]"
 soup tui --refresh 1.0 --limit 50
 ```
 

docs/commands.md

@@ -255,9 +255,9 @@ soup adapters bisect <ckpt>... --eval-command "..."  Binary search over training
 soup lock write --base-sha <h> --dataset-sha <h> --env-hash <h>  Write soup.lock (v0.67.0)
 soup lock write --base-sha <h> --dataset-sha <h> --env-lock soup-env.lock  Auto-derive --env-hash from soup-env.lock (v0.71.1)
 soup lock show / soup lock check              Show + drift-check (exit 3 on drift)
-soup compile <program.py> --eval <suite> [--optimizer mipro|gepa|textgrad|copro|bootstrap_fewshot] [--plan-only]  DSPy / GEPA / TextGrad prompt-program compiler — live (v0.71.13; pip install 'soup-cli[compile]')
+soup compile <program.py> --eval <suite> [--optimizer mipro|gepa|textgrad|copro|bootstrap_fewshot] [--plan-only]  DSPy / GEPA / TextGrad prompt-program compiler — live (v0.71.13; pip install "soup-cli[compile]")
 soup distill-prompt --traces <jsonl> --teacher <m> --student <m> --strategy sft|preference|kl [--provider ollama|anthropic|vllm] [--base-url <url>] [--temperature F] [--max-rows N]  Distill prompt-heavy traces via a live teacher (v0.71.13)
-soup compile-tools <spec.json|yaml> --eval <jsonl> [--optimizer textgrad|gepa] [--plan-only]  TextGrad / GEPA tool-schema optimiser — live (v0.71.13; pip install 'soup-cli[compile]')
+soup compile-tools <spec.json|yaml> --eval <jsonl> [--optimizer textgrad|gepa] [--plan-only]  TextGrad / GEPA tool-schema optimiser — live (v0.71.13; pip install "soup-cli[compile]")
 soup apple-adapter <source-dir> --direction hf-to-mlx|mlx-to-hf|hf-to-apple|mlx-to-apple --output <dir> [--sign] [--plan-only]  PEFT LoRA <-> mlx-lm adapter conversion — live (v0.71.21; *-to-apple upstream-gated exit 3)
 soup local-rl init --db <path>                Create personal-LLM flywheel SQLite schema (v0.68.0)
 soup local-rl status --db <path>              Print interactions / thumbs-up / thumbs-down counters
@@ -299,7 +299,7 @@ server over **stdio**, so any MCP client — Claude Code, Cursor, Cline, Continu
 can drive Soup conversationally. Install the extra first:
 
 ```bash
-pip install 'soup-cli[mcp]'
+pip install "soup-cli[mcp]"
 ```
 
 Register it with your client. For **Claude Code** (`.mcp.json` in the repo) or

docs/data.md

@@ -337,7 +337,7 @@ soup data generate --prompt "..." --validate
 # Auto-filter by quality (coherence scoring)
 soup data generate --prompt "..." --filter
 
-# Auto-dedup (MinHash, requires: pip install 'soup-cli[data]')
+# Auto-dedup (MinHash, requires: pip install "soup-cli[data]")
 soup data generate --prompt "..." --dedup
 
 # Full quality pipeline: validate + filter + dedup
@@ -591,7 +591,7 @@ soup data convert ./data/train.jsonl --to sharegpt --output converted.jsonl
 # Merge multiple datasets
 soup data merge data1.jsonl data2.jsonl --output merged.jsonl --shuffle
 
-# Remove near-duplicates (requires: pip install 'soup-cli[data]')
+# Remove near-duplicates (requires: pip install "soup-cli[data]")
 soup data dedup ./data/train.jsonl --threshold 0.8
 
 # Extended statistics (length distribution, token counts, languages)
@@ -814,7 +814,7 @@ Five checks: `length_bias` — the **#1 silent DPO degradation**: `chosen`
 systematically longer than `rejected`, reported as a Cohen's d effect size —
 `label_imbalance` (KTO desirable:undesirable ratio), `near_duplicates`
 (MinHash/LSH, reuses the `soup data dedup` kernel; requires
-`pip install 'soup-cli[data]'`, degrades to an advisory skip otherwise),
+`pip install "soup-cli[data]"`, degrades to an advisory skip otherwise),
 `identical_pairs` (`chosen == rejected` — zero preference signal), and
 `prompt_leak` (the prompt echoed verbatim inside the completion, a common
 synthetic-data pipeline bug). Same OK/MINOR/MAJOR taxonomy and exit codes as

docs/evaluation.md

@@ -432,7 +432,7 @@ Full-featured evaluation platform with standard benchmarks, custom evals, LLM-as
 
 ```bash
 # Install eval dependencies
-pip install 'soup-cli[eval]'
+pip install "soup-cli[eval]"
 
 # Standard benchmarks (wraps lm-evaluation-harness)
 soup eval benchmark --model ./output --benchmarks mmlu,gsm8k,hellaswag

docs/models.md

@@ -70,29 +70,29 @@ no PyTorch. Add `[train]` to fine-tune, or install other extras only when you ne
 
 | Extra | Install | What it adds |
 |---|---|---|
-| `train` | `pip install 'soup-cli[train]'` | Training stack: torch, transformers, peft, trl, datasets, bitsandbytes, accelerate |
-| `all` | `pip install 'soup-cli[all]'` | `train` + `serve` + `ui` + `data` in one shot |
-| `fast` | `pip install 'soup-cli[fast]'` | Unsloth backend (2-5x faster, lower VRAM) |
-| `vision` | `pip install 'soup-cli[vision]'` | Vision / multimodal fine-tuning (Pillow) |
-| `audio` | `pip install 'soup-cli[audio]'` | Audio / speech fine-tuning (librosa, soundfile) |
-| `mlx` | `pip install 'soup-cli[mlx]'` | Apple Silicon backend (mlx, mlx-lm) |
-| `qat` | `pip install 'soup-cli[qat]'` | Quantization-Aware Training (torchao) |
-| `serve` | `pip install 'soup-cli[serve]'` | Inference server (FastAPI + uvicorn) |
-| `serve-fast` | `pip install 'soup-cli[serve-fast]'` | vLLM inference backend (2-4x throughput) |
-| `sglang` | `pip install 'soup-cli[sglang]'` | SGLang inference backend |
-| `ui` | `pip install 'soup-cli[ui]'` | Web UI + inference server |
-| `tui` | `pip install 'soup-cli[tui]'` | Full-screen Textual dashboard (`soup tui`) |
-| `eval` | `pip install 'soup-cli[eval]'` | Benchmark evaluation (lm-evaluation-harness) |
-| `data` | `pip install 'soup-cli[data]'` | Deduplication (MinHash via datasketch) |
-| `data-pro` | `pip install 'soup-cli[data-pro]'` | Language detection + PII (langdetect, presidio) |
-| `deepspeed` | `pip install 'soup-cli[deepspeed]'` | Multi-GPU training (DeepSpeed ZeRO) |
-| `liger` | `pip install 'soup-cli[liger]'` | Liger Kernel fused ops |
-| `ring-attn` | `pip install 'soup-cli[ring-attn]'` | Ring FlashAttention (sequence parallelism) |
-| `onnx` / `tensorrt` | `pip install 'soup-cli[onnx]'` | ONNX / TensorRT-LLM export |
-| `awq` / `gptq` | `pip install 'soup-cli[awq]'` | AWQ / GPTQ quantized export |
-| `trackers` | `pip install 'soup-cli[trackers]'` | MLflow / SwanLab / Trackio logging |
-| `remote` | `pip install 'soup-cli[remote]'` | Remote datasets (s3 / gs / az / oci) |
-| `dev` | `pip install 'soup-cli[dev]'` | Tests + lint + types (pytest, ruff, mypy, pre-commit) |
+| `train` | `pip install "soup-cli[train]"` | Training stack: torch, transformers, peft, trl, datasets, bitsandbytes, accelerate |
+| `all` | `pip install "soup-cli[all]"` | `train` + `serve` + `ui` + `data` in one shot |
+| `fast` | `pip install "soup-cli[fast]"` | Unsloth backend (2-5x faster, lower VRAM) |
+| `vision` | `pip install "soup-cli[vision]"` | Vision / multimodal fine-tuning (Pillow) |
+| `audio` | `pip install "soup-cli[audio]"` | Audio / speech fine-tuning (librosa, soundfile) |
+| `mlx` | `pip install "soup-cli[mlx]"` | Apple Silicon backend (mlx, mlx-lm) |
+| `qat` | `pip install "soup-cli[qat]"` | Quantization-Aware Training (torchao) |
+| `serve` | `pip install "soup-cli[serve]"` | Inference server (FastAPI + uvicorn) |
+| `serve-fast` | `pip install "soup-cli[serve-fast]"` | vLLM inference backend (2-4x throughput) |
+| `sglang` | `pip install "soup-cli[sglang]"` | SGLang inference backend |
+| `ui` | `pip install "soup-cli[ui]"` | Web UI + inference server |
+| `tui` | `pip install "soup-cli[tui]"` | Full-screen Textual dashboard (`soup tui`) |
+| `eval` | `pip install "soup-cli[eval]"` | Benchmark evaluation (lm-evaluation-harness) |
+| `data` | `pip install "soup-cli[data]"` | Deduplication (MinHash via datasketch) |
+| `data-pro` | `pip install "soup-cli[data-pro]"` | Language detection + PII (langdetect, presidio) |
+| `deepspeed` | `pip install "soup-cli[deepspeed]"` | Multi-GPU training (DeepSpeed ZeRO) |
+| `liger` | `pip install "soup-cli[liger]"` | Liger Kernel fused ops |
+| `ring-attn` | `pip install "soup-cli[ring-attn]"` | Ring FlashAttention (sequence parallelism) |
+| `onnx` / `tensorrt` | `pip install "soup-cli[onnx]"` | ONNX / TensorRT-LLM export |
+| `awq` / `gptq` | `pip install "soup-cli[awq]"` | AWQ / GPTQ quantized export |
+| `trackers` | `pip install "soup-cli[trackers]"` | MLflow / SwanLab / Trackio logging |
+| `remote` | `pip install "soup-cli[remote]"` | Remote datasets (s3 / gs / az / oci) |
+| `dev` | `pip install "soup-cli[dev]"` | Tests + lint + types (pytest, ruff, mypy, pre-commit) |
 
 The complete, authoritative extras list is in [`pyproject.toml`](../pyproject.toml).
 

docs/performance-and-quantization.md

@@ -36,7 +36,7 @@ Train with simulated quantization for significantly better post-quantization qua
 
 ```bash
 # Install QAT support
-pip install 'soup-cli[qat]'
+pip install "soup-cli[qat]"
 ```
 
 ```yaml
@@ -71,7 +71,7 @@ QAT works with all training tasks (SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, P
 For H100 / H200 / B100 / B200 GPUs, train with float8 matmuls for ~2x speedup vs bf16 at comparable quality. This extends QAT infrastructure via `torchao.float8`:
 
 ```bash
-pip install 'soup-cli[qat]'   # torchao >= 0.5.0 includes torchao.float8
+pip install "soup-cli[qat]"   # torchao >= 0.5.0 includes torchao.float8
 ```
 
 ```yaml
@@ -106,7 +106,7 @@ Bool `true` stays on the int8 QAT path for backward compatibility. FP8 requires
 Models with 128k+ vocabularies (Llama 3.1, Qwen2) materialise a huge `(batch, seq, vocab)` logits tensor that dominates VRAM. Cut Cross-Entropy computes the loss in chunks instead:
 
 ```bash
-pip install 'soup-cli[cce]'    # or: pip install cut-cross-entropy
+pip install "soup-cli[cce]"    # or: pip install cut-cross-entropy
 ```
 
 ```yaml
@@ -368,9 +368,9 @@ data:
 Install optional performance packages:
 
 ```bash
-pip install 'soup-cli[liger]'     # Liger Kernel fused operations
+pip install "soup-cli[liger]"     # Liger Kernel fused operations
 pip install flash-attn --no-build-isolation  # FlashAttention
-pip install 'soup-cli[ring-attn]' # Ring FlashAttention (sequence parallelism)
+pip install "soup-cli[ring-attn]" # Ring FlashAttention (sequence parallelism)
 ```
 
 

docs/serving-and-export.md

@@ -97,7 +97,7 @@ Linux/macOS need only a C++ toolchain + CMake. CUDA llama.cpp builds are unteste
 Export models to ONNX format for use with [ONNX Runtime](https://onnxruntime.ai/):
 
 ```bash
-pip install 'soup-cli[onnx]'
+pip install "soup-cli[onnx]"
 soup export --model ./output --format onnx
 soup export --model ./output --format onnx --output ./model_onnx
 ```
@@ -107,7 +107,7 @@ soup export --model ./output --format onnx --output ./model_onnx
 Export models to TensorRT-LLM format for high-throughput GPU inference:
 
 ```bash
-pip install 'soup-cli[tensorrt]'
+pip install "soup-cli[tensorrt]"
 soup export --model ./output --format tensorrt
 soup export --model ./output --format tensorrt --output ./model_trt
 ```
@@ -212,7 +212,7 @@ Start a local OpenAI-compatible inference server:
 
 ```bash
 # Install server dependencies
-pip install 'soup-cli[serve]'
+pip install "soup-cli[serve]"
 
 # Start server
 soup serve --model ./output --port 8000
@@ -253,7 +253,7 @@ Use [vLLM](https://github.com/vllm-project/vllm) for significantly better throug
 
 ```bash
 # Install vLLM support
-pip install 'soup-cli[serve-fast]'
+pip install "soup-cli[serve-fast]"
 
 # Start with vLLM backend
 soup serve --model ./output --backend vllm
@@ -273,7 +273,7 @@ Use [SGLang](https://github.com/sgl-project/sglang) as an alternative high-throu
 
 ```bash
 # Install SGLang support
-pip install 'soup-cli[sglang]'
+pip install "soup-cli[sglang]"
 
 # Start with SGLang backend
 soup serve --model ./output --backend sglang
@@ -482,7 +482,7 @@ The OTLP endpoint is SSRF-hardened: only http/https schemes, plain HTTP only for
 Launch a local web interface to manage experiments, start training, explore data, and chat with models — all from your browser.
 
 ```bash
-pip install 'soup-cli[ui]'
+pip install "soup-cli[ui]"
 soup ui
 # -> opens http://127.0.0.1:7860 in your browser
 # -> prints auth token to console