CONTRIBUTORS.md

Contributors

Soup is built by its community. Thank you to everyone who has contributed code, tests, docs, and ideas. ❤️

This list is maintained by hand alongside the GitHub contributors graph. Merged a PR and don't see yourself here? Open a PR adding your line — that counts too.

Maintainer

• Alpamys (@MakazhanAlpamys) — creator & lead maintainer

Contributors

Listed by first contribution. PR numbers link the work.

• Salil Mhatre (@Deadpool2000)

  • soup version --json for machine-readable CI output (#6)
  • RAM + disk-space checks in soup doctor (#7)
  • soup runs clean for smart checkpoint space management (#9)
  • Official Docker support for easier onboarding (#20)
  • soup bench — model speed + VRAM measurement (#25)
  • --prompts-file option for soup bench (#30)
  • Happy-path + CPU-warning tests for soup bench (#31)
  • soup cost — cloud GPU training cost estimation (#42)
  • --nccl flag for soup doctor multi-GPU bandwidth checks (#178)
  • Ready-made qwen2.5-coder-7b-sft recipe (#285)
  • soup data split --stratify-semantic — a random split can leave a whole topic out of the validation set, so a regression in it is invisible; rows are now clustered by meaning and each cluster split proportionally (#388)

• Chinmaya Sahu (@csking101)

  • DPO example config, sample data, and tests (#48)
  • FP8 rowwise + rowwise_with_gw_hp scaling recipes (#62)

• Yixuan Xu (@mzl2233)

  • Guard diagnose-gate on distributed worker ranks (#169)

• dreamer0129 (@dreamer0129)

  • Rich-markup escape fix in legacy soup adapters commands (#175, adopted in-tree as #174)

• Vivaan Dhawan (@VIVAAN-DHAWAN)

  • Reject pickle/zip streams renamed to .safetensors via magic-byte check (#198)

• Shivam (@shivam2931120)

  • Tokenizer-aware repetition scoring for the echo-trap detector (#242)

• gittihub-jpg (@gittihub-jpg)

  • Manifest-level dotted-path custom transforms for soup build (#255)
  • --energy flag for soup bom emit — thread energy/CO₂ into the ML-BOM (#256)

• shatakshi-1404 (@shatakshi-1404)

  • Unit tests for the warmup.py auto-warmup-steps helper (#274)

• Kondamwar Akshaya Shrikant (@Akshaya-reddy18)

  • Friendlier error messages — richer CUDA-OOM hint + Hugging Face gated-repo and trust_remote_code mappings + tests (#282)

• Darsh (@CODING-DARSH)

  • Harden judge-URL validation against hostname prefix bypass (startswith → urlparse) in eval/gate.py (#288)
  • Apply configured vocabulary expansion (data.add_new_tokens / new_special_tokens) during SFT trainer init (#287)
  • Reuse the shared vocab-expansion helper in the vision + audio SFT paths (#291)
  • Honor configured vocab expansion in the DPO / IPO / KTO / BCO trainers (#293)
  • Honor configured vocab expansion in the ORPO / SimPO / GRPO trainers (#295)
  • soup mcp serve --allow-execute — the execution gate, kept a separate and stronger opt-in than --allow-mutating, with the tools still plan-only in this slice (#391)
  • Gated train_execute / export_execute behind a single-use server confirmation token, with the config snapshotted at plan time and protected directories digested by content rather than by mtime (#393)
  • Corrected a contributor's handle carried in the v0.73.2 CHANGELOG — a one-line fix to somebody else's credit, which is the kind of thing that normally goes unmade (#400)
  • Cross-tokenizer speculative drafts: a small model that would make a fine draft was unusable unless it happened to share a vocabulary. Four rounds, and what each one actually produced was a control that did not yet exist — a _FakeTarget that genuinely implements the repetition penalty (without it the fix was indistinguishable from a no-op), a _MAX_ALIGN_CHARS test that is not itself a 300-second wall-clock guard, and for the skip path both the (0,0) fixture AND the control that an extendable prompt still counts, since (0,0) alone is satisfied by a function that always skips. Then a clean textual merge of main silently restored the original symptom — #409's contract landed underneath and the assisted arm ran twice, the second call discarding the first's result — and they fixed it plus pinned the wiring, which had passed 163 tests while being deletable, because every CLI-level patch was lambda *a, **k (#417)

• Ekaanksh Patil (@Ekaanksh-dev)

  • Batch the PRM reward forward pass in PRMScorer.__call__ (single [B, T] forward) (#301)

• Sanjay Santhanam (@Sanjays2402)

  • Run built-in benchmark gate tasks through ForgettingDetector — every type: benchmark eval-gate task had always failed (#315)

• Nicolás Ramos (@nicolasramos)

  • backend: mlx was never dispatched — every MLX run trained through the transformers wrapper instead, and the saved MLX "adapter" was a full fine-tune because the model was never frozen before LoRA (#362)

• William Yang (@wilyan09007)

  • training.seed reached the SFT wrapper and nothing else — seventeen other task wrappers trained at HF's default 42 with no error, so replicates that differed only in the seed were the same run; the seed is now applied before the adapter is drawn, not only inside Trainer (#381)
  • Under use_fsdp2_compile, every checkpoint-* kept torch.compile's key prefix and resumed silently from a re-zeroed adapter — normalisation now runs as each checkpoint is written, ahead of anything that publishes it (#380)

• Amir Fathi (@AmirF194)

  • A streamed model's named_parameters() carried the wrapper's .inner. segment, so a name-keyed comparison against a resident model shared no names at all and a correctness gate reported 0/0 as a pass (#384)
  • training.stream_vram_override — the layer-streaming pre-flight measured free VRAM with a device-level driver query, so it could not see a per-process cap and there was no way to make it simulate one (#386)
  • The VRAM pre-flight never called its own calibration hook, so the guard against a stack whose loss path under-budgets by 12.5% sat inert with no caller (#390)
  • kl_control re-wrote the same β on every hold step, so a non-acting run was not the no-op log_only claims to be; the mitigation log now carries held / acted / released as a field rather than as free text (#414)
  • training.use_cut_ce: true was accepted and did nothing on every locally-produced checkpoint, because the architecture was matched against the model PATH rather than its config — the same defect v0.73.0 fixed for Liger. They found a second one on top of the reported bug: phi-2 dispatched to the Phi-3 patcher, which patches Phi3ForCausalLM and therefore patched nothing, while apply_cut_ce returned True and the run printed "Cut Cross-Entropy enabled" — success reported for work not done. Told the accurate version was stronger than their own framing, they took it rather than defending the original (#446)
  • live_eval.load_model_and_tokenizer had no quantization parameter, so every live-eval path loaded the base at full precision and an NF4-trained adapter was judged on a bf16 base it never saw. The reason this is worth reading is what happened when it was blocked: the parameter was threaded but no caller could reach it, and the CHANGELOG described five fixed commands — the exact "validated, documented, read by nothing" shape v0.73.3 is named for. Offered the choice of wiring it or splitting honestly, they split, and the entry now LEADS with "no live evaluation path sets it yet" and states the surviving user-visible defect in plain terms. They then closed two mutations that had survived, one of them the wire to get_compute_dtype() — the function v0.73.1 shipped for the T4 including_emulation trap, connected untested (#461)
  • build_optimizer_param_groups skips the catch-all base group when every parameter matched an lr_group pattern, and that branch had no test. Their one test is the sole guard on all three mutations I ran, including the realistic if base_bucket is not None: regression — an empty list is not None, so an empty base group gets emitted. It also pins group ORDER, which nothing did before and which matters more than it looks: optimizer.state_dict()["param_groups"] is positional, so a silent reorder breaks optimizer-state resume. And it deliberately does NOT assert dict equality, so adding a key to the emitted groups will not redden it — the restraint that keeps a new test from becoming a guard that fires on correct code (#469)
  • downsample()'s endpoint-pin branch was uncovered, and the gap was not the branch — it was that test_long_capped bounds the result with <=, so a regression in the stride arithmetic sails through it. Their test is the only thing that catches a + 1 there: 14 of 15 pass under that mutation. It asserts the exact row list plus an identity check on the endpoint, which is the right instrument for a branch guarded by is not. They also accepted that the assertion incidentally pins today's stride formula — which contradicts downsample's own docstring — and let it be filed as #473 rather than widening the PR (#470)

• Ben Younes (@ousamabenyounes)

  • MitigationLogWriter dropped every record in silence once its parent directory vanished mid-run — the controller kept acting while its evidence stopped growing (#398)
  • soup draft distill --steps N delivered only ~N/4.44 optimiser steps — val_split and gradient_accumulation_steps both divide the budget, and the epoch arithmetic ignored them (#399)
  • The soup ship MCQ scorer read �oxed {A} as no-answer — LaTeX permits a space before the brace and models emit it, and the cue tier cannot rescue it (#396)
  • --noise-floor shipped without a config surface, so it was the one soup ship gate-policy flag that could not be committed to soup.yaml; the bounds import from ship_verdict so the schema and the CLI validator cannot disagree (#410)
  • A dead MCP watcher left its run at running in the tracker forever — reconciled on read, with a Windows liveness branch because os.kill(pid, 0) there sends a console Ctrl+C rather than checking existence (#407)
  • The one-active-execution cap lived in process memory, so a restarted MCP server could double-book it (#408)
  • The soup ship leg-1 noise floor was measured in --task-mode metric only, so in the judge modes a win smaller than the instrument's resolution still counted; it is now measured everywhere and labelled, so a decode-only floor is distinguishable from one carrying judge variance (#419)
  • detect_disk_kind could not see through virtio, so a 1.5 GB/s cloud disk was classified HDD and refused the streaming tier — and when review found that the fix cited a rate from module state the cache never reset, producing 'hdd' (measured 2.00 GB/s, under the 1.0 GB/s NVMe floor), they removed the global rather than clearing it on the cache branch: the rate now travels in a frozen classification that is stripped on override, so the message cannot cite a verdict it did not produce (#411)
  • training.bnb_4bit_use_double_quant was validated and then read by nothing — every 4-bit path hardcoded True, so setting it changed the config fingerprint and nothing else. Made Optional[bool] = None rather than True, because a plain default emits the key into model_dump() and breaks round-tripping for 21 of 173 shipped configs; and when review showed the first round's tests fired on spelling rather than behaviour — and its companion passed against main's untouched file — both were deleted rather than patched (#418)
  • soup env check now audits the live environment against the bounds Soup declares about itself, so pip install vllm quietly downgrading transformers past the <5.0.0 cap is caught. The bound is read from package metadata rather than restated — and when review found the false-positive fix had, in closing it, made #368's own case unreachable, they narrowed enforcement to the ABI-relevant packages so both properties hold at once (#421)
  • soup bom emit / soup attest emit --attach-to-registry — a published soup card now carries its ML-BOM and in-toto attestation. Told the signed path registered the statement but not its detached .sig, so the card linked an attestation the registry alone could not verify, they fixed it the way they had already solved the same multi-file problem for bom --format both (#420)
  • soup draft measure and soup draft distill disagreed on what "shares a tokenizer" means, so a pair distill refuses could still be measured — and a completed acceptance measurement was then discarded when the assisted-throughput arm failed, because the report was only written after it. The persist-early half is the one worth naming: before this it was indistinguishable from writing at the end, and every pre-existing test passed against a version that only wrote from the except handlers. Four rounds, and each round they came back with the mutation matrix rather than an assertion that it was fixed (#409)
  • Layer streaming had no config key to disable pinning, so the only known workaround for a box that cannot page-lock enough memory was to not use the feature. training.stream_pin is Optional[bool] so unset reproduces today's behaviour exactly, and true announces and proceeds on the disk tier and on CPU rather than refusing an otherwise-fine config. The round that mattered was the third: their assertion on the require_pin wiring looked discriminating and was not — on CPU the expression is False whatever the config holds, so a constant satisfied it identically. They did not paper over that; they built the CUDA-gated version that distinguishes them and skips honestly on CI (#416)

• Faisal Fayaz (@Faisal01011)

  • Added the qwen3.5-4b-pretrain recipe — and shipped it with a test pinning the literal repo id, which is the only thing that catches a consistently wrong id (wrong in both RecipeMeta.model and the inline base:, so the two still agree). Every catalog-wide invariant passes that mutation; this is the defect class that shipped glm-5 pointing at THUDM instead of zai-org (#422)
  • Added the deepseek-v4-flash-grpo recipe, carrying the same literal-repo-id guard a second time — the consistently-wrong-id mutation stays green through every cross-field invariant and is caught only by that test (#432)
  • materialize_meta_adapters returns a count, and on newer peft it returns 0 as a matter of course — so 0 stopped distinguishing "nothing to do" from the silent no-training case its own docstring warns about. Demoted the count to a diagnostic and moved the decision into a separate postcondition, so the caller can no longer choose to ignore it; the trigger needs a peft this repo does not pin, so the test stubs the capability and asserts the decision (#435)
  • Then took the non-blocking follow-up from that review unprompted: the guard's lora_ restriction was called deliberate in its docstring but nothing pinned it, so a later broadening would have started refusing healthy streamed builds (#437)

• Shutaru (@Shutaru)

  • Kept the Transformers SFT import off the MLX dispatch route, so backend: mlx cannot reach the PyTorch/TRL stack even if sft.py stops being import-light later — and, told the PR did not fix the defect its title claimed, retitled it to match reality rather than defending the framing, leaving #394 open for the unexplained hang. The mlx-smoke job it adds asserts mlx is present, not merely that torch is absent: the earlier shape went green having executed nothing (#431)
  • Found and fixed a silent data-corruption bug in assistant-only loss masking: BatchEncoding is not a dict, so the guard missed and the mask was built from the mapping's key strings — no exception, normal loss curve. Split it out of #426 on request, then decided the case the issue asked to be decided rather than inherited: an all-zero mask with assistant messages present is rejected rather than honoured (measured: 0 trained tokens before, 2 after) (#439)
  • Layer streaming built its buffer-pool spec from layer 0 and refused any checkpoint whose layers disagreed, which excludes an MoE decoder outright. They relaxed the invariant properly — every layer's header read, specs unioned, a shared key that disagrees on shape or dtype still refused — and then met the harder half of the ask. This project's rule is that an arch enters the streaming allowlist only after a bit-exact gate against a resident run, and they could not load the real 35B resident; rather than argue for an exception they built a CPU parity gate on a toy heterogeneous MoE decoder, torch.equal on streamed-vs-resident logits, no CUDA marker, so it runs on every CI cell like the nine dense families. I confirmed it discriminates by silently streaming expert weights from layer 0 — same shapes, no crash — and it fails on the numbers. They also named their own alias table _UNVALIDATED_STREAM_ARCH_ALIASES and wrote the missing-resident-control caveat into the docs before anyone asked, which is why this was a review about closing a gap rather than about finding a concealed one (#426)

• Achuth Reddy Bangaru (@AchuthReddy-16)

  • soup train --no-reexec printed a launch command with the user's own flags dropped, so following it trained without --fsdp while still succeeding. Rather than patch the printed copy, they deleted it and derived the hint from the argv that actually launches the run — then, asked for a guard, wrote one whose exclusion set forces a decision for every new soup train flag instead of letting silence make it (#415)
  • A tokenizer output that is dict-LIKE but not a registered Mapping was read as a sequence, so its KEYS became tokens and the error named input_ids[0]='input_ids' — pointing the reader at their data when the cause was the type check. They went past the reported instance to the second, quieter one: the same gate at _apply_template_with_mask does not raise, it silently drops assistant_masks and trains unmasked, so widening only the loud gate would have left the worse half in place. Also renamed the helper data_doctor had been importing across module boundaries under a private name (#447)
  • Eleven callback modules imported transformers at module scope because a lazy-LOOKING factory was called EAGERLY — class X(_try_import_callback_base()) resolves the base in the class header — costing ~4.4 s per import and a 5x soup --help regression the moment a light command touched one. Measured after: 4.30s -> 0.03s, torch absent from sys.modules. The instructive half is the defect their first attempt introduced: deferring the classes while the builders still referenced bare globals, which PEP 562 __getattr__ does not serve, so four of five builders raised NameError on a cold import — and CI reported ONE failure, because eleven of twelve vanish in a full-suite run when an earlier file caches the attribute into globals(). Told that, they fixed all five, removed the ten # noqa: F821 that were silencing the diagnostic, and wrote the subprocess test that is the only thing which catches it: under a re-break, 37 in-process tests pass and only that one fails (#455)

• Emmanuel Ziggah (@blackcoderx)

  • On Windows a process that genuinely exits with code 259 was indistinguishable from STILL_ACTIVE, so it read as alive forever — defeating reconcile-on-read and able to wedge the MCP execution cap shut with no error an operator could act on. Disambiguated with WaitForSingleObject, and folded in the deduplication rather than fixing only the headline: two ~50-line copies of the liveness check became one shared module, with a test asserting identity so a third copy fails (#436)
  • soup data mix --optimize — the one command whose entire output is a config file — wrote one that would not parse, because data.train came out as a YAML list against a str field. Collapsed it to the highest-weighted dataset and kept the full ranked breakdown as a comment, so nothing the search learned is discarded; then found the same defect a second time in soup data mix --live's overlay and flagged it instead of widening the diff, along with the reason the suite was green on it — every --live test mocks subprocess.run, so the artifact is built and never loaded (#440)
  • Then found the same defect a second time, in soup data mix --live's overlay renderer, and fixed that too: every candidate proxy run was handed a config it could not load. Asked whether the PR should exist at all once they took on #443 rather than pushing or stalling silently — the right question, and the answer turned on something they had not seen (#443 has two resolutions and only one reverts it) (#445)
  • data.interleave was a documented, schema-validated key that nothing read at training time — the same "validated, documented, read by nothing" class v0.73.3 was named for, which they found while fixing #330 rather than being told about. Given a tightly-pinned scope they followed it literally, including the parts easy to fake: the back-compat guarantee holds by construction (a parallel loader beside the untouched single-path branch, not a refactor of both onto a shared path), and a renderer test loads its own YAML through the real schema and then through load_dataset(). Asked afterwards to close a hole in MY spec — widening train to a list blinded three consumers filtering on isinstance(path, str), one of them #297's plan->execute security gate, which contributed zero protected files — they fixed all three and wrote the ONE enumerating test that makes the next consumer a new row instead of the next defect (#460)

• Harshit Sharma (@harshitthek)

  • detect_device() did not know MLX, so an Apple Silicon run reported "CPU (no GPU detected)" and silently rewrote quantization: 4bit to none. The label was never the harm; asked for an explicit decision rather than a disappeared warning, they extracted resolve_quantization() with the mechanism named in its docstring — and extraction is also what made it testable, since the surrounding function is 0% covered (#428)
  • Added the qwen3.5-9b-grpo recipe with the literal-repo-id guard the house pattern calls for — the mutation that matters is an id wrong CONSISTENTLY in both RecipeMeta.model and the inline base:, which a test comparing the two to each other passes and which once shipped a broken recipe here. Also diagnosed a licence-header guard of mine that had turned nine CI cells red, inside half an hour and including the half I had missed — that an uncommitted file is absent from git ls-files, so the scanner never read itself until it landed (#448, diagnosis in #449)
  • A GPU-ceiling test compared two probes taken at different moments with a 10% tolerance standing in for machine drift, so it failed on correct code under bursty contention. The issue named widening that tolerance as the WRONG fix; they deleted it instead, moving the comparison inside a single measurement so there is no second moment to drift against — and then went past the ask by writing the control without @requires_cuda, so the guard the issue complained CI could never see now runs on all nine cells (#451)
  • Issue #456 reported that cut_ce split the model name on "/" only, so on Windows a parent directory naming an architecture selected a kernel for a model that is not that architecture, and it warned that liger might be differently wrong. They measured it instead of assuming: liger matched the WHOLE string, so it over-matched on every platform including POSIX, and it picks the wrong kernel rather than an unnecessary one — a Phi model under a llama_experiments/ parent got the Llama kernel. The finding was bigger than the report. Asked afterwards for the POSIX control that half of the fix had no test for, they added it, and it is load-bearing: a fix that normalizes separators but keeps the whole-string match still fails it (#458)
  • Two recipe PRs landed in sequence and the five documentation sites auto-merged silently while only the test files conflicted, so the stated recipe count drifted with nothing to catch it. Their ratchet derives the count from the catalog and fails on patterns that stopped matching, not only on wrong numbers — the half that keeps it working after someone rewords a doc line, and the half almost every guard of this kind gets wrong. It took two rounds on one principle: v1 asserted an exact site count and so reddened the build on a correct edit (a second, correctly-numbered mention; any sixth site), and relaxing that then silently gave up detecting a site dropped from DOC_SITES — a property the test's own docstring claimed. They closed both at once by pinning the roster's path AND pattern, and went past the ask with a test for the roster logic itself. They also reverted their own test-count edits back out on being shown those drift on every merge touching tests/, and filed #465 rather than widening the PR (#457)
  • Noticed that CONTRIBUTING.md's hand-maintained test-suite counts had gone stale and proposed a ratchet. Shown a measurement that a ratchet there would redden the build on more than half of all merges — 11 of the last 20 commits touch tests/ — they took the subtractive fix instead: delete the numbers so drift becomes impossible rather than merely detectable. Then, told their first shape test asserted "no digits" and would fire on this file's own house style ((v0.X.Y) tags sit on every sibling line), they narrowed it to a count shape. Verified both directions: a version tag with a bare digit keeps the suite green, a real count fails by name. Two rounds, both of which they resolved by making the guard smaller (#467)

• Osheun (@Osheun)

  • Added the glm-5.1-dpo recipe — and on a first PR, carried the literal-repo-id guard the house pattern calls for, in all four places, unprompted. It earned its keep immediately: the pre-existing family guard test_no_recipe_references_thudm_glm5 does NOT fire on a consistently-wrong THUDM/GLM-5.1, because it matches the lowercase glm-5 spelling — so the guard against that exact historical defect was theirs, not ours. They also updated CONTRIBUTING.md's recipe count, the site a sibling recipe PR missed and that I missed on a release (#452)

---

Want to join this list? See CONTRIBUTING.md — good first issues are labelled in the issue tracker.