Skip to content

v07114_qa.md

Latest commit

 

History

History
79 lines (58 loc) · 2.89 KB

v07114_qa.md

File metadata and controls

79 lines (58 loc) · 2.89 KB
Raw

v0.71.14 — Export QA log

Patch: v0.71.14 — "Export QA + serve(transformers) + GPU smoke finale" Box: Windows 11 + RTX 3050 Laptop GPU (4 GB, Ampere, compute capability 8.6) Date: 2026-06-05

Environment:

lib version
torch 2.5.1+cu121
transformers 4.57.6
optimum 2.1.0
onnxruntime 1.23.2
CUDA available yes (RTX 3050 Laptop GPU)

#71 — ONNX export pipeline — PASS

Verified soup export --format onnx end-to-end on a locally-built tiny GPT-2 (2 layers / 2 heads / n_embd=32 / vocab 512, saved with safetensors so it loads under torch 2.5.1).

soup export --model ./model --format onnx --output ./onnx_out

Result:

  • Export exited 0.
  • onnx_out/model.onnx written — 252,445 bytes.
  • onnxruntime.InferenceSession(model.onnx, providers=["CPUExecutionProvider"]) loaded it successfully.
  • Inputs: ['input_ids', 'attention_mask', 'position_ids'] — input_ids present as required by the acceptance criteria.
  • One forward sess.run(...) on input_ids=[[1,2,3,4]] produced output shape (1, 4, 512) = (batch, seq_len, vocab) — a real, runnable graph.

Notes:

  • Tracer warnings (aten::index advanced indexing, masking_utils boolean conversion) are emitted by optimum.exporters.onnx.main_export during the trace; the exported graph still loads + runs. These are upstream optimum/torch warnings, not a Soup bug.
  • The historical v0.53.2 limitation (TinyLlama-1.1B ONNX export OOM during onnx.load(load_external_data=True)) is a host-RAM constraint, unchanged; the pipeline integrity is what #71 verifies, and that is confirmed here.

#70 — GGUF export — DEFERRED (infra-blocked)

Needs a built llama.cpp C++ toolchain (the pipeline shells out to a pinned llama.cpp tag). The maintainer's Windows 11 + RTX 3050 box cannot reliably build the llama.cpp C++ chain — this is stated in issue #70 itself. Kept OPEN with the infra-blocked label; recipe + acceptance criteria live in tests/qa/v053_qa.md.

#72 — AWQ + GPTQ export — DEFERRED (infra-blocked)

Needs autoawq + auto-gptq (or gptqmodel). None are installed and their Windows wheels are unmaintained / build-from-source only (autoawq is archived). Kept OPEN with the infra-blocked label.

#144 — CUDA + llama.cpp manual QA smoke doc — DEFERRED

Depends on #70 (GGUF) and #72 (AWQ/GPTQ) actually running, both of which are infra-blocked on this box. Kept OPEN.

#74 — HF Hub push + Spaces deploy QA — DEFERRED (external account)

Needs an HF Write token + account (no credentials available on the QA box). Previously deferred in v0.53.4 to "a contributor with private HF credentials". Kept OPEN with the infra-blocked label.

#79 — [Tracking] Community-QA meta-issue — STAYS OPEN

Closes only once #70–#75 all land. Since #70 / #72 / #73 / #74 / #75 remain blocked on this box, the tracking issue stays open.