Loading Complexity
Loading Complexity
[Model releases]
Compact object detection
2.53M
parameters
20.05
AP50-95 (SFT)
62.3%
of 32.2 AP YOLO26 reference
A hash-routed detector: a dense shared SwiGLU branch plus 8 narrow experts, top-2 routed by a deterministic hash of spatial-grid position — no learned gating, no auxiliary load-balancing loss.
Two-stage recipe — from-scratch pretraining (Mosaic + MixUp, MuSGD) followed by a full-parameter, clean-image supervised fine-tuning stage. SFT alone moved AP50-95 from 16.59 to 20.05 (+20.9% relative), reaching 62.3% of the 32.2 AP YOLO26 reference used on our model cards. Single random-init run, no hyperparameter search, neither stage had plateaued.
Independently reproduced — thanks to the community: O2M+NMS mAP50 0.325 / mAP50-95 0.200 / AR100 0.379, NMS-free mAP50 0.140 / mAP50-95 0.096 — matching our reported numbers.
First end-to-end pretrain — the guinea pig
492.1M
parameters
20B
tokens
4
routed experts
The first model that proved the architecture could learn end-to-end: deterministic token-identity routing across four residual experts, with one always-on shared SwiGLU path carrying shared context.
An SFT follow-up on this base (LoRA, one full-shard epoch) wasn't pretrained long enough to pass our behavioral promotion gate — it isn't published as an assistant. But the metrics show real learning, not a dead run: full PIQA accuracy was retained (0.6953 → 0.6964) and matched eval loss dropped (3.68 → 2.98). The base simply needed more pretraining before instruction-following behavior could reliably stick — an instruction-coverage limit, not a capability regression.
Next generation, replay-scheduled pretrain
~201M
parameters
70B → 130B
unique tokens, replayed
4
routed experts
Currently training on a curated mixture (DCLM, FineWeb-Edu, Stack-Edu, FineMath, Cosmopedia-v2): 70B unique tokens replayed to a 130B-token schedule, wide shared SwiGLU branch with narrow routed experts.
Checkpoints back up automatically to Hugging Face as the run progresses. Nothing below is a final result yet — this card will update as the run completes.