01 / signal
Persisted token-ID tables select residual capacity.
Each layer stores a 2 × 32,000 route table that selects two distinct experts. The released checkpoint has no learned router or channel modulation.
Loading Complexity
complexity.deep / routing
Efficient models,less hidden magic.
We build inspectable transformer systems around deterministic token routing, shared expert capacity and executable open-source tooling.
routing_profile.py
201.2M
parameters
≈162B
source exposure
32K
vocabulary
top-2
hash routing
token_id
stable identity
route_table
fixed assignment
expert[e]
residual path
def route(token_id):
# fixed, inspectable, no learned gate
return shared(x) + experts[table[token_id]](x)
no learned routerstatus: full SFT released
[Research / source of truth]
Deterministic routing.
Claims kept precise.
Our published preprint documents the audited TR-Hash 492.1M/20B checkpoint — a separate release from the newer TR-HASH MoE 200M lineage, whose 130B base, refinement checkpoint and full SFT are now published. See /models for every current checkpoint and its exact status.
02 / capacity
A shared expert keeps the contextual backbone.
Every token keeps the shared SwiGLU path, while both selected experts transform the same contextual hidden state.
03 / audit
The exported checkpoint is the source of truth.
Direct inspection finds 492,097,536 trainable parameters, 24 distinct route tables, no duplicate top-2 routes and exact marginal expert balance.
04 / result
A complete 20B-token pretraining run.
The final checkpoint is step 76,293. The last scheduled held-out evaluation at step 76,000 records NLL 2.661519 and perplexity 14.32.
Scope guardrails
what the result does not claim
- One architecture and one seed do not establish superiority over dense or learned routing.
- The audit proves marginal expert balance, not uniformity over every ordered expert pair.
- Published benchmark results are tied to their stated checkpoints and evaluation protocols. They do not establish architectural superiority without matched baselines, multiple seeds and contamination audits.
[Open-source stack]
Research that ships as artifacts.
Models, runtimes, experiments and visual tools share one objective: make advanced architecture work easier to inspect and reproduce.
Released full-parameter SFT
TR-HASH MoE 200M
The released 32,000-token assistant cited by the public preprint: 201.2M parameters, a complete 130B base run, 32.07B refinement exposures and three epochs of full-parameter SFT.
201.2M parameters69.10% PIQA norm47.29% Combined ARC
Visual architecture laboratory
LABO AI
Build, inspect and execute PyTorch architectures as typed atomic graphs—with an agent that uses the same visible tools.
OpenAIElectronPyTorch
Interactive object detection
TR-HASH Vision v8 Demo
Run the compact hash-routed COCO detector on your own image, or draw a new example from 100 shuffled validation images without repeats.
2.53M parametersCOCORandom 100
Serving engine
TR-Hash-i64
Paged KV cache, continuous batching and an OpenAI-compatible API for token-routed inference.
InferenceCUDAAPI
Hugging Face organization
AETHORIA-AI
Every TR-Hash checkpoint, dataset and Space we publish, in one place — see /models on this site for the current releases with their real numbers.
ModelsDatasetsSpaces
PyTorch foundation
Complexity Framework
Canonical TR-Hash MoE execution, multi-hash routing, GQA/MHA decoders, CUDA paths and full-parameter pretraining, refinement and SFT.
PyTorchTR-Hash 200MFull SFT
[Inference / evidence]
Numbers with their context attached.
The cards preserve the separate CUDA-graph serving result. The figure reports zero-shot quality for the released 201.2M checkpoint, with model scale and protocol kept visible.
sustained throughput
8,078tok/s
observed peak
10,179tok/s
median TTFT
29.3ms
concurrent clients
100req
combined_arc_model_comparison.png
Full ARC public splits · zero-shot causal-choice · 201.2M versus 124M–774M
[Full-SFT expert geometry]
Inspect the routed contribution—not an old proxy.
This projection is regenerated from the released 201.2M full-SFT checkpoint on natural PIQA validation text, without a chat template. The layer menu exposes independently embedded layers 1, 4, 8, 12 and 16.
TR-HASH MoE 200M full SFT · expert_tsne_3d.html
Each point is one of the two actual routed residual contributions for a sampled token. The shared MLP output is excluded; vectors are L2-normalized before PCA and 3-D t-SNE. Cluster separation remains exploratory and is not evidence of specialization or model quality.
download 6,000 points[Research reports]
TR-Hash, documented end to end.
Every report lives with its architecture facts, audited claims, explicit limitations and a reusable citation. Current release status and measured checkpoints are tracked separately on the Models page.
Preprint · 2026
Deterministic multi-hash routing supports long-horizon training in a compact language model
Research Square preprint describing deterministic multi-hash routing, long-horizon training and the compact TR-HASH language-model experiments.
DOI10.21203/rs.3.rs-10788774/v1citation.bib
@article{Peyriguere_2026,
title={Deterministic multi-hash routing supports long-horizon training in a compact language model},
author={Peyriguere, Boris},
year={2026},
month={Aug},
publisher={Springer Science and Business Media LLC},
doi={10.21203/rs.3.rs-10788774/v1},
url={https://doi.org/10.21203/rs.3.rs-10788774/v1}
}Public report · 2026
TR-Hash Vision: Deterministic Spatial-Token Routing for a Compact Object Detector
A four-page report for the 2.53M-parameter hash-routed object detector: architecture, the two-stage from-scratch-pretrain-then-SFT recipe, and complete COCO AP results for both the one-to-many and NMS-free branches.
citation.bib
@misc{peyriguere2026trhashvision,
title={TR-Hash Vision: Deterministic Spatial-Token Routing for a Compact Object Detector},
author={Boris Peyriguere},
note={Research report},
year={2026},
url={https://www.complexity-ai.fr/papers/tr-hash-vision-v8-sft.pdf}
}[About the lab]
Open systems for understandable AI.
Complexity-ML is an independent open-source research lab in Paris, focused on transformer architecture, deterministic routing and practical model tooling.
01
Inspectable by design
Architecture choices, routing decisions and experimental caveats stay visible.
02
Artifacts over claims
Code, model releases, measurements and hosted papers accompany the narrative.
03
Efficiency with controls
Comparisons separate matched-token quality, training cost and serving throughput.