From a one-condition failure to a live research roadmap.
Fixed gravity. The model memorized one condition instead of learning a variable.
300 episodes, gravity held constant at 1.0, a 22.3M-parameter predictor trained from scratch on a pretrained visual encoder. Training and validation loss looked healthy — 0.1301 vs 0.1329 — but that only measured performance inside the one condition it had ever seen.
Randomizing gravity during training fixed in-range generalization almost entirely.
Gravity sampled from 0.3–2.5 per episode, 500 episodes, improved temporal pooling and a JEPA-style loss with a per-position mask token. In-range error fell from 541% to 5–31%. Extrapolation beyond the trained range — 85–135% — remained the open problem.
Four physics variables at once, plus a test of few-shot adaptation to the unfamiliar.
Gravity, restitution, ball radius, and friction randomized together — 1,000 episodes, 32–35M parameters, five staged tests (A–E). In-range performance held at the same 5–31% band even with four variables moving jointly. Out-of-range error narrowed further to 62–67%. Then Stage E asked a new question entirely.
Added actions, switched to real robot data — and few-shot adaptation stopped being reliable.
V0.3.1 only predicted what happens next; V0.4's goal was predicting what happens next given an action — the step toward robot control. The synthetic ball simulator was swapped for real Franka-arm data (Hugging Face's muacha/droid-30-converted, 30 episodes, ~437 training windows), with a frozen V-JEPA2 encoder feeding a new 30M-parameter action-conditioned predictor trained for 8-dimensional joint actions.
After 30 epochs, validation loss plateaued at 0.378 — about 20× worse than V0.3.1's 0.019. Few-shot adaptation, tested across six combinations of episode and learning rate, swung between +10.9% and −51.6% with no consistent pattern.
Back to controlled simulation, starting from V0.3.1's weights — and few-shot adaptation is now trained in, not bolted on.
V0.5 follows one rule: competitors are racing to collect more data; Kertory is racing to learn more from less of it. Three concrete changes from V0.4: (1) back to the controllable KertoryWorld simulator, initialized from V0.3.1's already-trained weights rather than from scratch; (2) intervention-effect supervision — each episode now pairs one starting state with two different actions, so the model is shown directly how the outcome should change with the action, not just one averaged result; (3) Reptile meta-learning folded into training itself — every fourth batch, the model takes a mock few-shot test, adapts in 3 steps, and nudges its real weights toward being faster to adapt, rather than adaptation being a separate step applied after training.
If intervention-effect + meta-learning works on synthetic physics, take the same method back to real DROID data.
This time with meta-learning included from the start, and a larger slice of DROID — roughly 100+ episodes instead of V0.4's 30 — aiming for few-shot adaptation that holds up consistently on real robot footage, not just in simulation.
Off the simulator, onto a small real robot arm — plus a permanent public demo.
Planned: a permanent Hugging Face Spaces demo (the current one is temporary), a test on a small physical robot-arm prototype outside simulation, and extending to multi-object, multi-task scenes. The goal across V0.3.1 to V0.7 is one running benchmark: how many examples it takes to reach a given improvement, tracked version over version.
