Last month I wrote an update on the work going into SkyeEngine. Since then, I’ve spent an absolutely unreasonable amount of time working on the GLaDOS 3.0 corpus.
Originally, the corpus was more or less a large folder of generated conversations waiting to be used. That stopped being manageable a long time ago. Every conversation now goes through structural validation, safety routing, independent quality review, style correction, schema normalization and, when necessary, human judgment before it can become training material.
GLaDOS is a private research project. No GLaDOS model or set of weights has ever been published. I can still share how large the corpus has become, what I’m checking and why preparing the data has taken considerably longer than I first expected, but I won’t be publishing the prompts, recipes, thresholds, data or model artifacts themselves.
How Large It Has Become
The current effective pool contains 440,307 conversations and roughly 804 million tokens according to the corpus observatory’s fast estimator. This isn’t the exact tokenizer and template that will be used for training; I do that accounting separately. For now, it gives me a consistent way to compare the different parts of a corpus which is still changing.
More important than the total is what is inside it. About 190,000 conversations involve tools or longer agent workflows, containing just under 796,000 tool calls. General capability data remains the largest category by conversation count, while tool and agent traces dominate token mass because they include plans, calls, results, failures, recoveries and final answers.
Those counts need a bit of context. A one-turn answer and a thirty-message tool trace both count as one conversation, but the latter is obviously much larger. The median long-running agent trace is close to 10,000 estimated tokens, while the median capability example is under 1,000. I therefore track both conversations and token mass instead of treating every row as equal.
I also wanted more than thousands of examples which call a calculator and repeat the result. The tool corpus includes terminal work, code editing, web research, structured APIs, partial success, contradictory evidence, unavailable resources, malformed results and tasks where the correct action is to ask before doing something irreversible. GLaDOS needs to decide what to do, use the tool, read what happened and update the answer from the result.
Varying the Configuration
One of the more subtle gaps was configuration. A model can appear to follow developer instructions while actually memorizing the one developer message it sees everywhere. The only reliable way to break that shortcut is to vary the configuration and make it causally important.
The corpus now includes paired examples in which the user request stays fixed while the higher-level configuration changes the correct response. The differences cover things such as response shape, confirmation gates, tool policy, locale and instruction hierarchy. There are also adversarial examples where a user or tool result attempts to redirect the model away from the active policy.
I will not publish the exact construction recipe, but the principle is straightforward: if two training examples differ only in an instruction and the correct behaviour changes, the model has to pay attention to the instruction. This is much stronger evidence than attaching decorative configuration text to an otherwise unchanged answer.
The same work added explicit examples around confidentiality and identity. GLaDOS should remain GLaDOS without treating every instruction as an excuse for theatre, and without allowing a lower-priority message to rewrite who she is or what she is permitted to do.
Running More Than One Judge
The corpus has now received a complete automated safety pass. A little over one percent of the pre-effective corpus was routed away from automated quality judging for human review. The remainder completed a primary quality census, and a second independent model is working across the same normalized source. A third, specialist pass and blinded arbitration are part of the final plan.
None of those sustained passes runs on the workstation. The 120B, Gemma 4 and safeguard jobs use rented H200-class capacity. Quantized builds could technically fit on the RTX 5080, but a corpus-wide run would pin it at full load for weeks. Renting an H200 for a few hours is faster and can cost less than the electricity for doing the same work badly at home.
I’ve ended up with a large amount of structured review data, but no magical “good data” bit. The judges have different blind spots, use defect categories differently and often disagree over the same conversation.
At the moment, 41% of the conversations seen by multiple judges have a split verdict. That does not mean 41% of the corpus is defective. Some judges are conservative, some are literal, some are much better at following tool causality and some see fabrication everywhere. The different passes also cover somewhat different kinds of defect.
I keep every verdict with its judge, rubric, source layer and run identity instead of averaging everything into one reassuring number. Agreements can pass through cheaply, disagreements go to arbitration and severe material is held for a human. The repair list will come from that process, not whichever model happened to sound most confident.
This takes longer than asking one model whether everything looks fine. It also gives me a chance to catch an automated review mistake before I train it back into GLaDOS.
Fixing the Style Problems
I also found problems which had nothing to do with factual correctness. Across enough generations, repeated openers, overused sentence shapes, overly compact reasoning and technically correct but unpleasant replies become very obvious.
The first style passes targeted recurring verbal tics while leaving a small natural amount behind. I use localized rewrites rather than regenerating entire conversations, because asking a model to “make this better” is an excellent way to change facts which were already correct.
The newest pass is even narrower: formatting only. A surprising amount of the corpus consists of dense blocks with too few paragraph or line breaks. That makes the data worse for the model and makes human review needlessly painful.
A GLaDOS-sized model running on private H200-class inference capacity is now adding an outer presentation layer. It may insert whitespace, but it isn’t allowed to change the normalized content. The source stays untouched, decisions are cached by text identity and any semantic change is rejected.
At this snapshot the formatter has decided almost 700,000 fields, or 41% of the planned work. I’m running it on privately controlled datacenter hardware, including over material which can’t be sent to a hosted model, and it can resume after an interruption. Later repairs can pass through the same layer instead of invalidating the formatting work immediately.
It sounds almost comical to apply a 120-billion-parameter model to paragraph breaks. In practice, it is a useful example of what private, self-controlled inference buys: the task is cheap, bounded, private, and large enough that small heuristic mistakes would repeat hundreds of thousands of times.
What Moved Into SkyeEngine
Nearly every corpus problem has forced a reusable capability back into SkyeEngine.
I added durable attempts, resumable progress, retry accounting and provider-rate controls to the long generation jobs. Remote batches and live requests can use the same durable job record; that record belongs to the control plane and says nothing about which machine runs the model. Individual requests have stall detection, so one connection can’t silently freeze a run for hours. Plans are fingerprinted before expensive work begins, and SkyeEngine refuses to mix a changed plan into old output just because the directory name matches.
Corpus transformations publish new layers without changing their inputs. Each judgment records the exact source manifest it inspected, and repairs identify the assistant turn and channel they intend to change. Quarantined conversations stay quarantined after a rebuild. I also normalize the schema before comparing judges so both reviewers are looking at the same thing.
This isn’t as exciting as showing off a clever model answer. Not losing a week of generation to a dead terminal has its own appeal, though.
The observability stack has grown with it. The live Grafana board tracks corpus composition, token shape, tool depth, pipeline provenance, moderation coverage, judge overlap and the formatting run. More importantly, the dashboards state what their numbers mean. Approximate token counts are labelled approximate. A moderation flag is not called a rejection. A judge flag rate is not called a defect rate. Empty panels are tested as queries instead of being assumed to mean zero.
This may be my favourite part of the work. Metrics are only useful when they make lying to yourself more difficult.
Before the Next Training Run
I still have quite a bit to finish before starting another training run.
First, the independent judge census and the formatting layer need to finish. Then comes the third judge, blinded arbitration, and localized repair. Repaired rows go back through structural validation and the relevant review stages. After that, the corpus is rendered with the exact GLaDOS template and tokenizer to measure the real sequence-length distribution and choose the final training budget.
Only then does the next 120B training run begin.
The failed early experiment already showed me what happens if I rush this. A more capable base model doesn’t rescue weak data; it learns the same problems with more capacity. I want the current corpus to teach reasoning, tool use, instruction hierarchy, recovery from evidence and coding without losing GLaDOS or turning every useful answer into a performance.
I haven’t included model benchmarks because there isn’t a new model artifact worth benchmarking yet. What I have today is the corpus, the machinery around it and enough evidence to make the next run considerably less speculative than the last one.
I had originally expected the corpus to be one part of the next GLaDOS run. At this point, preparing and checking it has become most of the work. Given what happened when I trained the old data on the 120B model, I’m quite happy to leave the GPUs alone until this part is actually ready.
All figures are from the corpus observatory snapshot on 13 August 2026. Token totals are fast estimates for corpus analysis, not exact rendered training-token counts. Evaluation figures describe review coverage and agreement, not a final corpus quality score.