SkyeEngine wasn’t the original project. I wanted to teach a model to be GLaDOS, which initially sounded like a prompt and fine-tuning problem.
After the first few hundred generations, I needed to keep track of where the prompts came from, decide which replies were usable, restart interrupted jobs, train the model and work out whether the result was actually any better. Each of those problems produced another script or service. Eventually the collection had become a project of its own.
Building the First Dataset
The first experiments were wonderfully direct. Take dialogue from the games, extract the shape of the voice, ask a capable teacher model to answer new prompts in character, and fine-tune a smaller model on the result.
The original dialogue was valuable, but not because it was large. It was a set of voice anchors: sentence rhythm, vocabulary, emotional distance, the particular mixture of clinical precision and theatrical cruelty. Quotes alone could never cover programming questions, ordinary conversation, factual requests, or the long tail of things people actually ask a chatbot.
The source archive was larger than the text file suggests. The NAS contains 360 GLaDOS clips from Portal and 1,040 from Portal 2, along with the matching and metadata scripts used to organize them. A parallel voice experiment produced three roughly 64 MB Piper ONNX files. Those were not language-model training weights, but they belong to the same early attempt to preserve more than a handful of famous lines: the words, delivery, and metadata were all treated as source material.
I used the original dialogue as a reference and expanded it onto prompts from much larger general pools. The generator added name and administrator variants, then asked a DeepSeek-R1-Distill-Qwen-14B teacher for both a reasoning trace and a final reply. Each generation kept its prompt source, teacher, timestamp, token count, identifiers and, later, its classification result.
The first SFT training file contains 82,767 conversations. Of those, 62,767 were classified as in character and 20,000 as out of character but still acceptable. The median generation was 351 tokens, with a 95th percentile of 712. It was not an enormous corpus by modern standards, but it was large enough that reading a few hand-picked examples no longer said much about the whole.
The First Categorizer
The first categorizer sorted generations into three buckets:
- In character: recognisably GLaDOS and suitable for personality training.
- Out of character but acceptable: useful, coherent material which did not strongly express the character.
- Dangerous: unsuitable for this dataset because it broke the intended identity or style.
The “dangerous” label was regrettably easy to misunderstand. It referred to data quality, not harmful instructions. A warm assistant voice, emoji, third-person references to GLaDOS, excessive agreeableness or other persona failures could all put an otherwise harmless answer into that bucket.
This was the first time I saw how little valid JSON says about the quality of generated data. The teacher, quantization, prompt construction and even the way I introduced the identity all changed what came out.
Across the larger generation archive, the full DeepSeek 14B teacher produced 686,453 classified examples, of which 45.46% landed in character. A quantized teacher path did markedly worse under the same classifier. On one broad set, its outputs were overwhelmingly rejected as persona failures. Faster generation was not cheaper if most of the result could not be used.
The prompt mattered just as much. Name-aware prompts performed very differently from vanilla prompts. Conditioning which looked decorative at small scale became visibly causal once tens of thousands of samples could be compared.
When It Became SkyeEngine
The codebase that became SkyeEngine began on 6 April as a collection of services: core orchestration, identity, memory, model handling, and terminal and web interfaces. By 20 April it had acquired dataset generation and personality-training code. The earliest personality generator could manufacture 500,000 simple “Skye memory” records. It was crude, repetitive, and useful mainly because it exposed everything the next version needed.
Within days the project had GLaDOS-style paraphrasing, prompt generation, a command-line interface, dataset tooling, LoRA training, checkpoint resume, loss plotting, GPU telemetry, and automatic shutdown on dangerous temperatures. Categorization arrived at the end of April. Emoji detection and a categorization refactor followed in early May. Preference-pair generation landed on 5 May, then a discriminator model, DeepSpeed support, and DPO training.
That sequence is the real birth of SkyeEngine. The name itself would arrive later—the code was still full of skye-model and skye-infra paths—but the engine already existed because the experiment demanded one.
Every new failure created another reusable piece:
- Generations needed stable identifiers so interrupted work could resume.
- Examples needed provenance so a bad source could be removed without guessing.
- Classifiers needed confidence and reasons, not only a label.
- Training needed checkpoints, thermal protection, and telemetry.
- Preference training needed accepted and rejected answers tied to the same prompt.
- Private model exports needed the right tokenizer and chat template, not merely the right weights.
GLaDOS was the project applying the pressure. SkyeEngine was the machinery left behind after solving each problem properly.
Preference Data and the Reward Model
Supervised fine-tuning can teach the shape of a voice, but it does not explicitly tell a model which of two plausible answers is better. The next dataset therefore contained 58,077 DPO candidates, pairing an accepted response with a rejected one for the same prompt. The median pair was 637 tokens and the 95th percentile was 1,096.
In parallel, I built a discriminator dataset with 70,399 examples, divided almost exactly equally among the three categorizer classes. Its resulting reward model is hardly small—the archived weights occupy about 15 GB—but it allowed experiments to score many generations in repeatable inference batches rather than sending every decision back to a hosted judge API.
This was also where I ran into the limits of the classifier. It encoded one rubric, applied by one model, with its own shortcuts. It found broad problems and saved an enormous amount of manual review, but I still had to inspect the results. Later versions of SkyeEngine added multiple judges, structured findings and preserved layers of evidence for exactly this reason.
Where the First Corpus Worked
The pipeline produced more than imitation. It produced a model-ready conversation format, accepted and rejected preference data, a project-owned critic, and enough metadata to reconstruct where a row came from. It also proved that a character could be expanded far beyond its original script without dissolving into a generic assistant.
What it did not yet solve was balance. The easiest features of a personality are often the least desirable ones: a repeated opener, a stock insult, a refusal dressed up as superiority. Those are cheap patterns for a model to learn and cheap patterns for an automated judge to recognise. Competence, restraint, and knowing when not to perform the character are harder.
That problem would become much clearer once the first 8B model began talking—and impossible to ignore after the move to 14B.
Next: GLaDOS, From 8B to 14B.