what if

Think about the way you text your closest friends versus how you write a work email versus how you talk to your parents/family. Three completely different registers, vocabularies, rhythms, and you switch between them instantly because you have spent years learning each context. Every AI assistant you use today starts from zero on that knowledge, requiring that you spend the first few exchanges for each chat re-establishing who you are or how you like to communicate or what you are trying to achieve. What if, instead, there was a version of chat, an AI that already knew all your registers because it learned them from data sources curated by you and your past experience. Or imagine you are a software engineer, with years of commits behind you and strong opinions about how code should be structured. A personal model that has access to your GitHub account could know when you say “write a function that does X” exactly what your expectations are without you having to explain a particular commenting style or project structure. Or picture a chef who has spent a decade building a distinct flavor philosophy who wants help brainstorming a new dish. A tool that is actually useful might be privy to the chef’s past recipes, places of work, and general flavor preferences, knowing, when asked, which ingredients keep appearing and which directions keep getting rejected, and can then suggest something that fits the chef’s culinary voice. What makes the useful version of each of these forms of machine intelligence different is the same thing in every case, namely that a model which knows this specific person and is trained on a curated set of data provided by that person is a different goal from knowing everything generally. Neil Lawrence describes each of us as a butterfly in a diving suit in The Atomic Human, the idea being we have far more cognitive capacity than we can express through the narrow bandwidth available to us, and a model calibrated to one person is the first thing that starts to address that by widening the channel between what someone thinks and what they can express.

just a rather very intelligent system

Every time Tony Stark suits up in the Iron Man films, J.A.R.V.I.S already knows what Tony needs before he asks. That relationship works because Tony built J.A.R.V.I.S specifically for himself and named it after Edwin Jarvis, the family butler who was the most constant presence in his entire childhood. J.A.R.V.I.S, according to Wikiepedia, also stands for Just a Rather Very Intelligent System which I am sure Tony, like many of us engineers, probably came up with after choosing the name. J.A.R.V.I.S carries Tony’s communication style, technical vocabulary, sense of humor, and priorities under pressure. Tony never needs to “re-establish” who he is at the start of a session because J.A.R.V.I.S understands who Tony is intrinsically. What makes J.A.R.V.I.S feel like an extension of Tony’s mind is that J.A.R.V.I.S, as a system, was calibrated to Tony, built for him, and constantly updated by him, with each new chat/conversation/project they work on together building on J.A.R.V.I.S’ conext and understanding. As we all know J.A.R.V.I.S is fiction. Nobody has built a system, at least not yet, that produces J.A.R.V.I.S becuse the tools and strategies deployed today are, in many ways, antithetical to a personalized design, preferring general intelligence. Put another way, every major AI assistant on the market today was built for everyone simultaneously, which means they are optimized for no one in particular, this is why, whenever you open a chat to GPT or Claude or Gemini, you have to start from zero, because it only knows everything, and not you.

iphones and the app store

Apple reframed the smartphone by first creating a phone with a touch screen and then creating a platform where developers could create apps and sell them to any iPhone user. The App Store made it possible to have a device that you could regularly upgrade by adding functionality through simply downloading any idea someone built; gaining access almost instantaneously. The iPhone plus App Store model compounded into one of the largest businesses in history because it matched how people actually want to relate to a powerful tool, owning it, controlling it, shaping it to their life.

The current AI landscape is the antithesis of that model. You do not own a model and have to instead, rent access to it, per token, rate-limited, on someone else’s servers, with your interactions potentially used to improve a model that serves everyone else too. And on their terms. Further, every session you have with a frontier model starts fresh apart from any “personalization” you have added through complex and often unwieldy system prompts that get injected at your cost through token budgets. The intelligence of frontier models like GPT is averaged across all of humanity’s patterns procured first through literally scraping the entire internet regularly, and then through specialized steps like supervised fine tuning and reinforcement learning. These steps are tuned with specific goals in mind for a general audience, and with the watchful eye of engineers and marketers who are trying to get the most bang for their buck. The more you use a model, unfortunately, the more you pay, with no equity in what you have contributed and no persistence from one conversation to the next.

It’s actually possible to create a model trained on a general dataset like the internet but that is specialized to your goals and in your style, just doing so would mean that these frontier providers would either need to spend more time on scaffolding training tools like SFT (the supervised fine tuning I mentioned before) and RHLF (reinforcement learning from human feedback), but that would also mean a cost to their bottom line because hallucination (error rate where the model messes up) actually makes them money since you are likely to re-ask the same question if an error happens in another way, earning them more money from token consumption. Your system prompt, how you “personalize a chat assistant” goes every time you make a request, even if it is just a retry, your results get processessed again along with the entire chat history and any internet search or tool call happens again too…all consume tokens, and tokens = money for these providers. Even if you pay per month, you get a token budget, and when it is used up, you get rate limited, something like “sorry, wait 2-hours until you can make another request.”

For context, I spent 198.76 USD in token charges above the $20 per month I spend on Claude trying to buffer rate-limits. These fees were largely attributed to failure rates in responses or from almost completed tasks cutoff prematurely. Also, because these models are generalized and serve everyone at once, models themselves have variable timeframes for when they add the most value tied to when the least amount of people are using them. As a result, I have found I, and others, tend to optimize working with AI tools in the wee hours because that is when they tend to mess up the least (since less people are using it), resulting in fewer fees, since less corrections are required.

For the last several months, I’ve been looking into ways to create my own specialized models that would give me independence from the billing of these providers, hopefully decoupling my bank account from theirs. The first strategy I tried was building an inference network where open source models are served through an API service I manage, removing rate limiting. This worked but open source models are only as powerful as their parameter size and the compute available to them, and unfortunately, getting compute at the level of these frontier providers literally costs millions of dollars, so while I did get a free version of ChatGPT working for myself and up to 50 other users at once, it wasn’t good enough for the day-to-day engineering I was hoping to replace. I did complete and deploy network at 429 Inference.com, anyone can join for free and get access, just reach out.

The next idea I had was to investigate writing a specific constraint language. I called it Jennifer Lang (ionically named after my Dad’s previous girlfriend). The goal of Jennifer Lang was to improve the determinism of language models like ChatGPT and Claude by constraining them in a virtual machine, and then transforming their outputs by replacing symbols/characters/styles with preferred alternatives, and only once they are transformed, return results. I wrote up Jennifer Lang and tried it out with Claude and GPT with some success but found it only improved determinism, the metric of not hallucinating and creating predictable results, by only a few percentage points, which, for my use case, wasn’t enough since my bank account would still suffer. I needed a solution that would permanently improve determinism, or at least allow me to use a model for free. Jennifer Lang worked by returning an error and exiting when the model returned a result that did not match the preprogrammed specifications which was great, because I immediately knew when a problem/hallucination/error happened, but was bad because it meant I would have to, once again, make another request, costing me more money, each time and error propagated. Practically, about 30% of my requests would error, aligned with industry research, on something as simple as “do not use em dashes.” For a response like “You are good very good – almost as good as Rudolph” Jennifer Lang’s virtual machine would replace the em dash with a comma, catching the error, and then process the rest of the request, but would then find another error, like the URL provided as a reference by the model 404’d. Jennifer Lang was really only useful for telemetry and for getting higher quality responses, but was not reducing my costs. I have open-sourced Jennifer Lang and made it publically accessible. The code is still private, but anyone can request access by reaching out to me. You can more read about Jennifer Lang, here.

Moving on to this week, I’ve changed strategies yet again, venturing into machine-learning-land where I’ve resolved to write and create my own model trained on a dataset that I curate. Keep in mind I have never written a bayesian inference model before and have very little experience with machine leaerning, generally. LOTS OF LEARNING. What I have found so far is that creating a language model is not at all easy, as I am sure you might have exepected. I thought at first, knowing what I know about language models, that the hardest part would be acquiring data, but it turns out that is only somewhat true. Large amounts of internet data are readily available via what’s called Common Crawl and The Pile. Both sources, terabytes of data in the case of Common Crawl, and 864GB in the case of The Pile, can be easily downloaded (if you have the space), and then used to train a model, given you have the compute to do so. Which no one does. To give you an idea, to process only about 80GB of data, my 5090 GPU with 32GB of VRAM takes about 5hrs, and that’s only with a small parameter model of 300M. It’s complicated to explain all the details here of parameter counts and tokens and all that, but the gist is that you can get the data, but once you have it, it’s very hard to use it and even harder to turn it into a model you can meaninfully use. Once you have the data, the next step is filtering downloaded data, which you can do with traditional programming strategies like regex, etc, and then putting it in a location your model can train on. This is colloquially called a training corpus. Corpus is a strange word that reminds me of corpse, but the origins of it are actually more humble, it simply means body in latin, so basically your models “body” of work. So I guess still corpse but not dead. After you’ve got your body of work, you then write your transformer and other programming such as a tokenizer using Python, and then check the code, run it, and wait HOURS. Mine took greater than 5-hours the first time, not sure because it was overnight while I was sleeping. When I woke up the next day, there was no error. Thank goodness. After pulling out the best checkpoint I sent through a query, simply saying “hello” to the model. The response I got was terrible garbled uselessness. Below you can find my exact command:

python3 generate.py --model nano --prompt "hello" --max-tokens 60 --temperature 0.8 --seed 7 2>&1 | grep -v "^I0\|^W0\|cudart\|CUDA error\|failed call\|cpu_feature\|absl\|FMA\|rebuild"
Loading checkpoint: step_0000300
Model loaded  (nano)

And what it looked like:

============================================================
PROMPT: hello
------------------------------------------------------------
hello, and
 < by a
,. a to be
,: as.
 we) the ". a as- a is to

 that/-i the>

 of

 <
 as. and

 of a
', < a to
============================================================

So as you can see, it wasn’t very useful. The next day, I learned that I needed to go through a process called supervised fine tuning or SFT for short but before that, I needed to improve the quality of my training data. To improve my body of work, I downloaded various specialized internet trained set ML engineers use to improve model quality on topics in general knowledge like Wikipedia and GitHub repos. Anyone can freely download them via HuggingFace, a HUGE thank you to all those who made these datasets free and accessible. After adding that data to my corpus and not committing it to git because of the size, I waited another 6 or so hours and got the following response:

Loading checkpoint: step_0048315  (device: cuda)
Model loaded  (small)

============================================================
PROMPT: hello
------------------------------------------------------------
hello in the presence of artificial intelligence and scientific knowledge.

We describe how to design and implement software for the digital twin ecosystem, with the aim of providing the user with a unique interface to the physical world.

In this article, we describe an end-to-end framework for the design and implementation of the digital twin ecosystem. We provide an overview of the methodology proposed by Brian Tao and Sudan to explore the design and implementation of the digital twin ecosystem.

The digital twin community has made new opportunities to build upon the design and implementation of digital twin technologies. By using digital twin technology, the virtual twin ecosystem is expected to be able to create a digital twin environment, to create and implement the digital twin ecosystem. The Dafny community has seen a large amount of research efforts, particularly for applications requiring users to participate in the design and development of the digital twin ecosystem. However, their integration with digital twin environments has not been widely adopted in real-world
============================================================

============================================================
PROMPT: write a python script for printing hello world
------------------------------------------------------------
write a python script for printing hello world--The purpose behind the development of language-mediated language learning is to generate language skills. We evaluate whether language skills can be used to perform language skills in the process of writing skills, and how such skills are supported. We evaluated our approach on five state-of-the-art LLM agents using four programming languages (4 programming languages), Claude Code, and GPT-4o. Our results show that skill-based tools can be used to generate code, but their ability to produce skills automatically remains unclear. Our results indicate that skill skills can be used to generate feedback from the environment, enabling developers to use their skills to complete tasks without relying on them.

Designers who would like to learn software or improve software maintainability may not exist. However, their impact on how code changes shape the behavior of the software and when it changes, and this gap is particularly evident. Here, we study the effect of developer effort on the design of code change outcomes. Our study
============================================================

============================================================
PROMPT: what is the meaning of life?
------------------------------------------------------------
what is the meaning of life?Debugging an Invoice

In this paper, we present a formal, step-by-step approach to analyze how AI systems behave on complex and evolving tasks. We use the Model Context Protocol (MCP) to generate a concrete model for each task, a structured prompt, and a multi-step prompt. The model is built on the Model Context Protocol (MCP). We define prompts for which to apply a prompt to a domain-specific prompt, and describe the prompt as a user-friendly prompt and define a tool for generating code that combines natural language, visual question answering, and human-in-the-loop control. A key aspect of our approach is how to integrate it into the MCP interface itself, where LLMs can be used to iteratively generate specifications for which they should not. We implement our approach in a prototype tool called LatticeFlow, a new LLM-based agent based on a toolchain. We evaluate our approach on
============================================================

============================================================
PROMPT: who is austin harshberger
------------------------------------------------------------
who is austin harshbergerAI systems, AI systems, and AI agents are increasingly deployed in the same domain, often overlooking how the environment evolves, the dynamics of behavior, and the consequences of these changes. This paper presents the first systematic study of AI agents' social context - how AI systems evolve from observable sources to behavioral realities - across three socio-technical dimensions: the interplay between what they see and what they see, and the emergence of AI agents that shape what they see. The analysis reveals that the "AI-enabled" paradigm emerges as a "realistic phenomenon": agents see a "future's" transition from passive observers to active observers, whereas AI agents see a "global world" moving toward a "civilization" that emerges through a "shaping of AI systems", but also through a "tactile reality". To examine this distinction, we propose a novel framework to quantify how AI agents move from passive observers
============================================================

You can see NONE of these are very good, which is where more fine-tuning came in. Eventually I ran SFT, which ended up eating up my entire hard drive. I’l spare you the details of why, suffice it to say, I messed up the way the pretraing checkpoints were saved in my code, so each checkpoint was 160GB of space (each checkpoint is an epoch and there was 10 for my model…). SFT gives a model the ability to respond in a question answer format. Essentially, you give it 100,000 question/answer formatted JSON files, and train it on that structure. After SFT, here’s what my model was returning:

============================================================
PROMPT: what is the meaning of life as you see it?
------------------------------------------------------------
what is the meaning of life as you see it?
What is life as we know it? What is it called?
What does life mean?
What is life as we know it?
How do you think of life?
What is life as we know it?
What does life mean?
What is life as we know it?
Why does life mean life?
Is life a good thing or a bad thing?
What are life as we know it?
How do you think of life as we know it?
What is life as we know it?
What is life as we know it?
How do you think of life as we know it?
What is life as we know it?
What is life as we know it?
What is life as we know it?
What is life as we know it?
What is life as we know it?
What is life as we know it?
What is life as we know it?
How do you think of life
============================================================

============================================================
PROMPT: what is the capital of the USA?
------------------------------------------------------------
what is the capital of the USA?
The U.S. is the largest nation in the world, with over 1.6 billion people living in it and over half the population residing in it. It is also a major producer of food, water, and electricity. With the development of our country’s economy, we need to develop our economy by increasing our consumption of natural resources such as oil and gas.
How will we get more access to electricity from the United States? The average American household uses about $20 per day for electricity in its daily life. In many cases, that amount will be spent on electricity for an entire day or even years. The amount spent on electricity is still determined by the year, but you can still estimate your monthly income using the current dollar. This number has been determined by the National Renewable Energy Laboratory (NREL), which calculates your annual income by comparing your income to your annual income per day.
What is electricity?
Watts is the primary energy source in
============================================================

These reponses seem worse but they are actually, technically, better and just require more refinement. Specifically a step called reinforcement learning or another strategy like DPO parings. Which leaves me to today, where I am now trying to decide, given how essentially useless my model currently is, what to do next, and instead of reading more reserach, and thinking aloud through my blog, writing this and another article, llm skill acquisition and style control outlining some of the research I did this morning.

what would it look like to make something different?

Designing a better model architecture or using an existing model and specializing it are probably next steps but it’s also possible to completely change strategies and use a different method of data aggregation and training, or at least I think that must be the case.

My Dad and I were watching a show on Apple TV about space, Star City, as we usually do, which often inspires me to try something new. I have been thinking about this idea of communication bandwidth between human-to-computer and human-to-human since reading The Atomic Human by Neil Lawrence. Basically, we, humans, can only communicate at around 150 words per minute, which is about 1800 bits per minute or 30 bits per second, because each word is roughly 12 bits of information. Computers can communicate data at a gigabyte per second. 1 gigabyte (GB) is 4,444,444.44 times bigger than 1,800 bits. This is calculated by knowing that 1 GB equals 8,000,000,000 bits. After thinking about it for a minute, you’ll see, quickly, that humans communicate at VERY slow rates compared to computers. According to research, our speed of communication capabilities are tied to language and how our brains interpret it. Apparently, even when typing at blistering speeds or communicating at 2x speed of around 300 words per minute, ALL humans cap out at a rate of thought around 50 bits per second, which is obviously terrible. This disparity had me thinking though, what if the reason computers take so long to train on language is because they are trained on the way humans communicate through written and spoken text? What if there was a way to communicate at many more bits per second? What if the alternative, whatever it is, could then be used to train a model. Would that mean the compute required to train and create reasoning/intelligent models would be less for the same amount of data, allowing MUCH MORE data to be trained on consumer hardware like my 5090 GPU?

The scaling laws are a thing I have written about lots on this blog over the last few months, essentially stating that the quality and intelligence of a model are directly tied to the amount of data, compute, and parameter count of the model. So if we can train on more data, but require less compute to do so, thereby increasing parameter count without loss, then wouldn’t it be possible to create a model which costs much less and anyone can train? Pulling further on the idea, I came up with a concept for visually representing data that can be processed natively by a model using direct floating point integers. Floating-point vector units are complicated for a non-technical audience to explain, but essentially all you need to know is that a model, such as ChatGPT, is trained on data and it turns that data into a mathematical representation and stores it as lots of vectors. The model then uses those vectors as weights to predict what you are asking for.

Putting it all together, I tested out this new concept I called photic token language, and got some interesting results when I processed different types of data through it. You can try it out here and read the docs for yourself. The problem with PTL is that it tries to do too many things at once. Chiefly, compress data AND make it easier for a human to understand. What I think needs to happen, instead, is a redesign of PTL focusing it only on data compression and contextualization via sentiment and other things versus trying to make it human-readable, too.

the witch of endor

In Michael Scott’s The Secrets of the Immortal Nicholas Flamel, the goddess Hekate places her hands on Sophie Newman and awakens the silver aura that was always latent in her, the magical power she was born with but could never touch, and from that moment Sophie can perceive things she could not perceive before. Then the Witch of Endor, ancient and blind and carrying ten thousand years of accumulated knowledge, takes Sophie’s hands and does something different and stranger. She pours herself into Sophie, memory and sensation, the smell of cities she experienced that burned before history was written, the weight of grief from losses centuries old, and threaded through all of it the living embodied knowledge of the elements, including Air, how it moves, what it costs to shape it, how to call it with intention. Sophie screams. She lives lifetimes in moments. When it is over, she remains herself but only barely, carrying the Witch’s knowledge inside her, and when she works the Air magic afterward, it comes through her silver aura, through her own instincts and values. She is still Sophie, amplified.

Sophie’s experienced is the clearest description I have found of what a capability adapter could actually be like when applied to a model specialized like J.A.R.V.I.S. The Witch of Endor’s transfer to Sophie is like uploading a specialized capability, someone else’s accumulated expertise, distilled and made available to integrate into the model that already knows you, expressed afterward through your own way of reasoning. An engineer with thirty years of systems architecture knowledge encodes their instincts into a software adapter. A chef with a distinct palate distills their technique into a culinary one. A designer with a trained eye creates an aesthetic adapter, a musician their harmonic sensibility, a writer their voice and approach to structure. What if, like the App Store, it was possible to update your model with these skills, only paying once for the capability instead of a compounding per-token cost resulting from essentially renting a general model that does not know you. What comes out is your voice applied to a domain where you previously had no footing. The existing paid AI landscape, all of it, general models, specialist training data, bodies of accumulated human expertise that have been digitized, starts to function as raw material for a marketplace of capabilities you can absorb into yourself, as new training like Sophie, emerging different model capabilities which you can then wield for your own purposes like Tony and J.A.R.V.I.S. Sophie, the model, was still Sophie, trained on all of your data and specific to you. She just also knew how to call the wind.

compilers

Grace Hopper built the first compiler after being told computers could not write programs. Her argument was that the layer between humans and machines was in the wrong place, and moving it would change what was possible. The assumption she replaced was that programmers had to speak machine language. Once that assumption fell, programming became something entirely different, and the compute that followed from it looked nothing like what came before.

Alan Turing had framed the goal of machine intelligence as general imitation of all humanity, and that framing has shaped where compute goes ever since. If the target is every human, you need a model wide enough to cover every human, and that model requires a data center. The scaling hypothesis follows directly from Turing’s framing. A personalized model like the one somewhat described clumsily here is a different argument, the same kind Hopper was making applied one layer up. A model should be specific to one person, not for everyone. The compute required for a single model training on specific data is proportional to one person’s entropy and episodic memory. This is a different abstraction built for a different objective, measured by a metric that does not yet exist formally, but is one I intend to look into, and it is the antithesis of what the current generation of AI companies are building toward. At least that’s my perspective.

I think this is worth building. What do you think?