Building Tony Stark's 'Jarvis' with Claude Fable 5

At a glance

DesignCourse spends six minutes and fourteen seconds showing off a personal AI assistant he built for his studio, a Tony Stark style companion that lives in a browser panel, talks through one of three swappable voice engines, and holds the keys to both his light rig and his businesses. He is blunt that it is only a couple of days old, that he skipped the whole OpenClaw wave entirely, and that Claude Fable 5 was good enough that he simply built his own from scratch instead. The demo is not sanitized: it correctly answers a question about his top YouTube video of the last thirty days, recites its own tool belt in Grok's voice, then fails four times in a row to turn on the DMX stage lights while he swears at it on camera. The most ambitious moment is the last one, where he asks it to build a light show synchronized to the BPM and mood of one of his own songs on YouTube, and the rig starts pulsing. The thesis underneath all of it, stated twice, is that building stopped being the hard part and marketing never stopped being it, which is why this assistant is pointed at his businesses rather than at his code.

The rig, in one picture

Everything below was named in the six minutes. Amber boxes were demonstrated live on camera. Neutral boxes were named out loud, usually by the assistant reciting its own capabilities, but never shown working.

The cold open: he leads with the failure

Most people building a personal Jarvis open on the moment it works. This video opens on the moment it does not. Before a single word of setup, before the title, you hear the assistant say "There. The rig is" and get cut off by its owner:

No, come on. Turn them the [expletive] on. This is not working right.

Got it. I'll blast the rig to full intensity.

That is the entire cold open, roughly ten seconds of a man losing an argument with his own software. It is a deliberate choice, and it sets the tone for everything that follows. The clip is not from some outtake reel either; it is lifted verbatim from the middle of the video, where the same exchange plays out in full and takes four attempts to resolve. He is telling you up front what the honest state of the art looks like in a home built voice agent: the capability is there, the reliability is not.

"We just built our own from scratch essentially"

Then the setup, delivered as if he is showing a friend something on his desk:

Okay, so here, check it out. This is my own Tony Stark AI personal assistant thing that I've been working on.

The origin story is one sentence long and it is the reason this video exists on a channel that normally teaches design and frontend work. He says he worked mainly once Fable 5 was out. And then, in his own words:

You know, I never got onto the open claw Hermes nano claw, you know, bandwagon. But Fable 5 was so like good that I saw I was like, you know, we just built our own from scratch essentially.

That is the pivot the whole project turns on. There was an entire ecosystem of prebuilt personal assistant frameworks to adopt, OpenClaw and the offshoots that trailed it, and he skipped all of it. Not because the frameworks were bad, and he says as much later, but because the model got good enough that assembling your own from scratch stopped being the expensive path. The framework existed to compensate for a model that could not hold a system together on its own. Once that stopped being true, the framework became optional, and what you get in exchange for building it yourself is a rig shaped exactly like your studio rather than a rig shaped like somebody's general purpose template.

He also puts an age on it, twice, and it matters for grading everything that follows:

Mind you, it's only a couple days in, um but it could still really do some cool stuff.

Two days old, and it already owns the studio and the books

Two claims about scope, in order of how much he cares about them. First, the toy: it has full control of his light studio. Second, the point:

But more importantly, it has access to my businesses. And the goal here is to get this thing so good that it can really just help manage my projects. It can generate assets. It can run ad campaigns. It can analyze them. It can do all that stuff.

Then the sentence that is the actual thesis of the video, dropped almost as an aside at the fifty second mark and repeated in the closing minute:

Because let's face it, building is easy now, marketing is not.

Hold onto that, because it explains a design decision that would otherwise look strange. Nothing in this assistant helps him write code. There is no repo integration, no test runner, no deploy hook, nothing pointed at the craft his channel is actually about. Every capability points at the parts of his work that happen after the thing is built: stats, campaigns, assets, analysis. He has aimed his assistant squarely at the bottleneck rather than at the fun part.

The voice layer: three engines behind one switch

The tour starts with the UI, and the first thing he explains is not the brain but the mouth. The assistant does not commit to a single speech provider. There is a selector, and three options behind it:

Um and that is I have a cloud version, which is Open AI's real-time GPT-2. Uh and then I have local, which is actually a free version, but it sounds like crap. And then there's also Grok. Grok has a real-time API as well.

(The captions render the first one as "real-time GPT-2," which is the transcription engine mangling the name of whatever OpenAI realtime model he is pointing at. The relevant fact is the category, a hosted realtime speech model, not the version string.)

This is a more considered piece of architecture than it first appears. Realtime voice is the single most expensive and most latency sensitive part of a setup like this, and it is also the part most likely to change vendor in a six month window. By putting all three behind one toggle in the UI, he can demo the difference live, fall back to the free local engine when he does not want to burn credits, and swap providers without touching whatever sits underneath. Two of the three get exercised on camera in the next three minutes.

Cloud demo: what was my top video in the last thirty days

He picks cloud, turns on voice, and asks a question that is not a party trick. It is a question with a real answer that lives behind an API and that he can independently verify:

Hey, can you tell me what my most popular video in the last 30 days was on the Design Course YouTube channel?

The reply, in the clean hosted voice:

Let me check your channel stats and pull the standout upload from the last month. Your top video was the Claude Fable 5 UI and UX one shots. It absolutely dwarfed everything else this month by a mile.

Three things are worth pulling out of eleven seconds of audio. First, it narrates its own tool use before running it ("let me check your channel stats and pull the standout upload"), which is the small piece of interaction design that keeps a voice agent from feeling frozen while a network call is in flight. Second, it does not just return a title, it returns a comparison, unprompted: it dwarfed everything else by a mile. Somebody wrote a system prompt that asks for the judgment rather than the row. Third, and this is the quiet joke inside the answer, the top video on a design channel that month was itself about Claude Fable 5 doing UI and UX in one shot. The assistant built on Fable 5 is reporting that the audience's favorite video is about Fable 5. His whole channel is being pulled in the same direction as his studio.

His verdict is two words long: "Okay, so as you can see, it's able to pull it up."

The 24 foot wall and the head that reshapes when it talks

The thing behind him in every shot is not a TV. It is a projection screen on a 24 foot wall, and what is on it is his own code:

I've orchestrated a 3JS scene where we have this kind of head model that has interesting shaders on it, and then when it talks and does stuff, it shapes the UI differently back there, which is really fun to, you know, kind of tinker around and play with.

A Three.js scene, a head model, custom shaders, driven by the assistant's state. This is the piece of the build that is purely for the feeling of it, and he does not pretend otherwise. It is "fun to tinker around and play with." But it also does the one job every voice interface needs and most skip: it gives the system a visible state. When you talk to something with no screen, you cannot tell the difference between thinking, listening, and dead. A head that reshapes while it speaks answers that question from across a room, which for a man whose studio is 20 feet from his house is not a small thing.

Worth noticing what this reveals about the builder. He is a design and frontend teacher by trade, so of all the parts of a personal AI stack he could over invest in, he picked the render layer. Somebody from an infrastructure background would have built the same assistant with a terminal and a log tail.

Grok demo: it reads its own tool belt aloud

He flips the switch to Grok and asks the question you would ask a stranger's system:

All right, tell me what your capabilities and tools are here in this project.

The answer is the closest thing this video has to documentation, so here it is in full:

I've got the full live AI toolkit wired in. I track your projects and goals, pull YouTube and business stats, generate on-brand images and videos, run marketing campaigns and content planning, and control the entire studio rig, lights, projector wall, streaming, and synced music light shows.

Five capability groups, delivered in one breath. That list is what Figure 1 is drawn from, and it is worth reading twice because of what it implies about the system prompt. The assistant is not enumerating raw tool names the way a function calling API would. It is describing itself in the register of an employee summarizing a job. "On brand" is the giveaway. There is a brand definition in there somewhere, which means somebody sat down and told this thing what his companies look like before ever wiring an image model to it.

Two more things about the answer. It is compressed to about eleven seconds of speech, which is the right length for a voice reply and a length no default model produces without being asked for it. And it front loads the business capabilities and puts the light rig last, which is the opposite of how the video itself is ordered. He demos the toy and mentions the businesses. The assistant states the businesses and mentions the toy.

The voices, compared out loud

His read on the swap, immediately after:

All right, so as you can see, it it has a different voice. There's different voices you can choose from. It's not quite as clear and pristine as the Open AIs, um but it's still pretty good.

That is the whole comparison, and it is a fair one. The word he reaches for twice about the hosted OpenAI voice is "pristine," and Grok gets "pretty good." The local free engine, by his own grading, is in a different tier entirely. What the toggle buys him is the ability to make that trade per session instead of per project.

The DMX failure, live and uncut

Now the part the video opened with, playing out in full. This is the most valuable ninety seconds in the whole thing, and to his credit he left every second of it in.

He is still on Grok. He asks for something that should be trivial for a system that just told him it controls the entire studio rig:

All right, turn all of the stage lighting and the Hue lights blue.

Done. Blue stage glow, baby.

Except it is not done. It turned the Hue bulbs blue and left the actual stage rig alone, and it reported complete success in a voice with a personality. He corrects it:

No, yeah, you actually need to turn on the actual DMX lights. Take them all blue, not just the Hue lights.

There. The rig is

He does not even let it finish the sentence:

No, come on, turn them the [expletive] on. This is not working right. There's all the almost of the lights are still turned off. Make sure the brightness is 100%.

intensity and reset the look.

All right, finally.

Four turns to execute one instruction. The failure mode is specific and it is worth being precise about, because it is not the failure people expect from a language model. It never misunderstood him. It correctly parsed "blue," it correctly parsed "all the lights," and it produced a confident completion report every single time. What it got wrong was which of several overlapping actuator paths counted as "the lights," and then, once it found the right path, what state counted as "on." Blue at zero brightness is, technically, blue.

The diagnosis: intent, iterations, and system prompts

He does not hand wave the failure. He names the layer it lives in:

All right, finally. So, as you can see, it's kind of frustrating when it comes to the tool calling because it doesn't always do what you expect it to do. And so, that's just a matter of refining the system prompts and making sure that it has access to the different iterations based on what your intent is so that it accurately can, you know, convey and do what you want it to do.

Unpack the middle clause, because it is the actual engineering lesson buried in a throwaway sentence. "Making sure that it has access to the different iterations based on what your intent is" means: when a person says "the lights," there are several distinct things they could mean, and the system needs those variants reachable and disambiguated rather than collapsed into one ambiguous tool. The fix is not a smarter model. The fix is a tool surface that does not force the model to guess which of four fixtures groups the word "lights" refers to, plus a system prompt that tells it what a person in this specific room means by that word.

That is a mundane conclusion and it is also the correct one. It is the same conclusion anyone who has wired an LLM to physical hardware arrives at: the model is rarely the weak link, the boundary between the model and your actuators almost always is. Note also that he is describing a debugging loop with no code in it. Refining a system prompt is the whole remediation plan for a hardware control bug, which is a genuinely strange sentence to be able to write.

The morning he is building toward

Then he stops demoing and describes the finished thing. This passage is the clearest statement of intent in the video, and it is worth reading as a spec rather than as a daydream, because every element in it maps to a component he either has or has named:

So, the ultimate goal for me is like I'm going to wake up, I'll be able to look at my phone and press an on button, I'll get my coffee, and while that's happening, it's going to turn on all my stuff in the studio. My studio is like 20 ft away from the house. And when I walk in that door, I'm going to have uh cameras that are always on that will detect when a person enters the door. It'll fire that as a signal to the AI. I'll be able to see my little AI companion on the screen and it'll give me a breakdown of the stats across my businesses, what has happened in terms of traffic, what has happened in terms of money, etc. etc. etc.

The architectural jump hiding in that paragraph is the one worth naming, because it separates this project from almost every voice assistant demo on YouTube. Everything he showed in the first four minutes is request driven: he speaks, it answers. The morning routine is event driven: a camera fires, and the assistant initiates. Once your assistant can start a conversation, it needs to know what is worth interrupting you for, and that is a much harder product problem than tool calling. It is also, notably, the exact capability that would make the four business tools in Figure 1 useful, since a stats dashboard nobody opens is worth nothing and a stats briefing delivered at the door is worth something.

Build me a light show for a song on YouTube

The last demo is the most ambitious, and he sets it up with a piece of practical caution that tells you he has been on YouTube a while:

Now one really cool thing I can have it do at this point is I can tell it to generate a light show for anything on YouTube. So, any track. Now, I don't want to do any track because I don't want to get demonetized, but I do have my own music and I'm going to see if it works for my own music.

So he uses his own. The request, spelled out letter by letter for the transcription:

I want you to build a light show for my music video on YouTube. The YouTube channel name is called Riffscape. That's r i f f s c a p e and the song is called Primitive. Go.

Nice. I'm queuing up the track and building a sync show for it now.

And then he narrates the pipeline while it runs, which is the only place in the video he explains how any single capability actually works:

So, what it's doing is trying to find the video on the YouTube API and then it's going to analyze the video and then create a light show that is in sync with the BPM and the def essentially the mood of the video.

The music plays. The lights move. And his reaction is the single most honest line in six minutes:

It's like it's working.

Not "it works." Not "as you can see." He is watching his own rig respond to his own song and grading it at roughly seventy percent confidence, in real time, on camera. Anyone who has ever demoed something they finished two days ago knows exactly that tone of voice.

There is a real technical claim underneath, and it deserves to be stated plainly: the assistant is asked to find an arbitrary piece of media by natural language description (a channel name spelled out loud plus a song title), resolve it through the YouTube Data API, derive tempo and mood from it, and turn that into timed DMX cues. That is four distinct capabilities chained with no human in between, and the failure of any one of them produces lights that are simply wrong rather than lights that error out. Given that the same system needed three corrections to turn the rig blue ninety seconds of runtime earlier, "it's like it's working" is a defensible grade.

The closing argument: OpenClaw fell off a cliff, and here is why

The last fifty seconds are not about the build at all. They are an argument, and it is the reason to watch this video even if you never intend to wire an LLM to a light rig.

All right. So, I'm excited to keep on sharing more as this, you know, project develops. And the big question is is, you know, a lot of people OpenClaw really fell off the cliff. There's not as much interest in it. And it's not because it's not capable. It's because most people don't have a really great use case for it.

That is a genuinely interesting diagnosis and he refuses the easy version of it. The easy version is that the tooling was not good enough. His version is that the tooling was fine and the demand was not there, because a general purpose personal agent is a solution looking for a person with enough surface area to need one. Most people do not have twelve things happening at once. They have a job and an inbox.

He does have the surface area, and he says so:

But I have three or four different projects that, you know, we can really test this on.

And then the closer, which restates the line he dropped at fifty seconds and turns it into the whole point of the project:

Because that's the biggest issue right now with the vibe coders. It's like everybody can build, but nobody's making money because the hardest part was never building. Even before AI, the hardest part is marketing. I mean, that's the hardest part. So that's going to be my goal with this.

Read the two halves of that together and the shape of the argument is clear. Personal AI agents lost momentum because most owners had no real work to give them. His answer is not to build a better agent, it is to point one at the specific problem that was hard before AI arrived and stayed hard after: getting anybody to care about what you shipped. The light show and the Three.js head are what got you to watch a six minute video. The businesses are what the thing is for.

He signs off asking you to subscribe and follow the progress of the project, so this is explicitly episode one of a series rather than a finished build log.

Where it stands

An honest ledger, now that the reconstruction is done.

What is genuinely demonstrated: two of three voice backends, a working YouTube stats query with a real verifiable answer, control over two separate lighting systems, a bespoke Three.js visualization driven by assistant state, and a media analysis to light cue pipeline that produces plausible output. For a two day old personal project that is a lot of working surface, and the breadth is the achievement rather than any single piece.

What is claimed rather than shown: every business capability. Project tracking, asset generation, ad campaign execution, campaign analysis, content planning, and streaming control are all recited by the assistant in its own voice and none of them are exercised on camera. That is the gap between what this video argues (an agent aimed at the marketing bottleneck) and what it proves (an agent that runs a light rig and reads a stats API). He is upfront that it is a couple of days in, so this reads as an honest snapshot of an early build rather than an oversell, but the gap is real and worth naming.

What this video is not: a tutorial. There is no code, no config file, no command, no framework named, no repository, and no explanation of what the assistant is actually built on beyond "we built our own from scratch" with Fable 5 in the loop. The one piece of technical detail he offers, that the fix for the DMX failure is refining system prompts and exposing intent specific tool variants, is real but general. If you came here for a build guide you will leave with an architecture sketch and nothing to run.

Where the DMX failure lands: in his favor, and it is the most useful thing in the video. Leaving in four turns of a system confidently reporting success while the lights stay dark is worth more to a viewer than another clean demo, because that specific failure mode (confident completion reports on partial tool execution) is what will actually bite anyone who builds this. He also diagnoses it correctly instead of blaming the model.

The most durable idea here is not the assistant at all. It is the observation that OpenClaw style general agents cooled off because of a missing use case rather than missing capability, and that the use case that survives is the one attached to work you were already failing to get done. That reframe transfers to any agent project, whether or not there is a 24 foot projection wall involved.

Key takeaways

• The model got good enough to skip the framework. He never joined the OpenClaw wave because Claude Fable 5 made building his own from scratch cheaper than adopting somebody else's abstraction. That is a real inflection point, not a preference.
• Two days of work bought a lot of surface. Voice in three flavors, YouTube stats, DMX and Hue control, a Three.js visualization, and a BPM synced light show generator. Breadth arrived fast; reliability did not.
• The failure mode is confident partial execution, not misunderstanding. It never misheard him. It picked the wrong actuator group, then the wrong brightness state, and reported success both times. Without eyes on the rig he would have believed it.
• The fix lives in the tool surface, not the model. His own diagnosis: refine the system prompts and expose intent specific variants so the model does not have to guess which fixtures "the lights" means.
• Put the speech provider behind a switch. Cloud, local, and Grok behind one toggle means he can trade cost against clarity per session and swap vendors without rebuilding anything underneath.
• A visible state matters more than it looks. The shader driven head is decorative until you are 20 feet away and need to know whether the thing is listening, thinking, or dead.
• The real target is event driven, not request driven. The morning routine flips the whole interaction model: a camera fires, the assistant initiates, and the briefing arrives without anyone asking for it.
• General agents cooled off for lack of a use case, not lack of capability. His diagnosis of why OpenClaw lost momentum, and the reason he aimed his own build at businesses rather than at chores.
• Building stopped being the hard part. The line he says twice: everybody can build now and nobody is making money, because the hardest part was always marketing, before AI and after it.

Chapters

• 0:00 Cold open: yelling at the lights
• 0:10 My own Tony Stark assistant, built from scratch
• 0:38 Two days in, and it already runs the studio and the businesses
• 1:12 Three voice engines behind one switch
• 1:38 Cloud demo: what was my top video in the last thirty days
• 2:00 The 24 foot projection wall and the Three.js head
• 2:28 Grok demo: it reads its own tool belt aloud
• 2:58 Comparing the voices
• 3:12 The DMX failure, live and uncut
• 3:44 Why the tool calling misfires, and the system prompt fix
• 4:06 The morning routine: coffee, cameras, and a business briefing
• 4:48 Build me a light show for Primitive on Riffscape
• 5:36 OpenClaw fell off a cliff, and why marketing is the hard part

(The video has no creator set chapters. These timestamps are estimated from position in the transcript across the 6:14 runtime, so treat them as close rather than exact.)

Notable quotes

This is my own Tony Stark AI personal assistant thing that I've been working on. DesignCourse, 0:12

I never got onto the open claw Hermes nano claw, you know, bandwagon. But Fable 5 was so like good that I saw I was like, you know, we just built our own from scratch essentially. DesignCourse, 0:22

Mind you, it's only a couple days in, um but it could still really do some cool stuff. DesignCourse, 0:42

Because let's face it, building is easy now, marketing is not. DesignCourse, 1:02

I have local, which is actually a free version, but it sounds like crap. DesignCourse, on the third voice engine, 1:18

Let me check your channel stats and pull the standout upload from the last month. Your top video was the Claude Fable 5 UI and UX one shots. It absolutely dwarfed everything else this month by a mile. The assistant, in the OpenAI realtime voice, 1:46

I've orchestrated a 3JS scene where we have this kind of head model that has interesting shaders on it, and then when it talks and does stuff, it shapes the UI differently back there. DesignCourse, on the 24 foot projection wall, 2:06

I've got the full live AI toolkit wired in. I track your projects and goals, pull YouTube and business stats, generate on-brand images and videos, run marketing campaigns and content planning, and control the entire studio rig, lights, projector wall, streaming, and synced music light shows. The assistant, in Grok's voice, 2:34

Done. Blue stage glow, baby. The assistant, reporting success on a job it had not done, 3:16

No, come on, turn them the [expletive] on. This is not working right. There's all the almost of the lights are still turned off. Make sure the brightness is 100%. DesignCourse, on the third correction, 3:28

It's kind of frustrating when it comes to the tool calling because it doesn't always do what you expect it to do. DesignCourse, 3:46

That's just a matter of refining the system prompts and making sure that it has access to the different iterations based on what your intent is. DesignCourse, diagnosing the failure, 3:52

I'm going to have cameras that are always on that will detect when a person enters the door. It'll fire that as a signal to the AI. DesignCourse, 4:22

I don't want to do any track because I don't want to get demonetized, but I do have my own music. DesignCourse, 4:54

It's like it's working. DesignCourse, watching his own light show run, 5:28

A lot of people OpenClaw really fell off the cliff. There's not as much interest in it. And it's not because it's not capable. It's because most people don't have a really great use case for it. DesignCourse, 5:40

Everybody can build, but nobody's making money because the hardest part was never building. Even before AI, the hardest part is marketing. DesignCourse, 5:58

Resources mentioned

• DesignCourse on YouTube is the channel this comes from, and the channel whose stats the assistant queries live on camera. Its uploads include the "Claude Fable 5 UI and UX one shots" video that the assistant names as the top performer of the last thirty days.
• Claude Fable 5 is the model he credits for making a from scratch build worth doing at all, and the model whose UI capabilities are the subject of his own top video that month.
• OpenClaw is the personal agent project he says he never got on the bandwagon for, and whose loss of momentum he diagnoses in the closing minute. He also name checks "Hermes" and "nano claw" in the same breath, the wider wave of offshoots he skipped along with it; the Hermes reference is most likely Nous Research's Hermes Agent.
• OpenAI realtime is the hosted speech model behind the "cloud" setting, the one he calls clear and pristine. The captions garble the model name; the category is what matters.
• Grok and the xAI API provide the second realtime voice path, used for the capabilities question and for the entire lighting argument.
• Three.js drives the 24 foot projection wall: a head model with custom shaders that reshapes as the assistant speaks.
• Philips Hue is the smart bulb system that turned blue correctly on the first request.
• DMX512 is the stage lighting control protocol behind the fixtures that took three corrections to come on, and the target of the light show cue list.
• YouTube Data API is what he says the assistant hits to resolve a song by channel name and title before analyzing it.
• Riffscape is his own music channel, spelled out loud on camera. The track used for the light show demo is "Primitive," and he picks his own music specifically to avoid a demonetization strike on somebody else's.
• Zubair Trabzada building the same idea with Claude Fable 5 is the companion piece on this site: a different builder, the same model, a Jarvis pointed at a personal knowledge vault instead of a light rig and a set of businesses. Reading them side by side is the cheapest way to see which parts of a personal assistant are converging and which are just the builder's own taste.