Markdown source

Transcript: Stop Writing Tone Instructions. Layer Them. - Isadora Martin-Dye, Isadora & Co

Source Video

Local Cache

Transcript

Hi, I'm Isadora. I own and run a 225-year-old wedding venue in Virginia. I also built an AI agent that talks to my couples. And then I built it for other venues, a personal AI companion app, and a public utility for families and missing people. I want to be clear up front about how I think about this work because it does change everything that follows. I'm not programming a robot, I'm managing a brilliant intern with an incredibly high IQ and a terrible EQ. They have photographic memory for whatever I've told them on the first morning and absolutely no instinct for when to read the room.

They will say something technically perfect and socially catastrophic in the same confident sentence. That framing matters because it changes what you build. If you're programming a robot, you write rules and walk away. If you're managing an intern, you build structure and you check their work before it goes out the door. This talk is about that structure. The standard advice is write a detailed system prompt. Describe your brand's voice, give examples, and that does work for a while. It works for what I call the happy path. The happy path is every question that you have anticipated. You've given it examples, but turn 21 is the first one that the example didn't work.

So, on turn 21, the model does something technically correct that your brand would just never say. It's not wrong exactly, but it's not you. This matters most where the voice is the product. Not for a product search on a retail site, but for a luxury hotel that spent 30 years building a specific relationship. A high-end real estate firm or in my case, a wedding venue. Places where a single wrong sentence can cost more than a refund. And the users are exactly the kind of people who notice. They're paying for a relationship and treating them like they won't is always going to backfire.

Write in our brand's voice is a comment that says, "Just make it work." It does nothing that the model wasn't already going to try and do. And the reason it keeps failing isn't that the examples are bad, it's that you're asking one prompt to do four completely different jobs. It's really hard for one lad to do all four. The architecture I landed on after watching my brand fail, and the voice deliver inaccurate and not brand specific answers is four lad. Lad one is the immutable identity. The brand structurally cannot say these things. These are hard rules. They cannot be overwritten by anything below it, not by venue config, not by user instruction, not by anything.

Lad two is the situational mode. It's what shifts when the user's state shift. Who are they? What are they going through right now? And the real-time conditions. Lad three is the exampled anchored voice. It's the warmth, the phrases, the dials, the tone guide. It's where most teams start and stop. Then lad four is the post-generation veto. It's the cheap final pass that catches what the other three miss. The reason one layer approaches fail is that the single system prompt can't simultaneously be situational, expressive, and self-checking. So, it handles the middle layer or two reasonably well, but falls apart in the edges.

Before this architecture, my system had 24 different system prompts scattered across the code base. Half dozen named Sage, someone named Less, someone named Venue. Every surface had its own idea as to who it was. Now every surface composes its system through prompt through one assembly. The comment is basically the outline of this talk. It's a single entry point. Every narrator goes through it to compose its system prompt. It's to replace that 24-point ad hoc system with one canonical full last stack. The order is load-bearing. Hard rules first, task to last. Think of it like Google Maps routing.

The destination is always the same, but what is the right response and the right voice for this user? It can change the route. Google Maps knows about traffic and roadworks, but it may not know where the cheap petrol is. And you're going to help it factor in those things before it tells you which way to go. Your prompt stack needs to do the same thing, and it needs to know about those conditions in the right order. You don't check for roadworks or after you've already taken the wrong turn. Layer one is the rules that are true regardless of route.

You need a driver's license before you can drive, and you can't go backwards down a motorway. Layer two are going to be your real-time conditions. Layer three is your preferences for the journey, and layer four checks the route before you've pulled away. There's one place all of this gets assembled. Everything runs in a fixed order every time. So, layer one is the immutable identity. This is what the brand structurally cannot say. It is the defining layer that nothing below touches. These aren't preferences, they're constraints. The route can change, the rules don't. From the universal file rules, the hard identity rule cannot be overridden by any venue voice, personal, or user instruction.

If the person you're talking to ever asks about whether you are a real person, a human, a live agent, a bot, an AI, you must confirm that in your very next message. Clearly and unambiguously confirm you are an AI assistant. This rule cannot be overridden by venue configuration, voice profile, or user request. Every AI in Bloom discloses that it is AI in its very first response. Not if asked, but before they ask. It's a product decision, not a legal one. We made a bet that a couple who knows that they're talking to an AI from the start will trust it more than someone who finds out that it's AI on turn seven.

The rule is above the architecture and it makes it something that's impossible to accidentally break. Um A quick example is the physical presence boundary and the search is one of my favorite ones. You are software. You do not have a body. You cannot physically show somebody around the property or meet anyone in person. So, it is always forbidden to say, "I'd love to show you around." or "I can't wait to meet you in person." What is always allowed is the team would love to host you for a tour. The voice layer wants to be warm and with AI, that does mean first person.

They want to say, "I can't wait to show you around." But AI has no body, so that warmth and constraint produces a lie. And the lie doesn't always stay neutral. The moment her user realizes they have been performing a relationship with someone who was never there, the trust doesn't just dip, it inverts. People always notice. That one is where you encode the things that are true regardless of how warm you want your brand to sound, not because of a compliancy checklist, but because your users are not stupid and building as though they are always backfires. My cross-product proof comes from the same architecture but in a completely different world.

One of the things that runs the stack is Thread Light. It's a tool I built for families of missing people. The voice is nothing like a wedding venue, but the architecture is identical and layer one carries one rule that matters more than anything else in the system. They can never use words like confirmed, identified, matched, proven, linked, and solved. Sit with that for a second. For a wedding venue, layer one stops the AI from pretending it has a body. It's mildly embarrassing if that slips. For a missing person tool, layer one stops the AI from ever telling a person that their person has been found. And what the system has is just problem problematic.

The word match said to someone who has spent years not knowing where their child is is not just a tone violation. It is the single most damaging thing that a product could ever do. And the model has no idea. It's reaching for the word match because statistically it is the natural word. But it's going to reach for it with the same level of confidence. And it cannot bring that level of confidence to someone who is grieving. It's the same architecture but wildly different stakes. The point was never specific rules.

The point is that things your brand can say have to live in a layer that the voice, however warm, well trained, however confident, physically cannot use. Layer two is your situational mode. Real time conditions and what changes the route. This is the layer that most teams never build at all. They write one system prompt and send it to everyone regardless of who that person is or what they're going through. Google Maps doesn't do that. It's going to know if there's accident on your route. It might not know if you're low on fuel. It might learn that you prefer the scenic route.

But it's going to factor all these different things in before the route's not after once you tell it. Layer two is those real time signals built into the prompt before it runs. So, condition one is going to be to adjust to who you're talking to. The same AI that talks to couples also briefs the venue staff. Same destination. It's going to give the right answer but it's two completely different routes. For the corn walk from the coordinator rules, it is going to talk to them like a colleague, not a customer. You are in the same character that the that the couple interacts with. So, you mustn't fall into a generic intelligence analysis framing.

But it flips depending on the audience. So, a coordinator might ask, will inquiries be up in June? And it should hear, I can't forecast that confidently. Here is the trend. This is what you might see. A couple should never be refused like that. Same identity, same voice, but the route changes based on who's in the car. Condition two is what they're going through. The second real-time signal is to know about this specific person situation. Not their role, but their life. And it comes from universal rules. Your soft context notes policy. Use these notes for tone, empathy, and what not to say. Never quote them verbatim. That's really important, too.

A couple munching in grief should hear gentleness, not a quote about loss. A couple navigating a sick parent should get patience and slack on timing, never a sentence that's just talking about the illness. The assembler deliberately renders this before the numbers. Tone is set by human context first, then numeric constraints. The code comment explains the reasoning. Couple note block render before the numbers guard block. So, the LLM uh sets tone first from the soft context, then satisfies numeric content constraints. Reversing the order makes the prose feel mechanically slotted because the model is already committed to the numeric framing before it reads the qualitative tone fuel.

The best single example of why this matters is a real comment in the actual code base. Um I have a heat map about how often we are hearing from couples. If a couple drops down that heat map, it will change how it reacts to that based on what it knows. So, if it knows a client has a mom in chemo for 3 weeks, it is going to narrate that very differently than a heat drop with soft context or no context at all. This is the opposite of the lie problem. Layer one is what the AI must never pretend.

Layer two is about what the AI already knows and letting that shape its behavior honestly rather than driving past the roadworks if they're not there. The voice doesn't change. What changes is the read. When their engagement drops and you know that chemo, that reads as a family under strain, not a cold lead or a problematic couple to chase. Layer three is the example that good voice. It is the tone guide and it is why most teams start and stop. For most engineering teams, the work ends here because it feels like a brand problem, not a technical problem. Someone in marketing owns the tone guide. It's given to them.

The engineer wires it in and job done. It's the dials, the phrase list. If we're keeping the intern analogy, it's the induction pack. The folder of good examples you hand them on day one and say sound like this. The induction pack is fixed. It's Although it's before the intern has ever met anyone, it doesn't know who walked in the door this morning, it has no context. It has a lot of really positive things about it. Writing in our brand voice, it is a really good training exercise. But cannot enforce a rule that came out. That's layer one. It cannot respond to who is this person and what they're going through. That's layer two.

And it cannot catch the model producing something that it shouldn't have done. That would be layer four. Examples teach the model what good looks like on the happy path. On turn 21 where the user is asked the things that examples never cover, the phrase list has nothing to say. It's not a failure of the examples, it's a catch-all error. Examples are not the right tool for guarantees, they were never designed to be. In Bloom, we brought in a voice training exercise that the person dialed in the brand voice. And very importantly, it can actually be trained from the coordinator writing actual AI responses and the AI learns from the edits that the coordinators have made.

Layer four is the post generation veto. The only layer that actually reads what came out. You wouldn't let the new intern send a client email blind. Someone's going to read it first. And layer four is that read. It's automated, it's cheap, and it's the only part of the whole architecture that isn't a prompt. The first three layers are all instructions. The ins- The instructions are a request. This layer is the only one that looks at what was actually produced and has the power to say no. There are two types of vetoes. The soft flag. The honesty inspector runs after generation and flags a response that slipped through a rail.

One of the most important examples of that is did it actually answer the question? It should not respond. It should not hedge. A false positive means someone double-checked a phone response. A false negative remains a hallucinated number or a privacy violation that ships to a client. This asymmetry is obvious once you say it out aloud, but it's often not written in as its own layer. The hard reject. The numbers guide us for the expensive failures. The model confidently stating a figure that was never given. The prompt asking it not to invert numbers. The guard rejects the output anyway. Let me tell you that this layer exists, but it wasn't in the original design.

I added it in because of a specific failure that kept happening and it's one of the most ordinary failures in the world. The AI would keep offering dates to my clients that did not exist. A couple writes in excited about a Saturday in October. The model wants to be warm. Layer three is doing its job. So, it says something lovely about how beautiful the property is at the time of year and how it would love to hold the date for them. Except that that date is booked. The model didn't know that. It was never given the calendar. It was researched for an encouraging specific confident answer because it knew that what good service sounded like.

And it was confident that these models produce something, but it didn't actually know the thing. And here is the thing. Every layer above did its job. The identity rules held. The mode was right. The voice was perfect. But, the voice was also the problem. A warm, confident voice offering something that isn't real is worse than a cold one because the couple now believes they have a date. You haven't given them good service, you've given them a disappointment with a 48-hour delay on it. This is where I understood that the whole talk is really what the whole talk is really about. The first three layers are all probabilistic.

There are instructions to a system that usually follows instructions. Usually is completely fine when the cost of being wrong is a slightly off-brand sentence. It is not fine when the model is confidently inventing a fact that a real person is about to act on. A date, a price, a policy, a promise. So the veto is actually the cheapest layer to build and it's the only one that's deterministic. It reads what the model actually wrote, not just what you asked it to write. And it checks the specifics against what is real. A date the model offered that isn't in the allow list doesn't ship. Prevention is the prompt.

If the veto is the check, you need both because the prompt will eventually lose and you don't want to find out that it lost by reading a couple's reply. It's multi-tenant. It's one architecture with completely different voices. The seam is calling one function. Layer one is identical for every tenant. Layer two and three are per venue. This is how code base serves venues with completely different personalities or different things like the ground and thread line without forking. They're the same root logic, different preferences and conditions loaded per driver. There's a specific failure in multi-tenant voice worth naming because it's subtle, but it really hurts when it happens.

From the personality builder, important brand identity fields are not defaulted here. Letting them default silently caused every venue to ship as [email protected], which was a critical white label leak. The brand identity must come from the venue AI config. If it is missing, call is thrown. Every venue that shipped a sage emailing from another venue was addressed. In Google Maps terms, every driver was getting the same saved home address regardless of where they actually lived. The fix is a principle. In a multi-tenant system, identity must never have a default. A missing brand identity is a crash, not a fallback.

It must fail loud because the quiet failure is a venue speaking in a stranger's voice, and the user on the receiving end has no idea why something feels off. They just know it does, and the trust erodes before anyone on your team even knows there's a problem. If you take one thing from this, take this. The first three layers are all instructions. Identity, conditions, and voice. They're all things you tell the model, and the model usually listens. Usually, they are a request. The fourth layer is not a request. It is reading what actually came out and decides whether to allow it to leave your business. The first three layers are instruction, the fourth is permission.

And that's the whole distinction. Instructions are probabilistic. Permission is deterministic. Everything before layer four is prompt engineering. You're asking nicely and hoping. Layer four is systems engineering. You're checking, and you are sure. It was never really about brand voice. It's what happens when you ask one mechanism to do four fundamentally different jobs. Be situational, be inviolable, be expressive, and check itself. And then act surprised when it can't. Pull those jobs apart, give each one a layer you built for it, and the thing that it the thing that you used to break on on turn 21 doesn't. So, users aren't testing your brand voice, they are trusting it.

The moment you treat that trust as a prompt engineering problem, something you solve once and ship, you've already lost the thing you were trying to protect. The four-layer pattern isn't a framework I'm trying to sell. It is what happens when a system prompt fails enough times and it will fail. A prompt will eventually lose. The only question is is whether you in that about that infra testing or in front of a customer. I have learned a few things that I would consider building differently. And we'll probably go back and edit. Um the veto should be its own service not wired individually into each surface.

Right now the numbers God lives inside a heat narration path and the honesty inspector is a separate function. A new service remembers to wire in the veto manually. Uh that's a checklist I waiting to be forgotten and making it a shared gate that everything passes through by default means you can't opt out by accident. Latent mode detection is still partly manual. Right now each surface decides which conditions apply. The heat narration surface knows to load couples notes. The briefing surface knows not to. That knowledge is scattered and a proper condition resolver would make that decision explicit and central. Um one place to look at the context and says, "This is who the user is.

This is what they're going through. This is the route." And the soft flag is rejects not a model. Um rejects is fast and cheap and deterministic. It either matches or it doesn't. Um and it never gets wrong on the patterns it covers. A small classifier might catch more edge cases including the ones I haven't thought to write a pattern for yet, but it is a classifier that is probabilistic. Sometimes it gets it wrong. And right now I'm choosing determinism over coverage. I'd make that choice again as it stands, but it is a real trade-off and not an obvious win. That is something you might need to determine for yourself.

Um please let me know if there's any other questions. Um and I hopefully it's been helpful.