The machine in my writing
Article · 10 min readHamid Jafari-Zadeh
For years, many apps claimed to be AI chatbots. All of them were dumb! Then in 2023, my friend Diego sent me a link to ChatGPT and said this one is different. I was updating my CV at the time. “This AI chatbot is great! It can really help with your CV too,” he told me.
I knew Diego’s intelligence. Otherwise my first reaction would have been “ah, another Shitbot!” So I took my time to see what it was.
First prompt: hmmm. Second: interesting. Third: this looks like something… I went on from 7 pm until 1 pm the next day, with small breaks whenever I had to wait for the limit to reset! I talked with it about everything: IT commands, scenarios, projects, architecture, my CV, psychology, philosophy, the future, social science… At the end I was like: wow, this is something else, and the world is going to change real fast.
Ever since, I’ve used it daily, more and more every day. Since that night I’ve also known it’s going to replace many human workers, especially in anything that needs digital interaction. For years, technology replaced workers in other sectors. Now it is time for IT people!
I’m a technical generalist and I like building. I don’t like to do things twice! That’s why I’ve automated things wherever I worked. With AI I started building tools that do what I do, adding something new to my toolbox almost every day.
I’ve spent years around production systems and incidents. When something crashes and millions of users feel it, you fix it, and then you need a postmortem. That means reading thousands of lines of logs, runbooks and timelines. Hours of human work. In my sandbox, a production-like setup where I break things on purpose, I gave the logs and the runbook to an LLM, and the postmortem draft was there in minutes.
But the probabilistic nature of LLMs makes them hard to trust, especially in production and sensitive environments. Since then, my R&D has been around visibility, trust and governance (SEED comes from there), and I’ve built many tools that automate my daily life.
The sound of AI
As helpful as it is, one thing still sucks: the sound of AI when it writes. In the past couple of years I’ve been researching, reading and reading: papers I review for friends and for the journal Cognitive Systems Research (ORCID), people’s posts on LinkedIn, social media… Everyone is using it. Even me, right now, in this very post!
When I give AI a task, I’m satisfied almost 100% of the time. With writing, I’m not. When people write themselves, it comes out of their own brain, and it’s kind of unique: the tone, the structure. They go back and forth fixing grammar and typos. Give the same text to AI and the outcome is often quite great, especially with the newer models. But the AI traits are still visible, and the voice of the machine is all over it. So in many cases the writing doesn’t feel authentic. I’ve written things, looked back at them as a reviewer, and didn’t feel it was me. It didn’t sound like me.
For example, one of my first automations is Communication Agent. I designed it in 2024 as a customer-support agent, as a consultant helping one of my first clients with a proof of concept. Along the way I turned it into my own email assistant. It fetches my unread emails, turns them into tickets, and drafts a response based on earlier conversations and the resources I gave it. All I need to do is read the draft and approve it, and it goes out automatically.
I have almost never approved one because they never sound like me!
It hit me harder a few weeks ago, when I needed to write an email and a proposal for a project. I had a lot of material: documents about the same product, my conversations with other customers, my CVs and much more. I asked an AI to draft it, and the outcome was outstanding. A good start, reasoning, a middle section, an end. But there was one big problem. It didn’t sound like me! It was AI-ish.
I listed the problems and asked it to make those parts human instead of machine-like, to make them sound like me, like what I had written before. I had never counted how many times I go through a text, but this time I did, because I wanted no sign of AI in it. It took eight rounds of back and forth. At some point I wrote some sections myself, and the AI just fixed the typos. And that made it authentic!
Nowadays you see posts from so many people, and something is common. You don’t see them, you see AI!
I took a bunch of texts people wrote on LinkedIn and asked Gemini to find the patterns of the AI sound. This part is fully generated by Gemini, shortened:
At a foundational level, AI text generation is driven by probabilistic models optimized to predict the next most statistically likely token. Furthermore, these models undergo Reinforcement Learning from Human Feedback (RLHF) to prioritize helpfulness, safety, and clarity.This combination of probability and alignment creates a specific, detectable fingerprint in the resulting text. When analyzing a document to determine AI involvement, the indicators generally fall into four distinct layers: statistical, structural, lexical, and semantic.
- The “Sandwich” Framework: AI almost always structures text with a polite, contextualizing introduction, a structured body, and a summarizing conclusion. It struggles to start in the middle of a thought or end abruptly.
- Transitional Crutches: AI uses formal transitions to glue its logically discrete thoughts together. Overused examples include: Furthermore, moreover, additionally, conversely, it is important to note, notably, subsequently.
- The “God Trick” (Omniscient Neutrality): The text lacks an anchor in physical, lived reality. A human writing a Jira ticket will reference specific, messy context (“the legacy database crashed again because Dave’s script ran late”). AI abstracts this into sanitized generalities (“System instability occurred due to timing conflicts in legacy chron jobs”).
A good answer, and a good example too! An introduction, bullets and a summary at the end (I cut it). Its second sentence starts with “Furthermore”, the first word on its own list of crutches.
I’ve been aware of this for the past few years. Still, how much I care whether someone sees the AI depends on the purpose. A research paper has to be the person, because it’s purely their own thinking, research and reasoning. The same for a proposal, and for many other things. For simple communication, I don’t care that much.
Why I built Writing Agent
But it’s been on my mind for quite some time (questions don’t leave my head easily, some more than others), and it’s one of the reasons I built Writing Agent.
Back then, my main argument wasn’t the quality of the text. It was the pipeline. Writing should be a separate part of the process, its own step, instead of the last thing the same LLM does inside the agent. Separation of concerns. Especially in RAG systems: they gather information from many places, and context can get lost in between. The system forgets, or even hallucinates. A writer should be there to plan, check against the requirements and the facts, and miss nothing. It should also keep the output almost the same every time, in structure, tone and so on, because being reproducible is what makes a system trustworthy. Almost any AI loses that after a few prompts, and then you have to keep reminding it: keep it short, keep it this way, make it that way!
When I open-sourced it last November, the goal was a production-grade system: clean code and an architecture with separation of concerns, so if someone liked it, they could easily get into the AI logic and change it themselves. I also wanted the production side and the visibility to be there: health, ready and metrics endpoints, rate limits, a runbook. I announced it on LinkedIn with a post I prepared with Writing Agent itself. I believe it’s a good post. But as you can see, the AI is right there: “I’m excited to open-source”, a list of 🔹 bullets, “principles that scale with trust”.
Writing Agent plans, drafts, lets a critic score the draft, fixes it, polishes it and scores it again. You pick an audience and a situation, and it runs with any local model on Ollama or can be plugged into an external API. The writing side is still basic. It has a scoring system, not an evaluation pipeline with a golden set of good examples for each audience, situation or preset. The system prompts are basic too, and the UI is a single static page of HTML and JavaScript. Its first purpose was never to mimic me. But that last proposal pushed me to dig into the machine voice, so I tested it on three texts.
Three tests
First run: 653 milliseconds. Way too fast. The app was running in Docker and couldn’t reach Ollama on my Mac, so it silently fell back to simple rule-based edits. The punctuation came out broken, and the critic said clarity went up by two points. The
/ready endpoint knew the model was missing, so the visibility I wanted for production was there. But only for whoever checks /ready. The screen said nothing to the person using it.Second run, with the model connected, on a short paragraph that already sounded like AI. Every AI sign survived, and it even added “for real-world effectiveness”, which nobody says out loud. Its own critic gave the input a 3 for clarity and the final text a 1. The pipeline returned the final text anyway.
Third run, the good one: a quick Slack message full of typical non-native mistakes.
i am not agree with deploy in friday evening. last time we done it, the on-call guy was fixing it all his weekend. still we dont have any rollback plan. lets doing it tuesday morning when everybody are here and can to watch it.
Audience: manager. Situation: disagreement. The result:
We should not deploy on Friday evening. Last time, the on-call person had to fix issues over their weekend without a rollback plan in place. Let’s schedule it for Tuesday morning when everyone is present and can monitor the deployment.
A good rewrite. But the main argument was that we still have no rollback plan, and the rewrite moved it into last weekend’s story.
To take a screenshot for this post, I sent the exact same message again, with the same audience and situation. The answer came back different:
We should deploy on Tuesday morning instead of Friday evening. Last time we deployed, the on-call person had to work through their entire weekend. We still don’t have a rollback plan in place. It would be better to do it when everyone is available and can monitor the deployment.
This time the rollback plan stayed where it belongs. But the same input gave two different answers, and only one of them kept the facts. That’s the reproducibility I wrote about above. The critic didn’t change its mind, though: in both runs it gave the broken message a 5 for clarity and the rewrite a 4. The length target was 45% of the input, and the final text came out longer than the input. The fix step was the only one that hit the length, and its clarity dropped to 2. Then the polish step undid it.
What it needs next
So the next version needs to say it loudly when the model isn’t there, never return a text that scores worse than the input, check that the facts are the same before and after, and give the same answer to the same input. The scoring needs to become a real evaluation: a golden set of good examples for each audience and situation, and a check for the AI signs, not only clarity, tone and length. And the big one: learn my voice from texts I actually wrote. Right now it only has knobs, like “warmth: balanced”.
My vision for it is to become as close to me as possible. I give it my raw writing, it reasons and understands, and the final text is me! Without me going through it eight times. One round is acceptable. Two is a lot. Three is too much!