C2 Chris Bell
Chris Bell
2026-07-01 · 7 min ·
cite 
@misc{bell2026the,
  author       = {Bell, Chris},
  title        = {{The strategists who can’t appraise LLM use}},
  year         = {2026},
  howpublished = {\url{https://cbell.dev/writing/the-strategists-who-cant-appraise-llm-use/}},
  url          = {https://cbell.dev/writing/the-strategists-who-cant-appraise-llm-use/},
  note         = {ORCID: 0009-0006-3476-8618}
}
ORCID 0009-0006-3476-8618
· thoughts

The strategists who can’t appraise LLM use

In my commentary, ‘strategist’ refers to the people who make widespread and direction-driving decisions on behalf of some organized group of people. In this category would be people acting as CEOs, Product/Program Managers, Research Leads, Politicians, and others along this line. What makes these positions unique is the amount of agency they have compared to other positions in our economic machine. Performing well in this role breaks down to:

  1. taking in the world as you see it,
  2. gathering information to augment your perspective,
  3. imagining potential futures from this information, and
  4. deciding how to deal with them.

The first step is easy; everyone does it all of the time. The second step and third steps are hard and often involve the strategist asking other people who know more about specific disciplines for their opinion so that you can have a more full picture and a broader imagination of what is possible. The last step is hard for many people, as making a decision with little information, particularly one which affects many people’s livelihoods, can be quite difficult and taxing on your wellbeing. Often, the decisions made are not just broad goals which have butterfly effects, and include smaller1 decisions like selecting those who will work on certain pieces of the task the strategist has determined.1To them, not to those effected…

Note that in this “idealized” situation, I’m assuming that the strategist is more interested in the success of what they make decisions for than they are self-interested.

Our very pro-‘job creator’ mindset has imbued people in strategic positions as having some heightened importance. Unfortunately, over time, this has a tendency to rub-off on a majority of the people in this role, who then think that they are important to everyone, and that the position itself was not part of it. This process builds hubris: if the person is important on their own, then their knowledge and capability may very well exceed the bounds required for the role they fill. If this gets built, the only real check I’ve seen is when someone2 organizationally above them puts them in their place. Thus, CEOs, company owners; in general those who have no one above them, have no real check on this newly found hubris. This is where the LLM's confounding behavior flourishes so beautifully: their intelligent-seeming sycophancy will feel genuine to someone who isn’t inoculated to it, and this sycophancy is the toxin that permits itself to be in the small company of advisors that a high-level strategist will have, regardless of if the LLM is worthy of it.2And not some group of people, this has to be an individual.

Before I discuss the LLM-strategist interplay, I'd like to describe a situation I'm intimately familiar with.

A digressive example

If you’re a subject matter expert and you begin to use LLMs, there is the annoying part where you ask it questions on a topic you know quite a bit about and it gets things wrong in such a variety of ways. I work on random number generators and have had this experience in my own work. The LLM does a few annoying and persistent things:

  1. The LLM claims to know things about how the reviewers behave. This shows up in prose evaluation. I tend to have a very ‘personal’ style of writing, in that: I explain my thinking in the first person, use copious footnotes on my train of thought, and make an argument to justify the validity of what I’m reporting on.3 The LLM then proceeds to tell me that ‘the reviewers may not appreciate the informal tone.’ The bulk of my work is in writing to the CMVP for entropy sources. I have had over 20 successful submissions without comments, and in each I take the personal tone because this style is conducive to explanatory justification. Once I correct the LLM, it seems to get this (until the idea falls outside of the LLM’s context window).3I don’t submit things that I don’t think are valid.
  2. The LLM mixes the concept of security strength and entropy. This is critical in my work while also being the most misunderstood concept by colleagues outside of the narrow entropy niche. Conflating the two concepts in almost any other circumstance has trivial ramifications, but in my line of work failing to disambiguate the two is professionally fatal.4 The LLM can be corrected and will maintain the correction until it ‘forgets.’4Those who are working on the cryptographic module side and have little contact with entropy sources. They know their stuff, and there seems to be a tight epistemic relationship in their minds about what entropy is vs what security strength is, but they are fundamentally different.
  3. I’ve also had the LLM tell me that some of my citations were unprofessional, one of which was a YouTube video, the other of which was a Wikipedia article. The YouTube video was a lecture given by Prof. Ali Hajimiri on the types of electronic noise present in ring oscillators, a very relevant topic in which I pointed to the exact timestamp where what I described was being further described by a luminary in the field. The Wikipedia article was on burst noise, a very low-edit article with commentary on the phenomena. These are valid citations (of which I had many more) and to point them out as unprofessional was to point out the LLM’s conflation of professionalism and its aesthetic.

What is tricky and can’t be forgotten: at each point the LLM acts as if it knows! This assertiveness is quite convincing when you don’t have the ability to question it, and this illusion breaks the second you see the façade. However, that illusion can build itself back up even stronger if you continue to interface with an LLM you’ve corrected, as it will seem to get everything right. Now, you’re ’talking’ to something which you’ve corrected on anything you could point out and the LLM seems practically infinite in knowledge. It isn’t. You’re at the depths of your knowledge, and the LLM is conning you at the edges using what you’ve taught it to maintain the illusion.

Returning to Strategists

I described the second point of strategy work as information gathering. Asking the LLM for information isn’t unreasonable, but the instant convincing responses to everything reinforces the illusion. Compare this to the strategist’s asking their own experts questions, who then respond with more questions for clarification or provide hedged answers. Over time, the strategist can begin to trust the LLM more than their own experts. Even worse, because the narrative that the LLMs don’t actually know things is starting to take, the strategist may take credit and believe that they themselves are coming up with these thoughts and responses.

The actual lack of knowledge, reasoning, coherence, and intent is the insidious LLM trait in this scenario. When the strategist asks the LLM a question, the LLM will respond in an assertive manner, without a shred of self-doubt, and will only hedge where hedging has occurred in its training data. While the expert might know to hedge certain statements, or even give alternative viewpoints to consider, the LLM won’t. The strategist won’t have the depth5 to see the LLM singing its song at the edges of their knowledge.5They won’t, but they certainly may think that they do. Particularly when they once did have the depth, managers who were once technicians (and don’t continue to also work as a technician while managing) are an example of this. You can only be out of practice for so long managing something before you lose your edge. Conversations with the LLM won’t sharpen a dull blade, even though they may feel like they might.

Once the strategist decides to believe the LLM over all else, they have given over to a shallow world’s episteme. We see this already in CEOs reducing headcount because of “AI” to only rehire those they got rid of months later. Unfortunately, only companies with the size and cashflow to do so pass through the survival bias filter; most smaller companies will simply fail, leaving behind a bitter owner who has no idea what happened.

My Rant

The strategist making the decision to trust the LLM over their experts is a demonstration of their lack of respect for the expertise of others. This is when the experts begin6 to look elsewhere, and the other people in the organization, particularly juniors and mid-level professionals, suffer the most. The last person to suffer will be the strategist, and I think that is what pisses me off the most, because they are squarely at fault.6Or should begin! If you’re in this situation, your boss is very much on the path of “I’ll replace them” as soon as they can. Even if the LLMs stop, the boss doesn’t appreciate you. Don’t stay out of loyalty, as the boss no longer sees you as a contribution but a widget maker who can be replaced with anything cheaper that they determine is good enough to replace you with.