If You Can't Beat 'Em

I have been going through AI training the past few days. While it's also going on at work, I've been doing some stuff on the side for the better part of a week in addition to whatever I have to take for my employer. 

For the record, I'm not going to comment on anything that goes on at work, as that's neither the point of the blog nor is it appropriate for discussion in a social context. Therefore, I'm going to stick with what I've discovered during my personal explorations and learning while on my home PC, typically late in the evening.

For starters, here's the first thing I asked of OpenAI's ChatGPT when I got the chance:

This is OpenAI's ChatGPT's response to my prompt:
"what is the parallel context blog about". Interesting that
TTRPGs got top billing, but not bad. Definitely
better than an average Google search for PC.

(You knew that was coming, right? I mean, if Google can't find PC, what does ChatGPT say? Just as long as "What does the ChatGPT say?" doesn't end in "The ChatGPT says 'Moooo....'", I suppose.)

The first thing I realized is that you have to approach LLMs such as Chat GPT and Claude like a logic puzzle, or maybe a better approach is that of a programmer. You have to be specific enough and yet frame the query in such a way that the LLM knows how to proceed. 

This is Anthropic's Claude response to my prompt: "what
is the parallel context blog about". Note that Claude asked
a clarifying question that ChatGPT did not ask. And before
you yourself ask, I performed both queries on the two LLMs
within a 1/2 hour of each other.

This is Claude's response after I selected the proper
answer, which was Option #2.

***

The second thing, and to be honest it's the one that bothers me the most, is that when writing queries you have to explicitly tell an LLM that it's okay to say "I don't know". Maybe I'm just old fashioned, but I'd expect any LLM to state that up front if it didn't know something, not have to explicitly TELL the thing that it's fine to say "I don't know". To me, that seems to be such a fundamental flaw that I really have to wonder what the devs were thinking when they didn't build pure honesty into these LLMs from the beginning. 

If anything, it should be Law Zero before the rest of Isaac Asimov's Three Laws of Robotics come into play:

A robot must always speak accurately, and if a robot does not know the answer to a query posed of it a robot should respond with "I don't know the answer given the data provided."

For reference, here's the actual Three Laws that Isaac came up with back in the 1940s*: 

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

***

The third thing is a corollary to the first two, and that's the old adage "Trust But Verify." 

It's pretty much a given whenever someone --especially a salesperson, an executive, or a politician-- opens their mouth to assert something you go and check it, but the way LLMs are designed lend a certain amount of authority to their statements that humans can instantly warm up to, no matter if the statements are wildly inaccurate. 

That feeling of trust that you build out of an LLM is an emotional response, and that is a problem. When you start to trust the LLM, you make mistakes and stop verifying results. That's a pretty bad headspace to be in, because you will get burned. And if you're using that for work or to manage relationships, far more people will get burned too. 

And that's not even counting the potential to have a romantic emotive response to an LLM. This isn't the movie Her or even one of Asimov's robot stories, but real life. But even a short conversation with an LLM can easily show how seductive that is. 

Here's a snippet from a conversation I had with Claude about inspiration. 

NOTE: the "Ha" in my prompt was caused by Claude acknowledging my starting up the app by stating "You're up late tonight." Obviously Claude doesn't know my sleeping habits...

As you can see, Claude not only provided information in a bullet point style, it also presented things in a conversational way; this is the sort of conversation you'd expect at a coffee house or a bookstore in the evening. While the conversation itself is pleasant, it's also friendly in a sort of way that lets your guard down when you ought to have it up for verification purposes. 

It was in that moment I instantly realized the seductive appeal of becoming emotionally attached to an LLM.

"Holy Shit!" I blurted out. Thankfully this was late at night when my wife had gone to bed or I'd have to explain myself. 

***

In the end, I intend to keep moving forward, but I have to remain vigilant to the potential pitfalls here. LLMs can be useful tools, but we shouldn't assign more to them than what they already are. This might be self-evident, but you'd be surprised at what might happen if you're not paying attention.

*Yes, I know, I Robot came out in 1950, but it was comprised of short stories that Isaac wrote in the years prior to that. In the case of the Three Laws, Isaac created them for the story Runaround, which originally was published in 1942. I perused a recent printing of I, Robot at a bookstore and discovered that the references to the original short stories had been removed in the information. Bad move on the publisher's part, if you ask me.

Here's my copy from the 80s.
I originally read a copy from the library,
so that's why my copy is still in great shape.

#Blaugust2026