Karl Wiegers
1 min readNov 3, 2024

--

Boris, regarding your criticisms of the question we posed to the chatbots, consider that (1) a vague notion of what constitutes "importance" is just the sort of thing a human would have in mind when asking another human such a question; (2) Meilir and I are in fact experts in this field and so our personal expertise has some validity; and (3) there are indeed some objectively correct names that most other experts and practitioners would agree upon as good responses to the question. Variability aside, the inclusion of clearly inappropriate and even imaginary people in the answers is most disturbing to me.

You're right about (4): our question is so specialized that most people simply don't care about it. But that in no way lessens the validity of this test case, as it's an example of a reasonable sort of question that someone might pose to some source of knowledge. And if it's an inappropriate question, the chatbot should say, "Sorry, I can't help you with that."

--

--

Karl Wiegers
Karl Wiegers

Written by Karl Wiegers

Author of 14 books, mostly on software. PhD in organic chemistry. Guitars, wine, and military history fill the voids. karlwiegers.com and processimpact.com

No responses yet