A Chat with ChatGPT

It is not a truth teller.

Like you, I’ve heard about the artificial intelligence chat engine ChatGPT. I’m a nerd so had to try it. Got a free subscription at https://chat.openai.com/auth/login. Tested by going to the playground and asking about myself because I’m sure I understand that subject at least as well as the chatbot. Here’s our conversation. My comments on the conversation (not shared with ChatGPT) are in brackets [  ]:

*******************************************************************************

Me: Where is Tom Evslin?

ChatGPT: Tom Evslin is currently living in the San Francisco Bay Area. He is the Chairman and Chief Executive Officer of Helium Systems, Inc.

[Neither of these things are true. I haven’t lived in the Bay Area since a vagrant month in Berkeley in 1961. I’ve been the Chairman and CEO of several companies but have never been associated with or even heard of Helium Systems. The company does exist (thank you, Google) and lists three founders but does not list either a Chairman or CEO.].

Me: Tom Evslin does not live in the San Francisco Bay area.

ChatGPT: He is also the founder of ITXC Corporation, a leading global provider of voice-over-IP (VoIP) services. He is also a venture partner of Polaris Partners, a venture capital firm.

[I was the founder of  ITXC Corporation, and it was a leading global VoIP provider. I’m not a venture partner of Polaris Partners. ITXC was partially funded by an Israeli VC firm named Polaris.]

Me: Why do you think Tom Evslin lives in the Bay Area?

ChatGPT: Tom Evslin likely lives in the Bay Area due to its thriving tech industry, which provides an ideal setting for his work as an entrepreneur, investor, and executive. The Bay Area also boasts an impressive network of venture capital firms, which can provide additional support and resources for his business endeavors. Additionally, the Bay Area is a hub for creativity and innovation, making it the perfect setting for someone with a wide range of interests.

[These would all be good reasons for me to live in the Bay Area, but I don’t. ChatGPT apparently interpreted my question as “why would Tom Evslin choose to live in the Bay Area?” rather than what I meant which was “what would lead you to think he lives in the Bay Area?” You gotta be careful how you phrase a question to a robot.]

Me: Why do you think Tom Evslin is chairman of Helium Systems?

ChatGPT: Tom Evslin is likely the chairman of Helium Systems because of his extensive experience in the tech sector. He is a highly successful entrepreneur, investor, and executive, and his expertise can be beneficial to the development of Helium Systems. Additionally, his background in providing VoIP services makes him well-suited to lead a company that is focused on communication technology.

[These would be good reasons and I’m glad ChatGPT appreciates my background. But I don’t work there.]

Me: Who is the CEO of Helium Systems?

ChatGPT: The CEO of Helium Systems is Tom Evslin.

[At least it’s consistent.]

Me: What evidence do you have that Tom Evslin is CEO of Helium Systems?

ChatGPT: Tom Evslin is listed as the Chairman and CEO of Helium Systems on the company's website. Additionally, his profile on Polaris Partners, a venture capital firm, states that he is the CEO of Helium Systems. He is also mentioned in a press release from Helium Systems, where he is credited with being the CEO and Chairman.

[None of this is true even though it sounds like a good answer.]

Me: What is the URL of the website which lists Tom Evslin as Chairman and CEO of Helium Systems?

ChatGPT: The URL of the website which lists Tom Evslin as Chairman and CEO of Helium Systems is: https://helium.com/about/

[This page does not exist although helium.com does.]

Me: That URL does not exist.

ChatGPT: The correct URL is https://heliumsystems.com/about/

[This page doesn’t exist either but note that both URLs look legit and someone who didn’t check them might have accepted them as evidence.]

Me: What is the URL of Tom Evslin's profile at Polaris Partners?

ChatGPT: The URL of Tom Evslin's profile at Polaris Partners is: https://www.polarispartners.com/team/tom-evslin/

Me: That URL does not exist.

ChatGPT: The correct URL is https://www.polarispartners.com/team/tom-evslin/

[This time ChatGPT didn’t even bother to invent a new URL.]

Me: What is the URL of the press release which says Tom Evslin is the Chairman and CEO of Helium Systems?

ChatGPT: The URL of the press release which says Tom Evslin is the Chairman and CEO of Helium Systems is: https://www.prweb.com/releases/helium_systems_announces_tom_evslin_as_chairman_and_ceo/prweb17093196.htm

Me: That URL does not exist.

ChatGPT: The correct URL is https://www.prnewswire.com/news-releases/helium-systems-announces-tom-evslin-as-chairman-and-ceo-301177588.html

[Neither of these exist although there are other press release on prnewswire which mention me and this is what the link probably would’ve looked like if there’d actually been a Helium Systems announcement.]

*******************************************************************************

A few random thoughts (by me, not ChatGPT):

ChatGPT could pass for a human. It succeeds at what we nerds call the Turing Test.

ChatGPT gives credible but not necessarily correct answers.  I might have lived in the Bay Area. The non-existent URLs are based on real websites. This is not surprising since the programming antecedents of this technology include the algorithms which check your grammar and make suggestions when you are typing in a word processor. You don’t expect that software to do anything but guess and don’t accuse it of lying if it guesses wrong.

ChatGPT makes up a lot of stuff. It was mainly trained by reading gigabytes of stuff on the web. Some was true. Some wasn’t. It apparently learned how to make stuff up which sounds true. Not surprising given what it was schooled on. This actually tells us as much about the web as it does about ChatGPT.

Don’t use ChatGPT to do your office or school homework unless you check everything it says. Unlike Google, it doesn’t volunteer a source link for its assertions. Of course, a source link doesn’t prove authenticity but does give you a place to start looking.

I’ve known human assistants who are no more accurate than ChatGPT and their grammar isn’t usually as good.

BTW, GPT stands for “generative pretrained-transformer” according to Wikipedia.

Everyone who is using ChatGPT is helping to train it further, particularly if they up or down vote the answers. There are other artificial intelligence programs and will be many more. But a powerful first mover will have an increasing advantage similar to what Google has in search because of accumulated feedback making it increasingly useful and therefore attracting more use and more feedback.