Information searching with ChatGPT

ChatGPT will replace Google searches! Artificial intelligence passed Bar Exam for lawyers! New ChatGPT version 4 produces more up-to-date and accurate information than the previous one! Sound familiar?

Before you start using only ChatGPT in your information searches or producing texts, it's a good idea to understand it's limitations.

This image of a librarian was created using the AI tool DALL-E 2. DALL-E 2 was specifically asked for an image of a librarian who does not wear glasses. The request went beyond the imagination of the tool. It seems that in the AI world the idea of a eyeware-free librarian is unfathomable. This is one example of the limitations of AI tools: the quality of output is dependent on the quality of the data used in training the AI tool. See below the image for more information about using ChatGPT in information searching.

Tekoälyn tuottamassa kuvassa henkilö lukee kirjaa suuret silmälasit otsalla.
News article


Checklist for using ChatGPT in information searching

  • ChatGPT is not a Google-like search engine that searches the internet. In fact, it cannot access the internet to conduct searches.
  • The free version and newer version utilizing GPT-4 were both trained with a vast amount of data and content gathered from the internet before fall 2021. Neither version has access to newer information.
  • You cannot trust that the answers provided by the AI bot are based on facts. It uses a language model whose algorithm predicts which word is statistically likely to follow the previous word. It doesn't "understand" the meaning of these words or the content of the text it creates, let alone the accuracy of it.
  • The quality of the data used to train an AI tool is mirrored in the quality of its answers. The internet contains biased, inadequate information as well as outright lies. It is likely the same is true for the training data. Detailed information on the training data has not been made public.
  • ChatGPT hallucinates. The term artificial hallucination refers to a situation where the AI tool invents information, when it 's asked a question and the AI cannot find answers based on it's training data. For instance, in tests conducted by the library we found that ChatGPT invented articles or writers that do not exist.
  • Chat GPT has been trained with data in numerous languages. You can chat with it in many languages. However, it has been largely trained in handling and producing texts in English. The tool has not been as extensively trained in other languages. When working with other languages, it's important to keep in mind that this may have an impact on the results.
  • ChatGPT can be a useful writing tool in finding new perspectives, drafting the structure of the text, or overcoming the blank paper syndrome. Nevertheless, checking information, reading source materials, editing the text, and the actual thinking is still up to the human using the tool.
  • Checking the data created by the AI chatbot can be time consuming. ChatGPT's rival may bring some relief for this problem. According to preliminary reports, Google's AI chatbot Bard will include a search button that you can use to conduct a Google search on the same topic. Even then, you've only taken the first step towards evaluating the correctness of the information:  the sources found this way still need to be read, evaluated and the information checked.

This checklist was adapted from article Lähdekritiikin merkitys korostuu tekoälybottien aikakaudella published in eSignals. To find out more about library's ChatGPT tests, see also ChatGPT testissä: onko hypelle katetta?.