A chatbot operated by artificial intelligence, developed jointly by the University of Pécs Medical School (UPMS) and IBM, can already pass a written examination in neuropsychology. It will not become a virtual doctor that could fit into Star Trek just yet, but it will be an assistant that helps the doctor during complex surgical procedures or provides accurate and understandable health advice to lay people.
Written by Miklós Stemler
Nowadays, if we are not listening to featureless elevator music or some classical music on a telephone customer service, there is a good chance we are “talking” to a virtual assistant instead of a flesh and blood clerk, just as our questions online are not answered by people either. The often frustratingly stubborn, but rapidly "learning" machine helpers are present in more and more areas of life, and the researchers of the Medical School in Pécs and the developers of IBM have now created a version that can answer complex health questions as well.
Our new best and unknown friends: BERT & Watson
“Probably none of us get up in the morning looking forward to a fifteen- or thirty-minutes conversation with the telephone customer service of the gas supplier or the bank, which will consist predominantly of listening to music. In recent years, however, IT has reached the level of processing a particular body of knowledge with the help of artificial intelligence - just as a customer service representative acquires the information needed to provide assistance. In addition to the current state of technology, these applications are already surprisingly capable of human communication, both in oral and written form. This was highlighted by the epidemiological situation earlier this year, which made a lot of personal interactions difficult or downright impossible, making the need for such solutions even greater” - IBM’s Central and Eastern European data science and artificial intelligence architect, László Boa outlines the background.
The coronavirus epidemic, more specifically the new and largely unknown nature of the disease, has greatly increased the need for up-to-date, credible information, which has also overburdened customer services and information centres. In response, IBM offered its AI assistant named Watson, developed for businesses, to provide health information about the virus free of charge. “Watson” has previously demonstrated his skills in the popular American television quiz show, Jeopardy! as well, in which he defeated his human competitors in several rounds.
At the same time, the company and the University of Pécs have been cooperating for a long time in the field of various technologies aimed at natural language comprehension and technologies requiring high computational capacity, and this gave rise to the concept of a chatbot that quickly turned out to be suitable not only for providing information in connection with the coronavirus but also for general health counselling and even professional assistance.
Based on IBM's Watson technology, researchers at the UPMS Department of Behavioural Sciences have developed the central part of the system, which analyses and processes the data, with the help of a large-scale artificial intelligence model developed by Google that understands and processes natural language, “BERT”. We have probably already met BERT, as Google’s search engine has been using it since the fall of 2019 to supplement the text we type and make suggestions - first in English only, but now in more than 70 languages. The artificial intelligence tool interprets the text through a so-called deep learning network that imitates the human nervous system and is able to interpret the context in great detail due to its huge number of information filtering layers.
Chatbot under development
After Google made the source code open, the Applied Data Science and Artificial Intelligence research group in Pécs, led by senior lecturer dr. Ádám Feldmann, also delved into the system, as it operates more efficiently than other similar solutions. Among other things, BERT performs particularly well in text classification, i.e. in the substantive and meaning-based separation of certain texts and is able to highlight essential parts of texts. In addition, the system with hundreds of millions of parameters is able to interpret and answer questions, and this function was the most important for the UP researchers.
“The BERT model is, in fact, a framework capable of filtering and interpreting a given body of knowledge and formulating specific, short answers based on it. For example, if we ask who Winston Churchill was, it will identify during the processing of the question that it is a person, search for the relevant Wikipedia article, and then extract a specific answer from it: a former British Prime Minister.” - explains Ádám Feldmann.
BERT’s original training material was a 3.4 billion-word text corpus of the English-language Wikipedia and BookCorpus combined. This can be further refined by entering various newer language databases, thus creating new versions of the original model. Korean scientists, for example, have “trained” a medical database of more than 4 billion words, and the AI system has then become able to provide brief and correct answers to questions such as what Parkinson’s disease is. Ádám Feldmann, who teaches neuropsychology, asked questions from his own previous exam material, and to his great surprise, the model managed to pass the exam.
The team in Pécs focused on fine-tuning the basic BERT model, i.e. its theme-specific development, and produced three versions. The first one can answer coronavirus-specific questions, the second gives health advice to lay people in an understandable way. The third one is the “expert” model, which successfully passed the neuropsychology exam. In addition to the improvement of data processing capabilities, another big novelty of the development in Pécs is that the AI system, which is of great knowledge but not designed for end users, has become an application that can be used by anyone. All three of the above solutions can answer the questions in English.
“Today we expect basic skills from a chatbot such as not forgetting the information spoken/written five minutes earlier and that it should be able to perform simple interactions like ordering fast food or reserving a table. We provided this foundation with Watson and the UP team added to this the ability of the chatbot to give concrete answers in specific areas through processing a huge amount of knowledge” - László Boa summarizes the result of the joint work.
Dr. Chatbot is in the office
The result has become a confusingly human-like solution compared to traditional chatbots, which, when answering, adapts to the language style and level of preparedness of the person asking the question. For example, if we ask how to lose weight, we get the answer to lead a lifestyle where we burn more calories than we take in, and if we come up with a more challenging question about what causes Parkinson’s disease, it informs us that although the “pathology” of the dreaded disease is still less clear, both genetic and environmental factors are likely to play a role in its development.
The spectrum can range from the transfer of basic health information to answering the physician’s questions about specific steps during a complex surgical procedure, depending on the situation so that the surgery does not have to be interrupted. It can also be useful in education, if not as a teacher, but as a kind of teaching assistant in collecting and processing the material to be taught.
The model can be taught continuously, which means that on the one hand it is able to keep up with scientific developments and incorporate the latest information, but on the other hand, the databases it is provided with also determine the answers it gives - for example, it can become a coronavirus denier through “appropriate” selection. All this makes human supervision of the curriculum and the operation of the chatbot important. “In the case of AI-related developments, we must always keep in mind ethical considerations and that our goal is to complement human capabilities, not to replace people. In many cases, the validation of these aspects requires more time than the technological development itself, but this is the only way to guarantee that autonomous algorithms do not cause more problems than the benefits they generate” - says László Boa.
The chatbot currently operates at the UPMS Department of Behavioural Sciences, and according to its developers, Ádám Feldmann and Róbert Hajdu, it is an important goal to make it as widely available as possible for UP students and teachers in order to get as much feedback as possible and for the subsequent fine-tuning. All this can lead to newer applications, as the use can only be limited by the finiteness of human knowledge.