POTEcho is a quick and easy feedback system designed to facilitate communication between teachers and students, supporting teaching and educational development. The software has been in use at the Medical School in Pécs since 2019, under the supervision of the Feedback Committee, but it has recently been renewed and is available to teachers and students from the spring semester. Further development of POTEcho was coordinated by Dr Gergely Csaba (content), member of the Feedback Committee and a PhD student of the Division of Medical Education Development and Communication, and Dr Péter Bogár (technicalities), a PhD student of the 3D Printing and Visualization Centre. In this interview they talk about the details of the development, how to use the system and its benefits.
- What does POTEcho have to do with the feedback system in Neptun, which has been running for years with high fill-out rates?
G.Cs.: - The essence of both systems is feedback, but while the feedback in Neptun is conducted at the end of each semester, i.e. students can fill in the more summative questionnaires after the completion of the course, in POTEcho this can be done during the semester, right after the classes. The two different feedbacks taken at different times can lead to different results, as international studies have suggested. If students are constructive about the questionnaires and formulate useful opinions, this can have an impact on their education, so they are of great importance. If POTEcho performs well, it can take over the end-of-semester surveys in the long run.
The 2019 version of POTEcho has been further developed to make feedback easier to manage from the user’s side, to make it available in a smartphone app, and to have a wider range of features, such as a transparent timetable display and the possibility to manage polls during a class.
P.B.: - The most important change is the automatization of feedback questionnaires and the benefits this brings, such as easy access, instant reports, or full queries. In previous semesters, if someone wanted to create a feedback questionnaire, they had to create and publish it manually, and there were several steps for students to access the questionnaires. Automatization has simplified these steps. Questionnaires are automatically created for each lesson (lectures, practices, and seminars), which students can access and complete on their mobile phones. The results are immediately visible to the instructors in the form of a short report, thus closing the feedback loop immediately.
The automatization also means that a short questionnaire is generated for all contact hours of all obligatory courses at the Medical School. Until now, the programme has been available on a voluntary basis for instructors and departments, but from now on it will be fully available for obligatory courses. Optional courses have not yet been included in the automatization process this semester.
- How to ensure that both the student and the instructor are assigned to their own lessons in the system?
P.B.: - To get the right user data, it was necessary to integrate the UP (PTE) login and the physical timetable data stored in Neptun. Everyone is familiar with the UP login, as it is required for the Medical School website and other university interfaces. The advantage is that there is no need for a separate registration in the POTEcho system, and the timetable data can be transferred from Neptun for each user. The lesson-student or lesson-instructor relationship is based on the physical timetable maintained in Neptun. We plan to transfer the continuously updated data from Neptun several times, before the semester starts and during the semester. POTEcho can then display the current timetable in a user-friendly way.
G.Cs.: - It is important to highlight that for proper information within POTEcho, it is essential to have a proper administration of the physical timetable in Neptun and to keep track of changes during the semester. If students see a class in their timetable that is not their own, or if their class is not there, they should report it to the departmental administrator. It is very important that these corrections are made in order to ensure correct and valid results. It often happens that a substitute teacher is needed at an unexpected time - especially in the clinical departments - so corrections can be made afterwards.
- Regarding UP (PTE) login, students may wonder whether the Neptun code can be used to identify who has expressed what opinion. How can their anonymity be guaranteed?
P.B.: - Technically, we solved the problem by separating the students from their responses at the time of submission. This way, the individual answers of the database in the background will no longer be suitable for backwards identification of the sender.
G.Cs.: - Anonymity was an important consideration from the start. What’s also important to note is that the given feedbacks could not be used as a catalogue or class-attendance sheet, which is also an important factor in order to protect the anonymity of the students. If someone is not in class, they should simply not complete the given questionnaire and they won’t have to worry about any disadvantages.
- From the students’ point of view, POTEcho creates a lot of questionnaires. How can it be ensured that they fill them in, or how can they be motivated to do so?
G.Cs.: - In order to increase the completion rate, we have tried to make the questionnaires easily accessible and relatively short, to reward the student for completing them and, perhaps most importantly, to make them understand the importance and consequences of doing so.
The software will be available not only via the website, but also as a free app for both Android and iOS. Hopefully, this will make things much easier for students through the possibilities offered by smartphones.
The questionnaires created for each lesson are guaranteed to be very short. On a scale of 1 to 5, the student can rate the lesson overall and optionally give a text response. In the preliminary discussions, there was also a strong demand for the latter. Instructors also have the opportunity to add one or two questions of their own to their questionnaire if they wish.
Completion will be rewarded with a prize draw, with five to six medschool souvenirs being drawn every week. In the first weeks this will be a flask, and in the later weeks we will offer other surprises.
- What do mean by ‘students can see the consequences’ of filling in the questionnaires?
P.B.: - The main aspect is reasonable transparency. Students can see the average of their answers to the lessons almost immediately after completing them, which makes the feedback more tangible for them, as their feedback does not just disappear in the system. In addition, the averages for individual subjects, departments and the faculty as a whole are expected to be made public later in the semester. We hope that this change will also motivate students to fill in the questionnaires responsibly.
G.Cs.: - It is very important for the students to be partners in the process. The only way for a teacher to improve is through realistic and relevant feedback, and through constructive criticism. Feedback opinions such as “what a pity that the instructor has a wedding ring” or “this instructor is very bad” are none of the above. However, feedback such as “I liked the group work because she got everyone actively involved in the lesson, but the exercises were too difficult, and we couldn’t do them on our own” would be constructive.
Student engagement is also an integral part of the Learning Culture concept of the PotePillars strategy, and we have tried to reflect this here, too. We hope that students will experience the changes in education through their fair feedback, and that will encourage more of them to take the time to evaluate.
Whether these changes are actually implemented is up to the instructors and the leadership. Transparency among instructors is achieved by effectively replicating the ‘subject’ hierarchy in the sharing of reports. In addition to the assigned-teacher, the results of a given lesson are automatically seen by the course director, the head of the given department and the faculty leadership. Course directors and heads of departments can also access summary reports on the website, helping to identify steps towards improving teaching and rewarding outstanding colleagues.
As Peti mentioned, it is expected that during the semester there will be data available to all users logged into POTEcho - average ratings for subjects and departments. In discussion with the members of the Feedback Committee, we agreed not to make the results of individual instructors’ public.
- The previous version of POTEcho could also be used to ask questions in class. Is this still possible?
P.B.: - Yes, we have kept the feature supporting the interactivity of the lessons in the new version, and we have improved it significantly. As many of you may know, there are several online tools available on the market that are capable of such queries. Unfortunately, free versions of these services have limited functionality, for example, the number of participants or the number of questions that can be asked. In POTEcho, these limits will be much more permissive.
What is a very exciting area is the integration into PowerPoint presentations. This means that the instructor does not need to go to a web page during the slide show to ask questions and show the answers but can do so during the presentation by simply moving forward. This functionality was already available with POTEcho, but in the current version it is much easier for instructors to manage.
- You have mentioned several improvements, which must have involved a lot of work. Who worked on the design and development of the software?
G.Cs.: - The idea was first conceived when we were students, but we started to work on it more seriously during our final year. Without the personal support of the faculty leadership, Dr Miklós Nyitrai, Dean of the Medical School, and Dr László Czopf, Vice-Dean of the Medical School, the project would not have been possible, and we are grateful for their support from the very beginning.
The first team included five person: Dr Zsuzsanna Füzesi, Chairperson of the Feedback Committee, Dr Ádám Schlégl, Secretary of the Feedback Committee, Dr Péter Maróti, Head of the 3D Printing and Visualization Centre of the UP, Peti and I. The programming was done almost exclusively by Peti. The development of the second version was a much bigger job. We tried to reach as many people as possible, especially those who had used the previous version of POTEcho regularly, in the brainstorming and design process, so we consulted with student representatives, teachers from several departments and administrative staff.
P.B.: - The programming itself was done jointly by the staff of the 3D Printing and Visualisation Centre at the UP and the IT staff of the IT company CoreTechs Solutions, which I coordinated. We already had enough work with the first version, but the new one took even more time, about ten times as many programming hours. We hope that all users will feel the qualitative leap that this brings.
- What will happen to the old version of POTEcho or how can one access the new version?
G.Cs.: - The old version has been archived, which means that all the features of the site are still available, so that teachers can view their previous results and questionnaires. If they want to use it for some reason, they still have the option to do so. However, we hope that the new version will be more popular with students and instructors alike.
The new version of POTEcho is available in the following ways:
Website:
Android app:
play.google.com/store/apps/details?id=hu.cts.potecho&hl=en
iOS app:
apps.apple.com/hu/app/potecho/id1663244449?l=hu
For more details about the new version of POTEcho, please visit the website of the Feedback Committee.