Edit 11-2-18 0:57 AM: Added SS of Dogda’s reaction to my blog post
Edit 11-1-18 1:01 AM: Changed the content to have more under The (Nintendo) Switch : x
Hello my imaginary friends and followers. I just wanted to give you all a quick update that if you follow me, you can truly become imaginary like this one follower:
Anyways, the third project for my Human-Computer Interaction class is to create a chatbot for some kind of purpose that requires a human connection. Some example my professor gave was teaching someone a specific topic, counseling someone who’s struggling emotionally, or debating on a contentious topic. I did this project with Leo, Eddie, and Ethan, and our original vision was to create a chatbot to counsel first years who were feeling lonely/isolated. However, it changed to helping athletes who were having difficulty transitioning to Oxy with a focus on soccer since Ethan is on the soccer team.
Our final chatbot can be found at https://github.com/nguyen41v/oxycsbot. Just make a copy or download the repository and you can run it! It was a little too hard to try to get the slack app working in other Slack workspaces, but if we are able to do that, I’ll add the link to add the bot to workspaces here.
The constraint that your chatbot should tap into human connections is part of the challenge, and asks whether chatbots can do more than scheduling a hair salon appointment.
As you are designing the chatbot, keep in mind that these are topics where “multiple choice” responses are not appropriate, nor the simple parroting we’ve seen with ELIZA. In fact, these are situations where saying the wrong thing may cause more harm than not saying anything at all.
Our bot’s name is Santi, named after another one of my imaginary friends.
Brainstorming for the Initial Focus:
The first step for our project was choosing what we wanted Santi’s focus to be. Who should it be for and what should it do? We settled on the broad idea of counseling students who are feeling isolated at Oxy since we had no preferences on what Santi should do. Our first task was to come up with conversation ideas, so we decided to do some individual research on college loneliness and put information we found into a list. Coming back together, our research gave us the idea to focus only on first year students since they are more likely to be experiencing loneliness considering that college is a different environment than high school, and first years are also surround by many faces they’ve never seen before. After creating some possible responses, we tested the bot with Junepyo, an Oxy student.
His user testing gave us some insight as to what kind of things didn’t account for, such as relationship problems. Junepyo also gave us some feedback saying that he didn’t like how Santi always responded with the same “Sorry, I’m just a simple Jane . . .” when it couldn’t do anything with his response. His feedback got us to thinking about randomizing the responses Santi would give so that it would seem more human. Junepyo was also referring to the bot as Santi when he gave us feedback, so we decided to keep that name and also change the responses so that Santi would refer to itself as Santi.
The (Nintendo) Switch:
However, as we generated more responses and talked more about the flow of the conversation, we realized that what we wanted to do would require a lot of different responses and coding; we would need to keep track of what the user has already talked to Santi about and whether Santi has been in a certain state before or not since most of our states link to other states or back to itself. It was slowly getting more and more complicated and required more and more responses as we put more possibilities into consideration. The work required to implement Santi was exponentially increasing as we progressed through it. We also didn’t know how to handle the darkness; what do we do if students need serious help that do warrant a psychologist or therapist?
We tried to account for those scenarios, but this fact combined with the fact that our topic was a bit too broad caused us to reconsider Santi’s purpose in life, much like how some first years might be doing to themselves.
Link to branch containing code from our original project focus: https://github.com/nguyen41v/oxycsbot/tree/lonely
Eventually we decided to change Santi to focus only on helping first year athletes transition from high school to Oxy. This was basically a sub-topic from our original vision, and we picked it because our team members have more experience with sports at Oxy compared to some of the other sub-topics, such as family and homesickness. Since we decided to only focus on athletes, we discarded a lot of the previous responses and flows since they weren’t related to sports, such as academics or art. This made implementing Santi a lot easier since we didn’t have to account for possible responses in those directions and we only needed to account for possible responses within the realm of sports.
We went back to the white board and worked more on creating the conversation flow, implementing it as we went in the code (albeit, a very basic version of the conversation).
We tried to keep Santi as an upbeat and understanding bot. The upbeatness of Santi would make Santi seem enthusiastic about sports and the user, while the understandingness will comfort the user. The first thing we did was create an introduction for Santi that will let users know what Santi does. Santi’s introduction will make it clear to users about what they should talk to him about.
User: Hi there!
Santi: :shocked_face_with_exploding_head:Hello, I’m Santi.
I help college student athletes better transition to college sports at Oxy.
How has your transition in sports been?
After that, we decided to filter users to different flows based on whether they’ve been having a good or bad transition, routing students to the bad transition path if they mention some problems while they’re on the good transition path. An example of a problem is a user indicating that they don’t think they’re doing well in the SCIAC conference. We thought about generic responses Santi could have, such as:
Santi: Have you told your coach?
Santi: I would recommend talking to your teammates. That might help with your transition. Do you have any of your teammates’ contact information?
These kind of responses might be able to provide direction for what the user can do to improve their situation, and it also moves the conversation forward. Not too many problems came up while we were designing the conversation flow, however, we did realize that the conversation was composed of mainly yes or no questions, which we tried to fix. Some ways we did this included asking the user to tell Santi more about their situation or asking the user about their mentor’s name.
Another thing that came up later on was the question of what to do if someone entered in a mentor name that the bot did not recognize. We could create a function that would loop on itself as long as the user does not enter a valid name, but what if the user never types in a valid name and how do we make the prompt to enter a valid name seem more natural? What we ended up doing was keeping track of the number of times a user entered the function, and having Santi give a different response if it’s not the first time the user is in the function (just realized that we never reset this value . . .). We also added new tags for words a user might have in their response that might indicate that Santi should stop asking them for their mentor’s name. For example, a user might say that they forgot or don’t remember their mentor’s name. As we created the flow, we did also did basic checks and tests just to make sure that functions linked to the right state after different responses.
The Final Stretch of User Testing and Changes:
Once our flow and code were mostly done, we did more in depth testing with actual responses to check what kind of other responses users might input that we might not catch with our current tag setup.
There were some funny moments in our testing, but it made us realize the variety of responses users could have, such as “not bad”, “not that well”, or “fine”. We implemented some more tags so that we could capture these more neutral responses because we were only considering positive and negative responses, and Santi shouldn’t say “That’s great to hear!” to a neutral response. Our tag was called “medium_rare” : ) We also added other tags and functions in order to make Santi a bit more sociable since it didn’t really converse as much at that point (but making a sociable-er Santi is hard). I got to teach Leo about using the terminal for git a little bit throughout our project. We had fun playing around with the terminal and merge conflicts in git (turn up your volume when you play the video).
Too bad it’s not Christmas yet. Shortly after this video, I looked up how to select lines/blocks of code to delete (in vim, you can do shift+v, select the lines you want to delete, and then d; putting in “:[#lineStart],[#lineEnd]d” or “[#lineStart]Gd[#lineEnd]G” apparently works too but I haven’t tried it yet).
We did some more user testing with other people and ourselves and found some bugs or tags we should consider, doing some more tests after each update to fix those bugs. Our user testing was done on other students, since all students have been first years before. We told them that Santi was meant to help first year athletes transition from high school to college.
The following video shows an example of what the conversation would be like for students experiencing difficulty in their transition. We implemented “haven’t” after this video, but the video is a good depiction of what kind of responses Santi would give.
This next video shows how Santi would interact with students if they weren’t experiencing a bad transition. In contrast to the previous video, Santi has more positive responses. Not all of the possible paths are shown in these two videos though, as Santi is also able to provide contact information for mentors on the soccer team, something that other sport teams do not have, among other responses.
We did multiple tests, adding some quick fixes with each one based off of the transcripts and then we were DONE! GitHub repo in case you want to see it again:
https://github.com/nguyen41v/oxycsbot
Ending Remarks:
Overall, the process of making Santi enlightened me on the difficulty of making a hybrid system (compared to Eliza based systems that takes in user responses and looks at the message structure to provide an appropriate response or dialog systems that have preset responses for users, hybrid systems take in any user responses and breaks them up, and based off of the content, it selects a few preset response) for conversational interfaces or chatbots. I knew from the start that it would require a lot of work just because of the way we were implementing it in python through the use of tags and specific states for different responses. This whole set up means a lot of hard-coded responses and tags, which means that we have to think of all the possible outcomes and what is common and different between them in order to send Santi to the right state. Sometimes similar responses can mean very different things, and Santi had to be able to catch that. For example, “not good” and “pretty good” both have “good” in them but they mean very different things. Santi had to be able to differentiate between those two phrases in order to give an appropriate response.
Unfortunately, we weren’t able to implement our initial vision of helping first-year students that feel isolated because of the amount of coding required for it, but we were still able to implement a small part of it specifically for athletes. We had also wanted to implement giving contact information for coaches and faculty on sports team, but we did not create a conversation flow for that and we also didn’t have all their information (you can view the start of our implementation in our code at line 705 though). Let’s take the time to appreciate all the work and effort people have put into designing and creating non-machine-learning based hybrid conversational interfaces.
Seluni's Adventures 