Gabelli School of Business Professor Yilu Zhou recounts the tale of a telecommunications company that incorporated and announced a new plan with a faster internet speed for their premium users.
Within 24 hours of the announcement, however, customers took to social media to criticize the new plan. Some of them even switched vendors.
“Because the analysis tools the company was using couldn’t pick up on a small group of customers complaining, the social media team didn’t find out there was an issue with the plan for three days,” says Zhou, Ph.D., associate professor of information systems. “The technique they were using was too simple.”
Zhou recently co-authored the paper, Text Analytics to Support Sense-Making in Social Media: A Language–Action Perspective with Ahmed Abbasi, director of the Center for Business Analytics at the University of Virginia; Shasha Deng, assistant professor at Shanghai International Studies University; and Pengzhu Zhang, director of the Center for Management Information Systems at Shanghai Jiaotong University.
The researchers propose an algorithm that helps to identify the order of conversations on social media, as well as the context of the messages. The algorithm also identifies influential users who generate the most discussions.
“Very often when you’re on any social media platform, users are constantly replying to a main post, but not every single reply is a direct reply to that main post,” says Zhou. “Sometimes a user may reply to a message that was posted five days ago, and very often these dialogues are intertwined and hard to untangle.”
Zhou says these sense-making challenges can cause companies to miss out on the early signs of dissatisfaction among consumers.
Making Sense of Complex Online Discussions
Zhou and researchers from the University of Virginia conducted three types of experiments in the telecommunications, healthcare, security, and manufacturing sectors. They also explored the pragmatics and the syntax of people’s conversations on traditional social media platforms like Facebook and Twitter, in addition to niche social networking sites like PatientsLikeMe.
In the first experiment, language experts categorized conversations on social media based on the “speech act.” A speech act is a message that is assertive, commissive, expressive, declarative, or directive. The language experts then created a gold standard for evaluation that Zhou used to compare the effectiveness of her team’s algorithm.
The second experiment asked non-experts a series of questions to answer based on reconstructed messages. A simple question might be: “Identify all of the ideas in an entire discussion about a company product or service.” A more difficult question might be “Identify the discussant who seems the most frustrated about an issue.”
“To answer the question, the user must understand [both]the context of the conversation and the tone of the user,” says Zhou. “Knowing which issues caused the greatest confusion might be useful to companies when they introduce a new policy or service.”
In stage three, the researchers conducted a four-month field study of a telecommunications company. The company has a team of 23 members doing social media monitoring of the reactions to company products and services. One group used the algorithm created by Zhou’s team and the second group used the company’s methodology. Whenever there was a problem or user complaint, a ticket was issued with a dollar amount.
Based on the volume of tickets received and how fast the issues were resolved, Zhou’s team’s algorithm was able to save the company $13.5 million, besting the existing methodology’s $9.1 million in savings.
“With our algorithm, they became more efficient and accurate,” she says.
The algorithm also helped the company retain 7.2 million contracts, to the existing methodology’s 4.5 million contracts.
Zhou says the findings can help businesses to make sense of complex online discourse.
“Social media is changing business,” says Zhou. “It’s changing the entire market research area and customer support.”
