Publications

You Have Earned a Trophy: Characterize In-Game Achievements and Their Completions

game analytics HCI user engagement

Achievement systems have been actively adopted in gaming platforms to maintain players’ interests. Among them, trophies in PlayStation games are one of the most successful achievement systems. While the importance of trophy design has been casually discussed in many game developers’ forums, there has been no systematic study of the historical dataset of trophies yet. In this work, we construct a complete dataset of PlayStation games and their trophies and investigate them from both the developers’ and players’ perspectives.

Haewoon Kwak

ACM WebSci, 2022

Identifying Regional Trends in Avatar Customization

computational social science game analytics HCI

We manually label 5,143 hateful expressions posted to YouTube and Facebook videos among a dataset of 137,098 comments from an online news media. We then create a granular taxonomy of different types and targets of online hate and train machine learning models to automatically detect and classify the hateful comments in the full dataset.

Peter Mawhorter, Sercan Şengün, Haewoon Kwak, D. Fox Harrell

IEEE Transactions on Games, 10(2), 2018

I Would Not Plant Apple Trees If the World Will Be Wiped: Analyzing Hundreds of Millions of Behavioral Records of Players During an MMORPG Beta Test

computational social science game analytics

We use player behavior during the closed beta test of the MMORPG ArcheAge as a proxy for an extreme situation-at the end of the closed beta test, all user data is deleted, and thus, the outcome (or penalty) of players’ in-game behaviors in the last few days loses its meaning. We analyzed 270 million records of player behavior in the 4th closed beta test of ArcheAge.

Ah Reum Kang, Jeremy Blackburn, Haewoon Kwak, Huy Kang Kim

Proceedings of the 26th International Conference on World Wide Web (WWW) Companion, 2017

Press coverage-New Scientist, IFL Science, PC Gamer, Massively OK, El Confidencial, Joongang Ilbo, and so on.

10+ papers citing this work (Google scholar)

Achievement and Friends: Key Factors of Player Retention Vary Across Player Levels in Online Multiplayer Games

game analytics user engagement

Retaining players over an extended period of time is a long-standing challenge in game industry. Significant effort has been paid to understanding what motivates players enjoy games. While individuals may have varying reasons to play or abandon a game at different stages within the game, previous studies have looked at the retention problem from a snapshot view. This study, by analyzing in-game logs of 51,104 distinct individuals in an online multiplayer game, uniquely offers a multifaceted view of the retention problem over the players’ virtual life phases.

Kunwoo Park, Meeyoung Cha, Haewoon Kwak, Kuan-Ta Chen

Proceedings of the 26th International Conference on World Wide Web (WWW) Companion, 2017

35+ papers citing this work (Google scholar)

Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games

game analytics HCI online harm

In this work we explore cyberbullying and other toxic behavior in team competition online games. Using a dataset of over 10 million player reports on 1.46 million toxic players along with corresponding crowdsourced decisions, we test several hypotheses drawn from theories explaining toxic behavior. Besides providing large-scale, empirical based understanding of toxic behavior, our work can be used as a basis for building systems to detect, prevent, and counter-act toxic behavior.

Haewoon Kwak, Jeremy Blackburn, Seungyeop Han

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), 2015

310+ papers citing this work (Google scholar)

Linguistic Analysis of Toxic Behavior in an Online Video Game

game analytics online harm

In this paper we explore the linguistic components of toxic behavior by using crowdsourced data from over 590 thousand cases of accused toxic players in a popular match-based competition game, League of Legends. We perform a series of linguistic analyses to gain a deeper understanding of the role communication plays in the expression of toxic behavior. We characterize linguistic behavior of toxic players and compare it with that of typical players in an online competition game. We also find empirical support describing how a player transitions from typical to toxic behavior. Our findings can be helpful to automatically detect and warn players who may become toxic and thus insulate potential victims from toxic playing in advance.

Haewoon Kwak, Jeremy Blackburn

SocInfo Workshop on Exploration on Games and Gamers (EGG), 2014

110+ papers citing this work (Google scholar)

STFU NOOB! Predicting Crowdsourced Decisions on Toxic Behavior in Online Games

game analytics online harm

We propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with large-scale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in detecting overwhelmingly majority cases and predicting crowdsourced decisions on them. We demonstrate good portability of our classifier across regions.

Jeremy Blackburn, Haewoon Kwak

Proceedings of the 23rd international conference on World wide web (WWW), 2014

Press coverage-Nature, Scientific American, Chosun Ilbo

190+ papers citing this work (Google scholar)