SoDA Lab

Publications

139 boards. Size follows citations, refreshed weekly from Google Scholar.

Method, platform, and year

LLMs Can Infer Political Alignment from Online Conversations

Byunghwee Lee, Sangyeon Kim, Filippo Menczer, Yong-Yeol Ahn, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2603.11253, 2026 2026 4 citations

Innocuous online preferences (the bands you follow, the slang you use) can reveal sensitive traits. We show that LLMs reliably infer hidden political alignment from DebateOrg and Reddit conversations, outperforming traditional ML. Accuracy improves further when text-level inferences aggregate to the user level and when we use politics-adjacent domains, underscoring a serious privacy risk.

LLMs & responsible AI

CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models

Fan Huang, Songheng Zhang, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2604.01366, 2026 2026 3 citations

LLMs show cognitive biases in behavior, but where are those biases encoded internally, and can they be removed? We introduce CogBias, a benchmark covering four bias families (Judgment, Information Processing, Social, Response), and show via linear probes that biases occupy separable directions in activation space. Activation steering reduces bias 26 to 32% with minimal capability loss.

LLMs & responsible AI Bias & fairness

Somatic in the East, Psychological in the West?: A Clinically-Grounded Evaluation of Cross-Cultural Depression Symptoms in LLMs

Shintaro Sakai, Jisun An, Migyeong Kang, Haewoon Kwak

ACL Workshop on Cross-Cultural Considerations in NLP (C3NLP), 2026 2026 3 citations

Clinical research finds Western depression patients tend to report psychological symptoms while Eastern patients report somatic ones. We test whether LLMs (now used in mental health) reproduce this cultural pattern. They mostly don't in English; prompting in Chinese, Japanese, or Hindi helps somewhat, but a strong culturally-invariant symptom hierarchy overrides the cultural cues.

LLMs & responsible AI Digital health

Understanding Moral Reasoning Trajectories in Large Language Models: Toward Probing-Based Explainability

Fan Huang, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2603.16017, 2026 2026 2 citations

How consistently do LLMs apply ethical frameworks across reasoning steps? We introduce moral reasoning trajectories (sequences of framework invocations) and analyze them across six models. Over half of consecutive steps switch frameworks; only ~17% of trajectories stay consistent. Unstable trajectories are more vulnerable to persuasive attacks, and linear probes localize framework encoding to specific layers.

LLMs & responsible AI

SynSym: A Synthetic Data Generation Framework for Psychiatric Symptom Identification

Migyeong Kang, Jihyun Kim, Hyolim Jeon, Sunwoo Hwang, Jihyun An, Yonghoon Kim, Haewoon Kwak, Jisun An, Jinyoung Han

Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2026 2026 1 citation

Mental-health symptom datasets are slow and expensive to build, limiting how well models generalize to real, diverse user posts. SynSym is an LLM-based framework that generates synthetic symptom training data by expanding each symptom into sub-concepts, varying linguistic style, and modeling co-occurrence. Models trained on SynSym data match or exceed those trained on real data.

LLMs & responsible AI Digital health

When Behavior Is Not Enough: Identity Biases in Human-AI Creative Collaboration

Hyunseon Won, Haewoon Kwak, Jisun An, Jinyoung Han

CHI Poster, 2026 2026 1 citation

When AI joins a creative collaboration, does behaving cooperatively erase identity-based bias against it? In a turn-based collaborative drawing study (N=30), participants worked with a human confederate believed to be either human or AI. Cooperative behavior improved overall perceptions, but creativity ratings consistently favored human partners despite identical behavior. Identity bias persists.

LLMs & responsible AI Bias & fairness

Zero-shot stance detection in practice: insights on training, prompting, and decoding with a capable lightweight LLM

Rachith Aiyappa, Shruthi Senthilmani, Jisun An, Haewoon Kwak, Yong-Yeol Ahn

PeerJ Computer Science 2026 1 citation

Can LLMs do stance detection on tweets without any fine-tuning? Using FlanT5-XXL with SemEval 2016 and P-Stance datasets, we find zero-shot prompting matches or beats fine-tuned baselines. Performance varies with prompt phrasing, decoding strategy, and a measurable positivity bias, and we surface cases where the LLM over-commits to a stance when none is actually present.

LLMs & responsible AI

Cross-Cultural Differences in Public Discourse on COVID-19 Vaccination in the United States and South Korea: Cross-Sectional Analysis Using Natural Language Processing

Sangpil Youm, Sou Hyun Jang, Haewoon Kwak, Jaeyoung Choi, Yong Jeong Yi

JMIR Formative Research 2026

How does public discourse around COVID-19 vaccines differ between the United States and South Korea? Analyzing nearly 8,000 question-answer pairs from Quora and Naver Knowledge-iN, we apply NLP to surface topical and sentiment differences. Both countries share concerns about side effects and efficacy, but diverge sharply on which questions get most attention.

Digital health

If I Change Your Eyes, Are You Still You? Identity Thresholds in Deepfake Perception

Chaewon Kang, Youjin Lee, Haewoon Kwak, Jisun An, Jinyoung Han

CHI Poster, 2026 2026

Deepfake harm depends on the moment a viewer perceives the synthetic face as a specific victim, but where that threshold sits is undefined. With 102 participants and 162 manipulated stimuli, we find eyes are the dominant identity cue, that pairing eyes with one more feature triggers identity perception, and that human-AI misalignment peaks exactly at that threshold.

User behavior & engagement Online harm

Leadership as Coordination Control: Behavioral Signatures and the Recovery-Advantage Boundary in Multi-Agent LLM Teams

Haewoon Kwak

arXiv preprint arXiv:2606.19111, 2026 2026

Does a team of LLM agents need a leader? We turn transactional, transformational, and situational leadership into explicit controllers over a shared action set, tested across four task regimes and three model families. No controller dominates. Control pays off only when the round-0 majority is wrong and the task is recoverable.

LLMs & responsible AI

PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media

Zoher Kachwala, Bao Tran Truong, Rasika Muralidharan, Haewoon Kwak, Jisun An, Filippo Menczer

ACL 2026

Different online communities have different rules: what gets you banned from one subreddit may be the norm in another. PluRule is a multimodal, multilingual benchmark with 13,371 rule violations across 1,989 Reddit communities, 2,885 rules, and 9 languages. Even GPT-5.2 performs only slightly better than a trivial baseline, exposing pluralistic moderation as a fundamental challenge.

Online harm

Urban Vibrancy Embedding and Application on Traffic Prediction

Sumin Han, Jisun An, Dongman Lee

Open MIND 2026

Urban 'vibrancy' (the dynamic human activity in a place) is well-measured by floating-population data but rarely fed into traffic models. We use VAEs to compress floating-population data into Urban Vibrancy embeddings, then forecast future embeddings with LSTMs and feed them into traffic models (RNN, DCRNN, GTS, GMAN), improving accuracy and responsiveness across the board.

User behavior & engagement

When AI Becomes Routine: A Decade of Public AI Mediation in Korean Go Commentary

Haewoon Kwak

AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2026 2026

AlphaGo was a spectacle, but what happens once AI becomes ordinary? Across a decade of Korean Go commentary on YouTube, roughly 1,900 hours, winrate graphs fill 98 percent of late broadcast time while only 2.63 percent of sentences mention AI. The metric stays, the source label goes unsaid, and with it the hooks for questioning it.

User behavior & engagement LLMs & responsible AI

XChoice: Explainable Evaluation of AI-Human Alignment in LLM-based Constrained Choice Decision Making

Weihong Qi, Fan Huang, Rasika Muralidharan, Jisun An, Haewoon Kwak

arXiv preprint arXiv:2601.11286, 2026 2026

AI-human alignment is usually measured by outcome agreement (accuracy, F1). XChoice fits a mechanism-based decision model to both human and LLM choices, recovering interpretable parameters: which factors matter, how sensitive each is to constraints, what trade-offs get made. Applied to U.S. time-use data, XChoice reveals systematic misalignment concentrated in specific demographic subgroups.

LLMs & responsible AI

A semantic embedding space based on large language models for modelling human beliefs

Byunghwee Lee, Rachith Aiyappa, Yong-Yeol Ahn, Haewoon Kwak, Jisun An

Nature Human Behaviour, 2025 2025 16 citations

How do beliefs interconnect, and what drives a person to adopt new ones? We fine-tune large language models on online debate data to map thousands of beliefs into a semantic space where proximity reflects coherence. Position in that space predicts which beliefs an individual is likely to adopt next and quantifies cognitive dissonance via distance between existing and new beliefs.

LLMs & responsible AI Beliefs & opinions

PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI

Srija Mukhopadhyay, Sathwik Reddy, Shruthi Muthukumar, Jisun An, Ponnurangam Kumaraguru

arXiv preprint arXiv:2512.24848, 2025 2025 3 citations

Personalized AI agents must access private data (emails, chats, purchases) to be useful, but they can leak it. PrivacyBench is a benchmark with conversational scenarios containing embedded secrets. RAG assistants leak secrets in up to 26.6% of conversations; a privacy-aware prompt cuts this to 5.1%, but retrieval still pulls sensitive data indiscriminately.

LLMs & responsible AI

Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

Rasika Muralidharan, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2510.07488, 2025 2025 1 citation

What does team science say about multi-agent LLM systems? We test three core team-science variables (structure, diversity, interaction dynamics) across four reasoning tasks. Flat teams beat hierarchical ones, diversity has nuanced effects, and agents are overconfident before tasks but more reflective afterward, with both appreciation for collaboration and difficulty integrating.

LLMs & responsible AI

What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?

Joseph Malone, Rachith Aiyappa, Byunghwee Lee, Haewoon Kwak, Jisun An, Yong-Yeol Ahn

arXiv preprint arXiv:2511.18616, 2025 2025 1 citation

When an LLM predicts someone's beliefs, what matters more: who they are (demographics) or what else they believe (prior stances)? Testing open-weight LLMs on online-debate data under four conditions, we find both signals improve prediction, but their relative value varies sharply across belief domains.

LLMs & responsible AI Beliefs & opinions

A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societies

Weihong Qi, Fan Huang, Jisun An, Haewoon Kwak

2025 IEEE International Conference on Data Mining Workshops (ICDMW) 2025

Can LLMs realistically simulate public opinion across different cultures? Comparing DeepSeek-R1/V3 with Qwen2.5, GPT-4o, and Llama-3.3 against survey data from the U.S. (ANES) and China (Zuobiao), we find performance varies sharply by topic and demographic, and that all models over-generalize within demographic groups, defaulting to a 'representative' voice.

LLMs & responsible AI Beliefs & opinions

A Model for Monthly, Local-Level Airbnb Changes Using Public Dataset

Youngjun Park, Jisun An, Dongman Lee

International Conference on Tourism Research 2025

How do Airbnb listings shift month-to-month across a city's neighborhoods? Using public datasets covering Seoul from 2017 to 2019, we model monthly variation as a function of housing supply, lodging density, tourist-attraction proximity, and retail activity, achieving strong fit (avg pseudo R² of 0.79) and identifying which factors drive short-term rental dynamics in each district.

User behavior & engagement

Cognitive Linguistic Identity Fusion Score (CLIFS): A Scalable Cognition‐Informed Approach to Quantifying Identity Fusion from Text

Devin R. Wright, Jisun An, Yong-Yeol Ahn

arXiv preprint arXiv:2509.16813, 2025 2025

Identity fusion (the psychological merging of self with a group, ideology, or brand) has historically been measured with surveys. CLIFS uses cognitive linguistics and LLMs to detect implicit metaphors in text and score fusion automatically, matching survey instruments while scaling. As a proof of concept, it improves violence risk assessment by more than 240%.

User behavior & engagement

Enhancing Regional Airbnb Trend Forecasting Using LLM-Based Embeddings of Accessibility and Human Mobility

Hongju Lee, Youngjun Park, Jisun An, Dongman Lee

arXiv preprint arXiv:2511.14248, 2025 2025

Forecasting regional Airbnb activity matters for housing policy, but typical models look at single listings at single time points. We forecast Revenue, Reservation Days, and Number of Reservations at the regional level by combining listing features with LLM-generated embeddings of urban accessibility and human mobility, cutting RMSE and MAE by roughly 48% versus standard baselines.

LLMs & responsible AI

Measuring capital with spatial media: How online popularity on Instagram shapes land value patterns in Seoul

Dasom Hong, Sumin Han, Youngjun Park, Jisun An, Dongman Lee

Area 2025

Do social-media check-ins reshape urban land value? Modeling three Seoul neighborhoods (Yeonnam-dong, Seongsu-dong, Gyeongridan-gil) that gentrified after going viral on Instagram, we show that adding online-popularity features to a spatial capital model outperforms physical-layout-only baselines, quantifying how Instagram visibility now factors into urban property prices.

User behavior & engagement

Quantifying Gender Stereotypes in Japan between 1900 and 1999 with Word Embeddings

Shintaro Sakai, Haewoon Kwak, Jisun An, Akira Matsui

arXiv preprint arXiv:2510.03905, 2025 2025

How did gender stereotypes shift in Japan across the 20th century? We train 100 year-specific word embeddings on Japanese corpora from 1900 to 1999 and trace stereotype strength in the Home, Work, and Politics domains. Work and Politics became more female-stereotyped over time, but Home did too, suggesting women came to be seen as occupying multiple roles rather than swapping them.

Bias & fairness

Public Perception of Generative AI on Twitter: An Empirical Study Based on Occupation and Usage

Kunihiro Miyazaki, Taichi Murayama, Takayuki Uchiba, Jisun An, Haewoon Kwak

EPJ Data Science, 2024 2024 94 citations

What does the public actually think of generative AI? We analyze 3M tweets from 2019 to 2023 and find broad interest across occupations, not just tech. Sentiment is generally positive and tracks exposure, with one exception: illustrators are notably negative, reflecting concerns over training-data ethics.

User behavior & engagement LLMs & responsible AI

A Survey on Predicting the Factuality and the Bias of News Media

Preslav Nakov, Jisun An, Haewoon Kwak, Muhammad Arslan Manzoor, Zain Muhammad Mujahid, Husrev Taha Sencar

ACL Findings, 2024 2024 38 citations

Fact-checking every story is impossible. Instead, this survey reviews how to profile entire news outlets, so any article can be flagged the moment it appears. We argue that factuality and political bias should be modeled jointly rather than separately, and survey state-of-the-art across text, social context, and beyond.

News & journalism Bias & fairness

ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?

Fan Huang, Haewoon Kwak, Kunwoo Park, Jisun An

LREC-COLING, 2024 2024 30 citations

As AI explains its own decisions in natural language, who should grade those explanations? We compare ChatGPT and human judgments on informativeness and clarity across binary, ternary, and 7-point scales, finding ChatGPT aligns well with humans on coarse-grained ratings, with paired comparison and dynamic prompting further improving alignment.

LLMs & responsible AI

The Impact of Toxic Trolling Comments on Anti-vaccine YouTube Videos

Kunihiro Miyazaki, Takayuki Uchiba, Haewoon Kwak, Jisun An, Kazutoshi Sasahara

Scientific Reports, 2024 2024 23 citations

Anti-vaccine YouTube comments are often dismissed as background noise, but do they actually push viewers toward hesitancy? We find that highly-liked toxic comments measurably increase fear in later comments, with contagion running in both directions. The result has implications for public health messaging and how moderators handle 'just trolling.'

Online harm

X-posing Free Speech: Examining the Impact of Moderation Relaxation on Online Social Networks

Arvindh Arun, Saurav Chhatani, Jisun An, Ponnurangam Kumaraguru

WOAH 2024 15 citations

What happened to hate speech on Twitter after Elon Musk's takeover and the relaxation of moderation? On 10 million tweets, we document significant increases in hate targeting LGBTQ+ users and liberals, with cohesive hate communities forming around influential 'bridge' users. Ironically, embracing free-speech principles enabled hate against the very concept of free speech itself.

Online harm

ToBlend: Token-Level Blending With an Ensemble of LLMs to Attack AI-Generated Text Detection

Fan Huang, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2402.11167, 2024 2024 11 citations

Detectors of AI-generated text are quietly competing against attacks that paraphrase or swap words. ToBlend goes further: it samples tokens randomly from an ensemble of LLMs, producing text that drops most detectors' performance significantly while human raters confirm quality holds. We also release a fine-tuned Llama-3.1 detector tuned to catch ToBlend output.

LLMs & responsible AI

Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance

Rachith Aiyappa, Shruthi Senthilmani, Jisun An, Haewoon Kwak, Yong-Yeol Ahn

arXiv preprint arXiv:2403.00236, 2024 2024 8 citations

Can LLMs do stance detection on tweets without any fine-tuning? Using FlanT5-XXL with SemEval 2016 and P-Stance, zero-shot prompting matches or beats fine-tuned baselines. We trace how performance varies with prompt phrasing, decoding strategy, and prompt perplexity, and identify a positivity bias that partially explains performance differences across decoding strategies.

LLMs & responsible AI

Enhancing stance classification on social media using quantified moral foundations

Hong Zhang, Quoc-Nam Nguyen, Prasanta Bhattacharya, Wei Gao, Liang Ze Wong, Brandon Siyuan Loh, Joseph J. P. Simons, Jisun An

arXiv preprint arXiv:2310.09848, 2024 2024 7 citations

Stance detection usually looks at message text alone. We add Moral Foundations Theory features (theoretically-derived dimensions of an individual's moral concerns) on top of standard semantic features, using both traditional ML and LLMs. Encoding moral foundations improves classification and illuminates which moral foundations associate with which online stances.

Beliefs & opinions

Token-Ensemble Text Generation: On Attacking the Automatic AI-Generated Text Detection

Fan Huang, Haewoon Kwak, Jisun An

arXiv preprint arXiv:2402.11167 2024 5 citations

Can AI-text detectors survive smarter adversaries? We propose ToBlend, which generates text by randomly sampling next tokens from multiple candidate LLMs. The blended output sharply degrades most mainstream detectors while staying fluent to human evaluators, exposing a robustness gap and motivating a fine-tuned Llama3.1 defender.

LLMs & responsible AI

Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity

Zoher Kachwala, Jisun An, Haewoon Kwak, Filippo Menczer

NAACL Findings, 2024 2024 4 citations

Comparing knowledge graphs that represent sentence meaning is harder than it sounds: existing metrics are slow and miss semantic nuance. Rematch is a new AMR similarity metric that ranks first on semantic benchmarks (1 to 5 percentage points over state of the art) and runs five times faster.

Networks & influence

Multiple Areal Feature Aware Transportation Demand Prediction

Sumin Han, Jisun An, Y. Park, Suji Kim, Kitae Jang, Dongman Lee

arXiv preprint arXiv:2408.12890, 2024 2024 1 citation

Short-term transportation demand prediction guides scheduling, fleet sizing, and transit-network design. Most models use one or two areal features; cities need many (land use, sociodemographics, POI distribution). Our ST-MFGCRN model fuses multiple areal features via sentinel attention, beating state-of-the-art baselines by 7 to 8% on BusDJ and TaxiBJ datasets.

User behavior & engagement

Spatio-Temporal Road Traffic Prediction using Real-time Regional Knowledge

Sumin Han, Jisun An, Dongman Lee

arXiv preprint arXiv:2408.12882, 2024 2024

Mid-term road traffic prediction (a few hours out) matters for car-sharing and ride-hailing, but existing models focus on short-term effects between adjacent roads. We embed real-time regional knowledge (POIs, satellite images, LTE access traces) into a road-level attention model via dynamic convolution and bipartite spatial-transform attention, beating baseline approaches.

User behavior & engagement

Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech

Fan Huang, Haewoon Kwak, Jisun An

WWW Companion, 2023 2023 477 citations

Implicit hate speech is hard for both detectors and explainers. We ask whether ChatGPT can produce useful natural-language explanations of why a subtle hateful post is hateful, and compare its output to human annotators through user studies. Promising for routine cases, but with consistent limitations on harder ones.

LLMs & responsible AI Online harm

Can We Trust the Evaluation on ChatGPT?

Rachith Aiyappa, Jisun An, Haewoon Kwak, Yong-Yeol Ahn

TrustNLP (Collocated with ACL), 2023 2023 142 citations

Most ChatGPT evaluations claim impressive numbers, but how trustworthy are they? Through a stance-detection case study, we show that training-set contamination silently inflates ChatGPT's apparent performance. We argue that fair evaluation of closed, continuously-trained models needs new conventions.

LLMs & responsible AI

Chain of Explanation: New Prompting Method to Generate Higher Quality Natural Language Explanation for Implicit Hate Speech

Fan Huang, Haewoon Kwak, Jisun An

WWW Companion, 2023 2023 64 citations

Generating clear explanations of why a post is hateful is hard, especially when hate is implicit. Our Chain of Explanation prompting method feeds the model heuristic words and the targeted group up front, lifting BLEU scores from 44.0 to 62.3 and producing more informative explanations on human evaluation.

LLMs & responsible AI Online harm

YouNICon: YouTube's CommuNIty of Conspiracy Videos

Shaoyi Liaw, Fan Huang, Fabricio Benevenuto, Haewoon Kwak, Jisun An

ICWSM Dataset, 2023 2023 21 citations

Conspiracy theories are widely propagated on YouTube, yet the data needed to study them at scale is limited. YouNICon is a dataset of videos from channels previously identified as suspicious, supporting both conspiracy-theory detection and topic classification. The dataset enables studying trends in conspiracy theories and how individuals interact with such channels.

Online harm

'This is Fake News': Characterizing the Spontaneous Debunking from Twitter Users to COVID-19 False Information

Kunihiro Miyazaki, Takayuki Uchiba, Kenji Tanaka, Jisun An, Haewoon Kwak, Kazutoshi Sasahara

AAAI ICWSM, 2023 2023 20 citations

Fact-checkers can't keep up with COVID-19 misinformation. We study spontaneous debunking by regular Twitter users: which fake tweets attract corrections, and who corrects them. Building an annotated dataset and a classifier, we find most fake tweets are never debunked, spontaneous corrections are slower than other replies, and political topics show a clear partisan asymmetry.

News & journalism Digital health

Wearing Masks Implies Refuting Trump?: Towards Target-specific User Stance Prediction across Events in COVID-19 and US Election 2020

Hong Zhang, Haewoon Kwak, Wei Gao, Jisun An

ACM WebSci, 2023 2023 14 citations

If you know someone's view on masks, can you predict their view on Trump? Across three seemingly unrelated COVID-era topics, we train stance-detection models on Twitter and find that opinions cluster predictably: past stances on one issue strongly forecast future stances on another.

Beliefs & opinions Digital health

Political Honeymoon Effect on Social Media: Characterizing Social Media Reaction to the Changes of Prime Minister in Japan

Kunihiro Miyazaki, Taichi Murayama, Akira Matsui, Masaru Nishikawa, Takayuki Uchiba, Haewoon Kwak, Jisun An

ACM WebSci, 2023 2023 7 citations

Newly-elected leaders typically enjoy a brief 'honeymoon' of high approval. We test this on social media by analyzing 6.6M tweets across 8 Japanese prime-minister transitions over 15 years. The honeymoon does appear, but the user base flips dramatically with each change, and sentiment patterns vary by topic and PM.

User behavior & engagement

What really matters?: characterising and predicting user engagement of news postings using multiple platforms, sentiments and topics

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen

Behaviour & Information Technology, 2022 2022 61 citations

What makes a news post go viral, and on which platform? Analyzing 3M posts from 53 outlets and 50M comments across 5 platforms over 8 months, we trace how sentiment and topic shape four levels of engagement (views, likes, comments, cross-platform sharing) and predict cross-platform diffusion with up to 83% F1.

News & journalism

Storm the Capitol: Linking Offline Political Speech and Online Twitter Extra-Representational Participation on QAnon and the January 6 Insurrection

Claire Seungeun Lee, Juan Merizalde, John D. Colautti, Jisun An and Haewoon Kwak

Frontiers in Sociology, 2022 2022 49 citations

How does offline political speech translate into online mobilization? We combine the text of Trump's speeches leading up to January 6, 2021 with Twitter data from QAnon hashtags over the same three days, using mixed quantitative and qualitative analysis. The results show measurable links between specific addresses and online participation.

Online harm

Measuring 9 Emotions of News Posts from 8 News Organizations across 4 Social Media Platforms for 8 Months

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen

ACM Transactions on Social Computing, 2022 2022 33 citations

How do emotions in news posts ripple into audience reactions? Across 133K posts and 2.8M comments from 8 outlets on 4 platforms over 8 months, we score 9 emotions (Plutchik's wheel) and predict reader emotional engagement before and after a post goes live, reaching over 70% F1 for some emotions and revealing systematic platform differences.

News & journalism

Understanding Toxicity Triggers on Reddit in the Context of Singapore

Yun Yu Chong, Haewoon Kwak

Proceedings of the 16th International AAAI Conference on Web and Social Media (ICWSM), 2022 (short) 2022 24 citations

Most toxicity research focuses on Western online communities. We extend trigger detection to an Asian context using Reddit conversations from Singapore, showing both that detection is feasible and that the triggers themselves can differ markedly from Western patterns. A reminder that 'toxic' is culturally specific.

Online harm

Modeling Political Activism around Gun Debate via Social Media

Yelena Mejova, Jisun An, Gianmarco De Francisci Morales, Haewoon Kwak

ACM Transactions on Social Computing, 2022 2022 16 citations

Can social media reveal who will engage politically on contested issues? On the U.S. gun debate, we use Twitter content and network structure to classify user stance, predict state-level gun policy, and forecast individual offline political action. Network features (especially the relative size of each side) emerge as strong signals alongside content and demographics.

Beliefs & opinions

Who Is Missing? Characterizing the Participation of Different Demographic Groups in a Korean Nationwide Daily Conversation Corpus

Haewoon Kwak, Jisun An, Kunwoo Park

Proceedings of the 16th International AAAI Conference on Web and Social Media (ICWSM), 2022 (short) 2022 2 citations

Conversation corpora power most interactive AI, but who actually contributed to them? Analyzing a Korean nationwide daily conversation corpus from the National Institute of Korean Language, we characterize the participation of different demographic groups (age and sex) and identify the voices that are over- and under-represented in this widely-used resource.

LLMs & responsible AI Bias & fairness

You Have Earned a Trophy: Characterize In-Game Achievements and Their Completions

Haewoon Kwak

ACM WebSci, 2022 2022 2 citations

PlayStation 'trophies' are one of gaming's most-discussed achievement systems, but no large-scale study exists. We build the first complete dataset of PlayStation games and their trophies, then characterize them from both the designer's and the player's perspectives, uncovering patterns in how trophies are designed and which ones players actually complete.

User behavior & engagement

MAANG? MANGA? Characterizing Spontaneous Ideation Contest on Social Media

Kunihiro Miyazaki, Takayuki Uchiba, Haewoon Kwak, Jisun An

IEEE BigData, 2022 (short) 2022

When Facebook became Meta, Twitter users spontaneously started proposing new acronyms for Big Tech ('MAANG,' 'MANGA,' and many more). We trace how candidates were proposed, spread, and judged, finding that early posters had fewer followers than later spreaders, rankings shifted slowly, and replies introduced most new ideas at the first depth.

Networks & influence

FrameAxis: characterizing microframe bias and intensity with word embedding

Haewoon Kwak, Jisun An, Elise Jing, Yong-Yeol Ahn

PeerJ Computer Science 7:e644, 2021 2021 54 citations

Framing nudges readers toward conclusions without making explicit arguments. FrameAxis identifies the semantic axes ('microframes') that a document leans on, using word embeddings rather than manual annotation. The method scales to large datasets and exposes bias and intensity from restaurant reviews to political news.

Bias & fairness News & journalism

Precision Public Health Campaign: Delivering Persuasive Messages to Relevant Segments Through Targeted Advertisements on Social Media

Jisun An, Haewoon Kwak, Hanya M Qureshi, Ingmar Weber

JMIR Form Res 2021;5(9):e22313, 2021 2021 43 citations

Health campaigns usually broadcast one message to everyone. We propose a five-stage framework for precision public health: define goal and metrics, segment the audience, tailor a message per segment, pilot, run, and evaluate using social-media ad-targeting tools. Two case studies demonstrate the framework in practice.

Beliefs & opinions Digital health

Predicting Anti-Asian Hateful Users on Twitter during COVID-19

Jisun An, Haewoon Kwak, Claire Seungeun Lee, Bogang Jun, Yong-Yeol Ahn

Findings of the Association for Computational Linguistics EMNLP 2021 2021 43 citations

Can we tell, before COVID-19 began, who would later post anti-Asian slurs on Twitter? Comparing users who did and didn't post such slurs after the pandemic's onset, we identify pre-pandemic features that predict later hate behavior, pointing to the role of news media and polarized communication networks in priming hostility.

Online harm

Populist Supporters on Reddit: A Comparison of Content and Behavioral Patterns Within Publics of Supporters of Donald Trump and Hillary Clinton

Andreas Jungherr, Oliver Posegga, Jisun An

Social Science Computer Review. March 2021. 2021 37 citations

If digital media empower populist movements, can we see it in distinct usage patterns? Comparing /r/The_Donald and /r/hillaryclinton during the 2016 election, we find Trump supporters expressed more anti-elitism and exclusion (but not 'people-centrism'), posted more, used distinctive vocabulary, and built tighter community ties.

Beliefs & opinions

How-to Present News on Social Media: A Causal Analysis of Editing News Headlines for Boosting User Engagement

Kunwoo Park, Haewoon Kwak, Jisun An, Sanjay Chawla

Proceedings of the 15th International AAAI Conference on Web and Social Media (ICWSM), 2021 2021 24 citations

When media outlets share an article on Twitter, the edited tweet often departs sharply from the original headline. Does that editing pay off? Using a parallel corpus of articles and tweets from 8 outlets, we apply propensity-score matching to estimate the causal effect of editing styles on engagement, surfacing both common and outlet-specific patterns.

News & journalism

Persona Perception Scale: Development and Exploratory Validation of an Instrument for Evaluating Individuals’ Perceptions of Personas

Joni Salminen, Joao M. Santos, Haewoon Kwak, Jisun An, Soon-gyo Jung, Bernard J. Jansen

International Journal of Human-Computer Studies, 2020 2020 108 citations

Personas are widely used but hard to evaluate because no validated measurement instrument existed. We develop the Persona Perception Scale (PPS), drawing on social psychology, persona studies, and HCI, and validate it with 412 respondents. The scale lets researchers measure how individuals perceive personas (credibility, willingness to use) and improve persona adoption.

User behavior & engagement

A Systematic Media Frame Analysis of 1.5 Million New York Times Articles from 2000 to 2017

Haewoon Kwak, Jisun An, Yong-Yeol Ahn

Proceedings of the 12th ACM Conference on Web Science (WebSci), 2020 2020 72 citations

Framing is hard to measure at scale because frames usually need hand-coding. We trained a state-of-the-art frame classifier and applied it to 1.5 million New York Times articles spanning 2000 to 2017. Short-term frame fluctuations track major events; long-term, the 'cultural identity' frame rises sharply, and a case study on mass shootings reveals three distinct patterns.

News & journalism

What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context

Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, Preslav Nakov

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) (2020) 2020 72 citations

Fact-checking every claim is impossible, so a complementary approach is to profile entire news outlets and flag everything they publish. We argue that bias and factuality should be modeled jointly using both the text the outlet produces and the social context it lives in, and we review approaches that combine the two.

News & journalism Bias & fairness

Are These Comments Triggering? Predicting Triggers of Toxicity in Online Discussions

Hind Almerekhi, Haewoon Kwak, Bernard Jim Jansen, Joni Salminen (short)

Proceedings of The Web Conference (WWW), 2020 2020 71 citations

Toxic replies often have an innocent-looking trigger somewhere up the thread. We define a trigger as a non-toxic comment that provokes a toxic reply, then train a neural model on shifts in sentiment, topic, and conversational context. The model identifies triggers with a 0.87 ROC-AUC, suggesting a way to head off escalation before it happens.

Online harm

Empirical Evaluation of Three Common Assumptions in Building Political Media Bias Datasets

Soumen Ganguly, Juhi Kulshrestha, Jisun An, Haewoon Kwak

Proceedings of the 14th International AAAI Conference on Web and Social Media (ICWSM), 2020 2020 34 citations

Many political-bias datasets rest on three common assumptions: labelers' politics don't bias their labels, articles follow their outlet's leaning, and an outlet's leaning is stable across topics. We test all three on a manually annotated dataset and find each can fail even at small scale, a warning for how bias datasets are constructed and trusted.

Bias & fairness News & journalism

“Trust Me, I Have a Ph.D.”: A Propensity Score Analysis on the Halo Effect of Disclosing One's Offline Social Status in Online Communities

Kunwoo Park, Haewoon Kwak, Hyunho Song, Meeyoung Cha

Proceedings of the 14th International AAAI Conference on Web and Social Media (ICWSM), 2020 2020 20 citations

Does showing your offline credentials affect how people respond to you online? On two Reddit science communities that adopted 'degree flairs,' we find the halo effect varies by community culture: in the strict peer-reviewed subreddit, any degree boosts votes and discussion; in the casual one, only PhD or MD flairs do.

User behavior & engagement

Identifying and Characterizing Alternative News Media on Facebook

Samuel S Guimarães, Julio CS Reis, Lucas Lima, Filipe N Ribeiro, Marisa Vasconcelos, Jisun An, Haewoon Kwak, Fabrício Benevenuto

IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2020 2020 9 citations

We propose a graph-based semi-supervised method to measure the political bias of Facebook news pages across many countries, separating alternative media, mainstream media, and public-figure pages. Validated on a U.S. dataset and applied to Brazil, where right-wing pages dominate generally but alternative news media break the pattern.

News & journalism

Going beyond accuracy: estimating homophily in social networks using predictions

George Berry, Antonio Sirianni, Ingmar Weber, Jisun An, Michael Macy (preprint)

arXiv preprint arXiv:2001.11171, 2020 2020 5 citations

Social-network studies often estimate homophily from predicted attributes (gender or ethnicity from names), but high prediction accuracy doesn't guarantee accurate homophily estimates. We show estimation works only when dyad-level residuals cancel out, and propose an 'ego-alter' approach that beats standard node and dyad classifiers.

Networks & influence

Discursive Power in Contemporary Media Systems: A Comparative Framework

Andreas Jungherr, Oliver Posegga, Jisun An

The International Journal of Press/Politics, 24(4), 2019 2019 249 citations

Today's news ecosystems mix legacy outlets, platforms, influencers, and amateurs in ways that resist easy comparison. We propose 'discursive power' as a measurable concept: the ability to introduce, amplify, and sustain topics, frames, and speakers in interconnected communication spaces, enabling empirical comparative work on hybrid media systems.

News & journalism

View, Like, Comment, Post: Analyzing User Engagement by Topic at 4 Levels across 5 Social Media Platforms for 53 News Organizations

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen

Proceedings of the 13th International AAAI Conference on Web and Social Media (ICWSM), 2019 2019 160 citations

Different platforms surface different topics: how does that shape engagement? Across 3.16M posts from 53 news organizations on Facebook, Instagram, Twitter, YouTube, and Reddit over 8 months, we map topic distribution by platform and topic-driven engagement at four levels, finding that cross-platform sharing of any given article is predictable with around 80% precision.

News & journalism

Political Discussions in Homogeneous and Cross-Cutting Communication Spaces

Jisun An, Haewoon Kwak, Oliver Posegga, Andreas Jungherr

Proceedings of the 13th International AAAI Conference on Web and Social Media (ICWSM), 2019 2019 80 citations

Online political talk happens in two very different settings: groups of like-minded users, and mixed spaces where opposing views meet. We use Reddit to develop an analytical template comparing interaction and linguistic patterns within and between politically homogeneous and cross-cutting communication spaces, finding markedly different behaviors in each.

Beliefs & opinions

Detecting Toxicity Triggers in Online Discussions

Hind Almerekhi, Haewoon Kwak, Bernard Jim Jansen, Joni Salminen (poster)

Proceedings of the 30th ACM Conference on Hypertext and Social Media (HT), 2019 2019 75 citations

What sparks a toxic exchange in online discussions? We define toxicity 'triggers' formally, then build an LSTM model combining textual features with measures of topic and sentiment shift across interactions. Trained on diverse Reddit communities, the model detects triggers with 82.5% average accuracy.

Online harm

Predicting Audience Engagement Across Social Media Platforms in the News Domain

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen

Proceedings of Social Informatics (SocInfo), 2019 2019 21 citations

Among other findings, results show that content with high engagement on one platform does not guarantee high engagement on another platform, even when news outlets use similar cross-platform posts; however, for some content, cross-sharing posts on a platform will increase overall audience engagement on another platform.

News & journalism

The Challenges of Creating Engaging Content: Results from a Focus Group Study of a Popular News Media Organization

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen (Extended Abstracts)

Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI), 2019 2019 21 citations

Creating 'engaging' content across social-media platforms is a daily grind for editors at serious news organizations, with no clear playbook. Through a focus-group study at a major news org, we find engagement work is effort- and time-intensive across three platforms, and we sketch a system design that would help editors select and adapt engaging passages from articles.

News & journalism

Social media mining for journalism

Arkaitz Zubiaga, Bahareh Heravi, Jisun An, Haewoon Kwak (Guest editorial)

Online Information Review, 2019 2019 12 citations

Social media now captures and announces breaking news faster than traditional outlets, reshaping the journalistic landscape. We survey how social-media mining is being adopted by professional journalists for newsgathering (drawing on citizen reporting and updates) and as an additional channel for news dissemination, opening both opportunities and challenges for newsrooms.

News & journalism

Tanbih: Get To Know What You Are Reading

Yifan Zhang, Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James Glass, Preslav Nakov (demo)

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019 2019 10 citations

Behind a news story sits a much larger context: the outlet's general factuality, partisanship, propaganda style, ideology, framing, and stance on key topics. Tanbih is a news aggregator that groups stories by event and generates these media profiles automatically, plus per-article propaganda detection and stance with respect to specific controversial claims.

News & journalism Bias & fairness

Stylistic Features Usage: Similarities and Differences Using Multiple Social Networks

Kholoud Khalil Aldous, Jisun An, Bernard J. Jansen

Proceedings of Social Informatics (SocInfo), 2019 2019 9 citations

We analyze ten common stylistic features indicative of user engagement for news postings on multiple social media platforms. We display the stylistic features usage differences of news posts from various news sources. Results show that there are differences in the usage of stylistic features across social media platforms (Facebook, Instagram, Twitter, and YouTube).

Networks & influence

What is Gab? A Bastion of Free Speech or an Alt-Right Echo Chamber?

Savvas Zannettou, Barry Bradlyn, Emiliano De Cristofaro, Haewoon Kwak, Michael Sirivianos, Gianluca Stringhini, Jeremy Blackburn

Companion Proceedings of the The Web Conference (WWW), 2018 2018 407 citations

Gab launched in 2016 as a free-speech alternative to Twitter, attracting users banned from other platforms. We analyze 22M posts from 336K users (Aug 2016 to Jan 2018) and find Gab is mostly used for news and world events, attracts alt-right users, conspiracy theorists, and trolls, with hate-speech prevalence much higher than Twitter but lower than 4chan's /pol/.

Online harm Bias & fairness

Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for Identifying and Classifying Hate in Online News Media

Joni Salminen, Hind Almerekhi, Milica Milenković, Soon-gyo Jung, Jisun An, Haewoon Kwak, Bernard J. Jansen

Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM), 2018 2018 210 citations

Automatic hate-speech detection is hard partly because 'hate' covers many distinct types and targets. We hand-labeled 5,143 hateful YouTube and Facebook comments, built a granular taxonomy capturing both the kind and target of hate, and trained classifiers to detect and categorize new comments. A linear SVM with TF-IDF reaches 0.79 average F1.

Online harm

Imaginary People Representing Real Numbers: Generating Personas from Online Social Media Data

Jisun An, Haewoon Kwak, Soon-gyo Jung, Joni Salminen, M. Admad, Bernard J. Jansen

ACM Transactions on the Web, 12(4), 2018 2018 182 citations

Marketing personas are hand-crafted, slow, and quickly outdated. We automate them from social-media data through behavioral segmentation, demographic grouping, and auto-generated names and photos. Validated on 30M viewer interactions across 198 countries and 4,200 videos, the personas predict content preference and remain stable over time.

User behavior & engagement

SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment

Jisun An, Haewoon Kwak, Yong-Yeol Ahn

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), 2018 2018 108 citations

Word meanings shift across communities and contexts, but most NLP tools collapse that nuance into a single sentiment dimension. SemAxis is a lightweight framework that characterizes words along arbitrary semantic axes in vector space, capturing community-specific meaning. On sentiment specifically, SemAxis beats state-of-the-art methods for building domain-specific lexicons.

User behavior & engagement

Assessing the Accuracy of Four Popular Face Recognition Tools for Inferring Gender, Age, and Race

Soon-gyo Jung, Jisun An, Haewoon Kwak, Joni Salminen, Bernard Jim Jansen

Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM), 2018 (short) 2018 95 citations

When computational social scientists infer demographics from photos, which face-recognition tool can they trust? Testing Face++, IBM Bluemix, AWS Rekognition, and Microsoft Azure across multiple datasets, we find gender inference works (above 90%, except IBM), race inference is offered by only one tool, and age is consistently inaccurate. Triangulation and manual checks recommended.

Bias & fairness

“Is More Better?”: Impact of Multiple Photos on Perception of Persona Profiles

Joni Salminen, Lene Nielsen, Soon-Gyo Jung, Jisun An, Haewoon Kwak, Bernard J. Jansen

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI), 2018 2018 89 citations

Does adding photos to a marketing persona make it more informative? Eye-tracking and interviews compare a single headshot, a headshot plus contextual scenes of the persona, and a headshot plus images of different people. Contextual images help; mixed people confuse readers, and viewers' own preconceptions shape what they see.

User behavior & engagement

What We Read, What We Search: Media Attention and Public Attention among 193 Countries

Haewoon Kwak, Jisun An, Joni Salminen, Soon-Gyo Jung, Bernard J. Jansen.

Proceedings of the 2018 World Wide Web Conference (WWW), 2018 2018 40 citations

Do people care about the same countries their national news covers? Across 193 countries over 14 months, we find that media and public attention align regionally but diverge by topic. In 63% of countries, the two attend to entirely different countries, suggesting audiences seek information their local outlets are not providing.

User behavior & engagement

Fixation and Confusion: Investigating Eye-tracking Participants' Exposure to Information in Personas

Joni Salminen, Bernard J. Jansen, Jisun An, Soon-Gyo Jung, Lene Nielsen, Haewoon Kwak

Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR), 2018 2018 31 citations

What confuses users of persona profiles? In a 29-participant eye-tracking and think-aloud study with three persona layouts, we measure where users fixate, what they say, and how the two relate. Inconsistencies between persona elements (especially between photos and quoted social-media voices) drive most confusion. We propose a 'confusion ratio' to flag problematic profile elements.

User behavior & engagement

Identifying Regional Trends in Avatar Customization

Peter Mawhorter, Sercan Şengün, Haewoon Kwak, D. Fox Harrell

IEEE Transactions on Games, 10(2), 2018 2018 4 citations

Avatars are a quiet form of self-expression, but the values built into avatar systems are rarely studied. Using deep learning and clustering on roughly 100,000 avatars from a major creation platform, we surface regional trends: realistic (US, mirroring demographics), idealistic (Japan, reflecting media tropes), and creative (community-spread), and link each to user activity.

User behavior & engagement

Persona Generation from Aggregated Social Media Data

Soon-Gyo Jung, Jisun An, Haewoon Kwak, Moeed Ahmad, Lene Nielsen, Bernard J. Jansen (Extended Abstract)

Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI), 2017 2017 126 citations

We demonstrate a real-time persona-generation pipeline on a YouTube channel of a global media corporation: 30M+ interactions, 181 countries, 4,200+ digital products. The system identifies distinct, impactful user segments and decorates each with auto-generated names, photos, and personal attributes, producing believable personas from real-time online data.

User behavior & engagement

Achievement and Friends: Key Factors of Player Retention Vary Across Player Levels in Online Multiplayer Games

Kunwoo Park, Meeyoung Cha, Haewoon Kwak, Kuan-Ta Chen

Proceedings of the 26th International Conference on World Wide Web (WWW) Companion, 2017 2017 56 citations

Why do players stay or leave a game? Analyzing in-game logs of 51,104 players in an online multiplayer game across their virtual 'life phases,' we find what predicts retention shifts with progression: achievement features matter most early and mid-game, but at the highest level, social ties become the strongest predictor of who keeps playing.

User behavior & engagement

Generating Cultural Personas from Social Data: A Perspective of Middle Eastern Users

J. Salminen, S. Sengün, H. Kwak, B. Jansen, J. An, S. Jung, S. Vieweg, D. F. Harrell

Proceedings of the 5th International Conference on Future Internet of Things and Cloud Workshops, 2017 2017 47 citations

We adapt automatic persona generation for Middle Eastern social-media users, blending computational analysis of millions of YouTube interactions with interview data to enrich personas. The result: culturally adapted personas that capture how different demographic groups in the region actually engage with content, and a methodology that respects cultural context.

User behavior & engagement

Personas for Content Creators via Decomposed Aggregate Audience Statistics

Jisun An, Haewoon Kwak, Bernard J. Jansen (short)

Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2017 2017 44 citations

Content creators distributing on online platforms need to understand their audiences. We generate personas from online user data using non-negative matrix factorization to identify user segments, then add names and photos for personality. The result is data-driven personas that represent real audience groups, replacing manually gathered data.

User behavior & engagement

Demographics of News Sharing in the US Twittersphere

Julio Reis, Haewoon Kwak, and Jisun An, Johnnatan Messias, Fabrıcio Benevenuto

Proceedings of the 28th ACM Conference on Hypertext and Social Media (HT), 2017 2017 43 citations

Who shares news on Twitter, and what does that mean for what most readers see? In the first detailed study of news sharers' demographics, content, and audience, we find that male and white users dominate news sharing, biasing the visible mix toward those groups' interests, with implications for personalized news design.

News & journalism

Who Are Your Users? Comparing Media Professionals' Preconception of Users to Data-Driven Personas

Lene Nielsen, Soon-Gyo Jung, Jisun An, Joni Salminen, Haewoon Kwak, Bernard J. Jansen

Proceedings of the 29th Australian Conference on Computer-Human Interaction (OZCHI), 2017 2017 30 citations

Personas are supposed to align cross-team user understanding, but do they? Through 16 qualitative interviews with media producers at Al Jazeera English, we find understandings are highly varied, built from a mix of personal experience, self-projection, assumptions, and company data, and only sometimes match the data-driven personas.

User behavior & engagement

Culturally-Grounded Analysis of Everyday Creativity in Social Media: A Case Study in Qatari Context

D. Fox Harrell, Sarah Vieweg, Haewoon Kwak, Chong-U Lim, Sercan Sengun, Ali Jahanian, Pablo Ortiz

Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C&C), 2017 2017 13 citations

How do MENA-region users (whose cultures rarely shape platform design) creatively adapt social media to express identity? Combining computational analysis of 42,000+ Qatari social profiles with 255 qualitative profile analyses and interviews, we identify five distinct values driving virtual-identity creativity, and propose guidelines for systems that respect those values.

User behavior & engagement

What Gets Media Attention and How Media Attention Evolves Over Time - Large-scale Empirical Evidence from 196 Countries

Jisun An, Haewoon Kwak (short)

Proceedings of the 11th International AAAI Conference on Web and Social Media (ICWSM), 2017 2017 12 citations

Why do some news topics get sustained media attention while others fade? Most prior work was limited to a few countries or topics for lack of global longitudinal data. We use large-scale data from 196 countries to provide empirical analyses of how media attention forms, peaks, and dissipates over time.

User behavior & engagement

I Would Not Plant Apple Trees If the World Will Be Wiped: Analyzing Hundreds of Millions of Behavioral Records of Players During an MMORPG Beta Test

Ah Reum Kang, Jeremy Blackburn, Haewoon Kwak, Huy Kang Kim

Proceedings of the 26th International Conference on World Wide Web (WWW) Companion, 2017 2017 11 citations

What do people do when they know their work will be erased? Using 270 million records from the final days of the ArcheAge MMO closed beta (where all character data was about to be deleted), we find no broad antisocial collapse, but a sharp drop in quest completion, leveling, and ability changes. Most players stopped planting their proverbial apple trees.

User behavior & engagement

Multidimensional Analysis of the News Consumption of Different Demographic Groups on a Nationwide Scale

Jisun An, Haewoon Kwak

Proceedings of Social Informatics (SocInfo), 2017 2017 10 citations

How differently do men and women, young and old, consume news? Examining 103,133 most-popular articles for different demographic groups in Daum News (Korea's second-largest portal) over 2015, we measure differences at four levels: items, sections, topics, and subtopics. Demographic differences appear most sharply at the subtopic level, not at the broader section or topic level.

News & journalism

Data-driven Approach to Measuring the Level of Press Freedom Using Media Attention Diversity from Unfiltered News

Jisun An, Haewoon Kwak

Proceedings of the ICWSM Workshop on NEws and publiC Opinion (NECO), 2017 2017 7 citations

The Press Freedom Index, published annually by Reporters Without Borders, is invaluable but expensive and slow because it's survey-based. We propose an alternative: measure press freedom from the diversity of attention an outlet pays, computed automatically from large-scale news data. The result tracks the official PFI while updating in near-real time.

News & journalism Bias & fairness

Convergence of Media Attention Across 129 Countries

Jisun An, Hassan Aldarbesti, Haewoon Kwak

Proceedings of Social Informatics (SocInfo), 2017 (short) 2017 4 citations

Are news outlets across countries converging on the same stories or going their separate ways? Using daily data on the 100 most over-reported topics in each country over 7 months, we find two patterns coexisting: globalization (shared attention across regions) and domestication (locally distinct coverage), shaped by geography, history, time, and topic.

User behavior & engagement

Multiplex Media Attention and Disregard Network among 129 Countries

Haewoon Kwak, Jisun An

Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2017 2017 3 citations

Which countries pay attention to each other in their news coverage, and which deliberately ignore? We build a multiplex Media Attention and Disregard Network across 129 countries over 212 days. The network is skewed, hierarchical, and asymmetric, but suggests a 'global village' overall, with persistent attention blocks around the Middle East/North Africa and Russia's neighbors.

Networks & influence

High correlation of Middle East respiratory syndrome spread with Google search and Twitter trends in Korea

Soo-Yong Shin, Dong-Woo Seo, Jisun An, Haewoon Kwak, Sung-Han Kim, Jin Gwack, Min-Woo Jo

Scientific Reports 6, Article number 32920 (2016) 2016 154 citations

Can web searches and social-media chatter predict an emerging-disease outbreak? During the 2015 MERS outbreak in Korea, daily Google searches and tweets for four Korean keywords correlated strongly (above 0.7) with new and quarantined cases over the next three days, suggesting that simple digital signals can complement traditional epidemiological surveillance.

User behavior & engagement

#greysanatomy vs. #yankees: Demographics and Hashtag Use on Twitter.

Jisun An, Ingmar Weber (short)

Proceeding of the 10th International Conference on Web and Social Media (ICWSM), 2016 2016 79 citations

Demographics matter for online behavior, but most social-media studies skip them for lack of data. We use face-analysis software to infer gender, age, and race from profile images of 350K New York Twitter users, then track which hashtags each demographic group uses over a year. Top hashtags overlap broadly, but each group also has its own distinctive set.

User behavior & engagement

Are You Charlie or Ahmed? Cultural Pluralism in Charlie Hebdo Response on Twitter

Jisun An, Haewoon Kwak, Yelena Mejova, Sonia Alonso Saenz De Oger, Braulio Gomez Fortes

Proceeding of the 10th International Conference on Web and Social Media (ICWSM), 2016 2016 48 citations

How did the world respond to the Charlie Hebdo shootings on Twitter? We test whether cultural background alone predicts stance on free speech (Huntington's 'Clash of Civilizations') or whether local social context matters too. Both: 'civilization' explains broad patterns, but non-Arabs embedded in mixed networks used #JeSuisAhmed five times more often.

User behavior & engagement

Two Tales of the World: Comparison of Widely Used World News Datasets: GDELT and EventRegistry

Haewoon Kwak, Jisun An

Proceeding of the 10th International Conference on Web and Social Media (ICWSM), 2016 (short) 2016 39 citations

GDELT and Event Registry are the two big repositories researchers use to study global news. We compare them across scale, news sources, and news geography. They differ sharply in scale and sources but agree closely on news geography — a useful warning and reassurance for anyone choosing between them.

News & journalism

Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos Using GDELT and Deep Learning-based Vision APIs

Haewoon Kwak, Jisun An

ICWSM Workshop on NEws and publiC Opinion (NECO), 2016 2016 32 citations

What kinds of objects appear in news photos, and do they tell the same story as the article text? Analyzing over 2 million news photos from January 2016 using deep-learning vision APIs, we measure object frequency, photo sentiment, alignment with text tone, gender representation, and visual portrayal of political candidates. The first study of its scale.

News & journalism

Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games

Haewoon Kwak, Jeremy Blackburn, Seungyeop Han

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), 2015 2015 500 citations

Toxic behavior in team-based competitive games is widely felt but poorly understood empirically. Using over 10 million player reports on 1.46 million toxic players in League of Legends (with crowdsourced verdicts attached), we test hypotheses drawn from theories of toxic behavior at unprecedented scale, providing a foundation for systems to detect, prevent, and counter such acts.

Online harm

Breaking the News: First Impressions Matter on Online News

Julio Reis, Fabrıcio Benevenuto, Pedro Olmo, Raquel Prates, Haewoon Kwak, Jisun An

Proceeding of the 9th International Conference on Web and Social Media (ICWSM), 2015 2015 253 citations

News headlines are the first impression in the click economy. Analyzing 69,907 headlines from four major media corporations over 8 months in 2014, we extract sentiment features and find a strong link between a headline's sentiment polarity and both the popularity of the news and the dynamics of its comments. Sentiment shapes how news travels.

News & journalism

Whom should we sense in 'social sensing'-analyzing which users work best for social media now-casting

Jisun An, Ingmar Weber

EPJ Data Science, 4, Article number 22, 2015 2015 29 citations

Social media 'now-casting' (predicting flu activity, unemployment) usually counts every tweet equally, but social-media users don't equally represent the offline population. We show that filtering by user characteristics (profile completeness, group membership) substantially improves now-casting accuracy, even when most users are excluded the right subset can outperform the whole.

User behavior & engagement

Consumers and Suppliers: Attention asymmetries. A Case Study of Aljazeera’s News Coverage and Comments

Sofiane Abbar, Jisun An, Haewoon Kwak, Yacine Messaoui, Javier Borge-Holthoefer

Computation+Journalsim (C+J) Symposium, 2015 2015

News-geography research usually relies on what outlets publish, but what about what audiences actually read and react to? We pair 20,000 Aljazeera articles with 2M+ comments by 90,000 readers (countries inferred from IP) to contrast the producer's lens (coverage) with the consumer's lens (attention), exposing where the two diverge.

News & journalism

STFU NOOB! Predicting Crowdsourced Decisions on Toxic Behavior in Online Games

Jeremy Blackburn, Haewoon Kwak

Proceedings of the 23rd international conference on World wide web (WWW), 2014 2014 311 citations

League of Legends crowdsources toxicity judgements through the 'Tribunal,' but human review is slow and expensive. We train classifiers on 10M+ player reports covering 1.46M toxic players and the corresponding crowdsourced verdicts. The model captures strong-majority cases well and ports across regions, suggesting a practical way to reduce moderation cost.

Online harm

Didn’t You See My Message? Predicting Attentiveness to Mobile Instant Messages

Martin Pielot, Rodrigo de Oliveira, Haewoon Kwak, Nuria Oliver

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2014 2014 277 citations

Instant messaging silently signals 'last seen,' creating social pressure to reply fast. We show those signals are also weak predictors of actual attentiveness. Tracking 24 users over 2 weeks, we train a model on phone signals (notifications, screen activity, proximity, ringer mode) that predicts whether a user will view a message within minutes at 70% accuracy.

User behavior & engagement

Linguistic Analysis of Toxic Behavior in an Online Video Game

Haewoon Kwak, Jeremy Blackburn

SocInfo Workshop on Exploration on Games and Gamers (EGG), 2014 2014 167 citations

What does toxic player chat actually look like, linguistically? Using crowdsourced data on 590,000+ reported toxic players in League of Legends, we trace the language patterns that distinguish toxic players from typical ones, and find empirical support for how a player gradually slides from typical into toxic behavior. The findings enable earlier detection and warning.

Online harm

Partisan Sharing: Facebook Evidence and Societal Consequences

Jisun An, Daniele Quercia, Jon Crowcroft

Proceedings of the Second ACM Conference on Online Social Networks (COSN), 2014 2014 150 citations

Even if Facebook users see politically diverse articles, do they only share ones that match their views? We test for 'partisan sharing' across individuals, time, and topic importance, and find strong evidence it exists. A controlled experiment with 70+ BBC viewers shows partisan sharing distorts perception of reality, but is also associated with more political knowledge.

Beliefs & opinions

Searching for a Unique Style in Soccer

Laszlo Gyarmati, Haewoon Kwak, Pablo Rodriguez

KDD Workshop on Large-Scale Sports Analytics, 2014 2014 137 citations

Is there really such a thing as a unique team style in modern soccer? We introduce 'flow motifs' — statistically significant patterns in pass sequences — to compare teams' play. Most teams converge on similar structures, but FC Barcelona's tiki-taka turns out to be a genuinely distinctive, tightly constructed pattern rather than a flurry of random passes.

Networks & influence

Recommending investors for crowdfunding projects

Jisun An, Daniele Quercia, Jon Crowcroft

Proceedings of the 23rd international conference on World wide web (WWW), 2014 2014 136 citations

Most Kickstarter projects fail because creators can't reach enough investors. We propose ways to recommend Twitter accounts likely to pledge to a given project, translating hypothesis-driven analyses of pledging behavior into recommendation strategies. The best strategy predicts a list of plausible investors with 84% accuracy.

User behavior & engagement

Sharing political news: the balancing act of intimacy and socialization in selective exposure

Jisun An, Daniele Quercia, Meeyoung Cha, Krishna Gummadi, Jon Crowcroft

EPJ Data Science volume 3, Article number 12, 2014 2014 115 citations

Why do people share certain political articles and not others? We propose a Political News Sharing model integrating gratification, selective exposure, socialization, and trust/intimacy. On Twitter data, sharing is driven by what one agrees with (selective exposure) but also by source credibility and informativeness, and users sometimes share opposing articles when those match their followers' leanings.

News & journalism

A First Look at Global News Coverage of Disasters By Using the GDELT Dataset

Haewoon Kwak, Jisun An

Proceedings of Social Informatics, 2014 2014 86 citations

What determines whether the world's news media notice a disaster? Using the GDELT dataset that monitors news in over 100 languages, we fit a hierarchical regression and find population, political stability, and damage all matter as expected, but also expose strong regionalism in news geography that's only visible at truly global scale.

News & journalism

From Cells to Streets: Estimating Mobile Paths with Cellular-Side Data

Ilias Leontiadis, Antonio Lima, Haewoon Kwak, Rade Stanojevic, David Wetherall, Konstantina Papagiannaki

Proceedings of the 10th ACM International on Conference on emerging Networking Experiments and Technologies (CoNEXT), 2014 2014 75 citations

Cellular networks already log subscribers' coarse location continuously. We ask whether such sparse, irregular data is enough to reconstruct each device's actual daily path. If so, the same approach extends from smartphones to the millions of future connected 'things,' enabling continuous mobility tracking without GPS or dedicated apps.

User behavior & engagement

Understanding News Geography and Major Determinants of Global News Coverage of Disasters

Haewoon Kwak, Jisun An (extension of SocInfo'14)

Computation+Journalism (C+J) Symposium, 2014 2014 34 citations

What determines whether a disaster gets global news coverage? Using the GDELT dataset that monitors news in over 100 languages worldwide, we fit a hierarchical mixed-effect regression model and find that population, political stability, and damage all matter (as prior research suggested), but also expose strong regionalism that shows up only when the dataset is genuinely global.

News & journalism

Has Much Potential but Biased: Exploring the Scholarly Landscape in Twitter

Haewoon Kwak, Jonggun Lee (poster)

Proceedings of the 23rd International Conference on World Wide Web Companion, 2014 2014 7 citations

Can Twitter measure a paper's scientific impact? We track how 75,677 research papers from the top 100 journals get shared on Twitter (403,165 tweets by 142,743 users over 135 days). Twitter's reach for scholarly content is real, but attention is highly skewed toward a small subset of top journals, limiting its value as a fair impact signal.

Bias & fairness

Fragmented Social Media: A Look into Selective Exposure to Political News

Jisun An, Daniele Quercia, Jon Crowcroft (poster)

Proceedings of the 22nd International Conference on World Wide Web (WWW) Companion, 2013 2013 88 citations

Selective exposure has been theoretically promising but hard to test empirically. Looking at news articles shared on Facebook, we find concrete evidence that users predominantly share like-minded articles and avoid conflicting ones, with partisans especially so. Building tools to counter this would require identifying partisan users, but the subgroup is demographically indistinguishable from the average user.

News & journalism

Structures of Broken Ties: Exploring Unfollow Behavior on Twitter

Bo Xu, Yun Huang, Haewoon Kwak, Noshir S. Contractor

Proceedings of the 16th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2013 2013 75 citations

What makes a Twitter user stop following another? Using actor-oriented modeling (SIENA) on tightly-knit user groups, we find relational properties dominate: mutual following and shared followees reduce unfollow likelihood, and unfollows tend to be reciprocal (being unfollowed prompts unfollowing back). Surprisingly, homophily of interests and informativeness of interactions show no measurable effect.

Networks & influence

Why Do I Retweet It? An Information Propagation Model for Microblogs

Fabio Pezzoni, Jisun An, Andrea Passarella, Jon Crowcroft, Marco Conti

Proceedings of the 5th International Conference on Social Informatics (SocInfo), 2013 2013 51 citations

What makes a tweet get retweeted? Combining network structure with user behavior, we find that a message's visibility in a user's incoming flow strongly predicts whether they pass it on, but latent traits of the original poster also matter. We turn these findings into a propagation model whose simulated cascades match the empirical ones.

Networks & influence

Tower of Babel: A Crowdsourcing Game Building Sentiment Lexicons for Resource-scarce Languages

Yoonsung Hong, Haewoon Kwak, Youngmin Baek, Sue Moon

WWW Workshop on Multidisciplinary Approaches to Big Social Data Analysis, 2013 2013 28 citations

Sentiment analysis needs language-specific lexicons, but non-English languages often lack them. Tower of Babel is a language-independent crowdsourcing game that builds sentiment lexicons through play. A 135-participant study finds the game produces accurate classifications, faster than manual annotation and rated as more enjoyable.

User behavior & engagement

Why Individuals Seek Diverse Opinions (or Why They Don't)

Jisun An, Daniele Quercia, Jon Crowcroft

Proceedings of the 5th Annual ACM Web Science Conference (WebSci), 2013 2013 21 citations

Tools that expose people to diverse opinions are pitched as a fact-checking aid, but does anyone want them? Through surveys of 60 people and in-depth interviews of 10 in the UK and South Korea, we find people don't equally value opinion diversity, and even the same person benefits only sometimes, depending on prior convictions, emotional state, and social context.

Beliefs & opinions

Traditional Media Seen from Social Media

Jisun An, Daniele Quercia, Meeyoung Cha, Krishna Gummadi, Jon Crowcroft

Proceedings of the 5th Annual ACM Web Science Conference (WebSci), 2013 2013 7 citations

Twitter exposes who follows which news outlet and how outlets interact through their audiences. Building a map of 77 media outlets from Twitter subscription overlap, we can answer questions previously locked in media databases: how much do NYT and WSJ readers overlap? Are outlets competing or complementary? Do friends subscribe to similar outlets?

News & journalism

Visualizing Media Bias through Twitter

Jisun An, Meeyoung Cha, Krishna Gummadi, Jon Crowcroft, Daniele Quercia

ICWSM Workshop on the Potential of Social Media Tools and Data for Journalists, 2012 2012 111 citations

Tracking political bias in news outlets matters for media literacy. We map media sources onto a left-right political spectrum using Twitter co-subscription (who follows which outlet). Analyzing 7M follow links shows the political split naturally emerges from direct subscription data, and we demonstrate a real-time application visualizing the ideological landscape.

Bias & fairness News & journalism

More of a Receiver than a Giver: Why Do People Unfollow in Twitter?

Haewoon Kwak, Sue Moon, Wonjae Lee (4 page poster)

Proceedings of the 6th International AAAI Conference on Weblogs and Social Media (ICWSM), 2012 2012 63 citations

Why do people unfollow on Twitter? We fit a logistic regression with structural and behavioral features (ego, dyad, and triad properties) on large-scale data, finding 5 structural and 7 actional variables that significantly predict unfollows. One striking pattern: people are less likely to unfollow accounts that have acknowledged them, even in lopsided communications.

Networks & influence

Media Landscape in Twitter: A World of New Conventions and Political Diversity

Jisun An, Meeyoung Cha, Krishna Gummadi, Jon Crowcroft

Proceedings of the 5th International AAAI Conference on Weblogs and Social Media (ICWSM), 2011. 2011 249 citations

Who you follow on Twitter shapes which media you see, but indirect exposure (friends who follow outlets, retweets from outside your circle) matters too. Using public follow data, we show that indirect exposure expands the political diversity of news a typical user encounters by 60 to 98%, a finding not visible to traditional media research.

News & journalism Bias & fairness

Fragile Online Relationship: a First Look at Unfollow Dynamics in Twitter

Haewoon Kwak, Hyunwoo Chun, Sue Moon

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2011. 2011 238 citations

How often do Twitter users actually unfollow each other? Tracking 1.2M Korean-speaking users daily for 51 days, we find unfollowing is common. Quantitative analysis identifies key factors (reciprocity, relationship duration, followees' informativeness, overlap of relationships); 22 interviews reveal users unfollow those who tweet too often, about boring topics, or about mundane life details.

Networks & influence

Consistent Community Identification in Complex Networks

Haewoon Kwak, Sue Moon, Young-Ho Eom, Yoonchan Choi, Hawoong Jeong

Journal of Korean Physical Society, Vol. 59, No. 5, November 2011. 2011 21 citations

Community-detection algorithms tend to return different communities each time you change the input node ordering. We propose two metrics (pairwise membership probability, consistency) and use the first as link weights to drive a procedure that converges to consistent communities within six cycles or fewer, without sacrificing modularity.

Networks & influence

What is Twitter, a Social Network or a News Media?

Haewoon Kwak, Changhyun Lee, Hosung Park, Sue Moon

Proceedings of the 19th international conference on World wide web (WWW), 2010. 2010 10,650 citations

Twitter exploded from a nascent microblogging service to over 41M users in just a few years. We crawl the full Twitter graph (1.47B follow relationships, 4,262 trending topics, 106M tweets) and characterize its topology and information dynamics, asking whether Twitter behaves more like a social network or a news medium. The finding: more news than network.

News & journalism

Finding Influentials based on the Temporal Order of Information Adoption in Twitter

Changhyun Lee, Haewoon Kwak, Hosung Park, Sue Moon (poster)

Proceedings of the 19th international conference on World wide web (WWW), 2010. 2010 216 citations

Many methods identify influential Twitter users from follower or retweet counts, but ignore when information actually gets adopted. We propose a method that incorporates both link structure and the temporal order of adoption, surfacing a distinct class of influentials that previous methods miss.

Networks & influence

Understanding Topological Mesoscale Features in Community Mining

Sue Moon, Jinyoung You, Haewoon Kwak, Daniel Kim, and Hawoong Jeong (invited paper)

Proceedings of the Second International Conference on COMmunication Systems and NETworks (COMSNETS), 2010. 2010 9 citations

Community-detection algorithms often disagree on which nodes belong where. Building on an iterative-reinforcement approach across 12 real networks, we find about 40% of nodes are grouped inconsistently in large social graphs (Orkut, Cyworld). Using participation-coefficient analysis, we classify nodes into seven structural roles and compare two AS networks built from different sources.

Networks & influence

I Tube, You Tube, Everybody Tubes: Analyzing the World's Largest User Generated Content Video System

Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, Sue Moon

Proceedings of the 7th ACM SIGCOMM conference on Internet measurement (IMC), 2009 2009 2,255 citations

YouTube reshaped how people make and watch video, and traditional video-on-demand metrics don't apply. We analyze the world's largest user-generated content video system, characterizing the popularity distributions, lifetimes, and request patterns of user-uploaded videos, with implications for caching, monetization, and platform design.

User behavior & engagement

Analyzing the Video Popularity Characteristics of Large-Scale User Generated Content Systems

Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon

ACM/IEEE Transactions on Networking, Vol 17, Issue 5, 2009 2009 673 citations

User-generated video systems differ fundamentally from traditional video-on-demand. Analyzing YouTube and Korea's Daum Videos, we characterize UGC popularity distributions, the lifetime of a video's popularity, and the relationship between requests and content age. We also surface the impact of content aliasing and illegal uploads on ranking accuracy.

User behavior & engagement

The Wisdom of the Few: A Collaborative Filtering Approach based on Expert Opinions from the Web

Xavier Amatriaain, Neal Lathia, Josep M. Pujol, Haewoon Kwak, Nuria Oliver

Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval (SIGIR), 2009 2009 230 citations

Standard collaborative filtering suffers from data sparsity, cold start, and noise. We propose computing recommendations from a separate set of expert reviews (instead of the user-rating graph itself), weighted by similarity to each user. Validated on Netflix data and a user study where participants preferred our recommendations, the method preserves accuracy while avoiding CF's weaknesses.

Beliefs & opinions

Mining Communities in Networks: a Solution for Consistency and Its Evaluation

Haewoon Kwak, Yoonchan Choi, Young-Ho Eom, Hawoong Jeong, Sue Moon

Proceedings of the 9th ACM SIGCOMM conference on Internet measurement (IMC), 2009 2009 93 citations

Three popular community-detection algorithms (CNM, Wakita, Louvain) all return different communities every time you change input node ordering. We propose pairwise membership probability as a consistency metric, then use it as link weights in an iterative procedure that converges to consistent communities within six cycles without sacrificing modularity.

Networks & influence

Connecting Users with Similar Interests Across Multiple Web Services

Haewoon Kwak, Hwa-Yong Shin, Jong-Il Yoon, Sue Moon (poster)

Proceedings of the 3rd International AAAI Conference on Weblogs and Social Media (ICWSM), 2009 2009 11 citations

Online services group users with shared interests, but their group-recommendation signals stay siloed inside each service. We propose connecting users across multiple web services through their user-labeled tags, which carry less privacy risk and stay fresher than profile data. A vector-space model with intra-personomy normalization finds similar users effectively across six services.

Networks & influence

Comparison of Online Social Relations in Volume vs Interaction: A Case Study of Cyworld

Hyunwoo Chun, Haewoon Kwak, Young-Ho Eom, Yong-Yeol Ahn, Sue Moon, and Hawoong Jeong

Proceedings of the 8th ACM SIGCOMM conference on Internet measurement (IMC), 2008 2008 279 citations

Is a Cyworld 'friend' an active relationship or just a declaration? We compare friend-list networks against activity networks formed by actual guestbook comments. The two have similar topology, but only the activity network (with reciprocated comments as edges) behaves like a true social network: more assortative, with fewer artificially dense cores.

Networks & influence

Analysis of topological characteristics of huge online social networking services

Yong-Yeol Ahn, Seungyeop Han, Haewoon Kwak, Sue Moon, Hawoong Jeong

Proceedings of the 16th international conference on World Wide Web (WWW), 2007 2007 1,422 citations

Do online relationships behave like real-life ones? We compare three online social networks of 10M+ users each (Cyworld, MySpace, orkut), analyzing degree distribution, clustering, and degree correlation. Cyworld's distribution shifts toward multi-scaling over time; the three networks differ in scaling exponents, and certain online behaviors diverge from real-life social patterns.

Networks & influence

Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions

Fan Huang, Haewoon Kwak, Jisun An

ACL Findings

Are LLMs persuaded out of their beliefs? Using the Source–Message–Channel–Receiver (SMCR) communication framework across six mainstream LLMs and three domains (facts, medical QA, social bias), we measure how stable each model's stated beliefs are under persuasive pressure. The smallest model flips on 82.5% of attempts at the first turn. Counterintuitively, asking models to verbalize confidence makes them more vulnerable.

LLMs & responsible AI