Hanjia Lyu

I am a fourth-year Ph.D. student of the Computer Science Department at the University of Rochester (UR), where I am advised by Prof. Jiebo Luo. Previously, I completed my master’s in Data Science at UR and bachelor’s at Fudan University. I am interested in computational behavioral science. My research goal is to develop and apply innovative machine learning strategies to understand and predict human behavior, ultimately addressing real-world challenges for social good.

Email / CV / Google Scholar / Github / Twitter / LinkedIn

Quick Links

What's New Research Awards Teaching Service Internship Publications Other Fun Stuff

What's New

[10/15/2024] I will serve as an Area Chair for NAACL 2025.

[08/29/2024] I was awarded the 2024 Google PhD Fellowship. 🎉🎉

[07/30/2024] Our new study analyzing Reddit comments post-ChatGPT launch explores the tech community's and general public's perceptions of AI.

[06/15/2024] I will serve as an Area Chair for EMNLP 2024.

[05/15/2024] Our study on the factors contributing to crowdfunding success has been featured in The Wall Street Journal.

[11/13/2023] 🔥 Check out our latest exploration on GPT-4V(ision) for social multimedia analysis tasks including sentiment analysis, hate speech detection, fake news identification, demographic inference, and ideology detection!

[07/13/2023] In our new studies (ICWSM 2024, ICWSMW 2023), we uncover media bias in 1.8 million headlines across the political spectrum. Our findings suggest that media bias is growing. The work is also covered by StudyFinds, and shared by ScienceBlog, RealClearScience, Nation, Mirage.News, Phys.org, and the American Association for the Advancement of Science (AAAS) via EurekAlert!

[03/21/2023] Our survey entitled Human Behavior in the Time of COVID-19: Learning from Big Data, is accepted for publication in the Horizons in Big Data 2022 article collection of Frontiers in Big Data.

[03/09/2023] I was awarded the 2023 Arts, Sciences and Engineering Donald M. and Janet C. Barnard Fellowship. 🎉🎉

[03/25/2022] Our study on the scale and scope of the influence of misinformation and fact-based news about COVID-19 vaccines on social media platforms on the vaccine uptake is covered by multiple news outlets including Medical Economics, News Medical, News Wise, UK Today News, and Medical Xpress.

[05/19/2021] In a new study, we dissect the public responses to the #StopAsianHate movement using large scale Twitter data and provide findings that can help design better ways of reducing tension and misunderstandings between ethnic groups. The work is also covered by Rochester Beacon and Futurity.

[08/12/2020] We won the first place of the University of Rochester Biomedical Data Science Hackathon Summer 2020.

[07/23/2020] Tech Xplore reports a series of our recent studies that show Twitter mirrors our attitudes and feelings about COVID-19. These findings are shared in a lightning presentation for chairs of computer science PhD-granting departments and heads of major industrial research labs at the Computing Research Association (CRA) Virtual Conference.

[05/30/2020] Our work on analyzing the use of controversial terms of COVID-19 on Twitter is highlighted by IEEE Spectrum, the flagship magazine and website of the IEEE. Incidentally, this story shares the headlines with the story about the successful launch of the SpaceX Dragon 2 spacecraft. Cool! This work is also covered by Tech Xplore.

[03/26/2020] Our work on analyzing the performance of the crowdfunding campaigns on GoFundMe over a wide variety of funding categories is covered by The New York Times.

Research

Representative works are highlighted.

* indicates co-first-authorship.

2024
Click for zooming up.	Semantics Preserving Emoji Recommendation with Large Language Models Zhongyi Qiu, Kangyi Qiu, Hanjia Lyu, Wei Xiong, Jiebo Luo IEEE Big Data Conference, 2024 [PDF] We propose a new semantics preserving evaluation framework for emoji recommendation, which measures a model's ability to recommend emojis that maintain the semantic consistency with the user's text.
Click for zooming up.	In the Eyes of the Bystander: Are the Stances on Different Conflicts Correlated? Yiyao Tao, Hengyu Zhang, Babli Dey, Selenge Tulga, Hanjia Lyu, Jiebo Luo IEEE Big Data Conference, 2024 [PDF] This study investigates how public opinions toward one conflict might influence or relate to another.
Click for zooming up.	CRTRE: Causal Rule Generation with Target Trial Emulation Framework Junda Wang, Weijian Li, Han Wang, Hanjia Lyu, Caroline Thirukumaran, Addisu Mesfin, Hong Yu, Jiebo Luo IEEE Big Data Conference, 2024 [PDF] We introduce a causal rule generation with target trial emulation framework (CRTRE), which applies randomize trial design principles to estimate the causal effect of association rules.
Click for zooming up.	ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents Xinnong Zhang, Jiayu Lin, Libo Sun, Weihong Qi, Yihang Yang, Yue Chen, Hanjia Lyu, Xinyi Mou, Siming Chen, Jiebo Luo, Xuanjing Huang, Shiping Tang, Zhongyu Wei arXiv*, 2024 [PDF] We introduce ElectionSim, an innovative election simulation framework based on large language models, designed to support accurate voter simulations and customized distributions, together with an interactive platform to dialogue with simulated voters.
Click for zooming up.	Personalized Multimodal Large Language Models: A Survey Junda Wu, Hanjia Lyu, Yu Xia, Zhehao Zhang, Joe Barrow, Ishita Kumar, Mehrnoosh Mirtaheri, Hongjie Chen, Ryan Rossi, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Namyong Park, Sungchul Kim, Huanrui Yang, Subrata Mitra, Zhengmian Hu, Nedim Lipka, Jiebo Luo, Julian McAuley arXiv, 2024 [PDF] This paper presents a comprehensive survey on personalized multimodal large language models, focusing on their architecture, training methods, and applications.
Click for zooming up.	X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation Hanjia Lyu, Ryan Rossi, Xiang Chen, Md Mehrab Tanjim, Stefano Petrangeli, Somdeb Sarkhel, Jiebo Luo arXiv , 2024 [PDF] This study introduces a novel framework which prompts LMMs to explicitly identify and reconcile supportive and conflicting information between text and images to generate more comprehensive and contextually richer item representations.
Click for zooming up.	Towards Advancing Text-Based User and Item Representation in Personalized Recommendation Hanjia Lyu CIKM , 2024 (Doctoral Symposium) [PDF] This study introduces innovative approaches using Large Language Models (LLMs) to generate detailed textual descriptions that enhance both user and item representations.
Click for zooming up.	Representation Bias in Political Sample Simulations with Large Language Models Weihong Qi, Hanjia Lyu, Jiebo Luo arXiv, 2024 [PDF] Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao Dataset, and China Family Panel Studies to simulate voting behaviors and public opinions.
Click for zooming up.	Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese Hanjia Lyu, Jiebo Luo, Jian Kang, Allison Koenecke arXiv, 2024 [PDF] To measure LLM performance disparities in Simplified versus Traditional Chinese, we introduce a comprehensive benchmark dataset, SC-TC-Bench, including prompts and answers across three real-world scenarios: regional term recognition, name-based qualification perception, and location entity selection.
Click for zooming up.	INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance Chenwei Lin, Hanjia Lyu, Xian Xu, Jiebo Luo arXiv, 2024 [PDF] We propose INS-MMBench, the first comprehensive LVLMs benchmark tailored for the insurance domain. INS-MMBench comprises a total of 2.2K thoroughly designed multiple-choice questions, covering 12 meta-tasks and 22 fundamental tasks.
Click for zooming up.	Harnessing GPT-4V(ision) for Insurance: A Preliminary Exploration Chenwei Lin, Hanjia Lyu, Jiebo Luo, Xian Xu arXiv, 2024 [PDF] We explore GPT-4V's capabilities in the insurance domain. We categorize multimodal tasks by focusing primarily on visual aspects based on types of insurance (e.g., auto, household/commercial property, health, and agricultural insurance) and insurance stages (e.g., risk assessment, risk monitoring, and claims processing).
Click for zooming up.	Excitements and Concerns in the Post-ChatGPT Era: Deciphering Public Perception of AI through Social Media Analysis Weihong Qi, Jinsheng Pan, Hanjia Lyu, Jiebo Luo Telematics and Informatics, 2024 [PDF] [Poster] [388 Subreddits] In this study, we investigate how mass social media users perceive the recent rise of AI frameworks such as ChatGPT.
Click for zooming up.	A Benchmark and Chain-of-Thought Prompting Strategy for Large Multimodal Models with Multiple Image Inputs Daoan Zhang, Junming Yang, Hanjia Lyu, Zijian Jin, Yuan Yao, Mingkai Chen, Jiebo Luo ICPR*, 2024 [PDF] We develop a Contrastive Chain-of-Thought prompting approach based on multi-input multimodal models. This method requires LMMs to compare the similarities and differences among multiple image inputs.
Click for zooming up.	Learning to Evaluate the Artness of AI-generated Images Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo TMM, 2024 [PDF] This paper presents ArtScore, a metric designed to evaluate the degree to which an image resembles authentic artworks by artists (or conversely photographs).
Click for zooming up.	Holistic Visual-Textual Sentiment Analysis with Prior Models Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo MIPR, 2024 [PDF] We propose a new method that improves visual-textual sentiment analysis by introducing powerful expert visual features.
Click for zooming up.	SoMeLVLM: A Large Vision Language Model for Social Media Processing Xinnong Zhang, Haoyu Kuang, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, Zhongyu Wei ACL, 2024 [PDF] [Project Page] [Code] We propose SoMeLVLM, a large vision language model tailored for social media processing via extensive and comprehensive supervised fine-tuning.
Click for zooming up.	Creating An Authoring Tool for K-12 Teachers to Design ML-supported Scientific Inquiry Learning Xiaofei Zhou, Jingwan Tang, Hanjia Lyu, Xinyi Liu, Zhenhao Zhang, Lichen Qin, Fiona Au, Advait Sarkar, Zhen Bai CHI (Late Breaking Work), 2024 [PDF] We design a novel web-based tool, ML4SI, for K-12 teachers to create ML-supported SI learning activities for K-12 STEM education.
Click for zooming up.	Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection Jinfa Huang, Jinsheng Pan, Zhongwei Wan, Hanjia Lyu, Jiebo Luo arXiv, 2024 [PDF] We propose the Chain-of-Evolution Prompting for boosting large multimodal models for hateful meme detection.
Click for zooming up.	Mixture of Weak & Strong Experts on Graphs Hanqing Zeng, Hanjia Lyu*, Diyi Hu, Yinglong Xia, Jiebo Luo ICLR*, 2024 [PDF] [Poster] [Code] We propose Mixture of Weak & Strong Experts on graphs, where the weak expert is a light-weight Multi-layer Perceptron (MLP), and the strong expert is an off-the-shelf Graph Neural Network (GNN).
Click for zooming up.	LLM-Rec: Personalized Recommendation via Prompting Large Language Models Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, Jiebo Luo NAACL, 2024 [PDF] [Poster] We investigate various prompting strategies for enhancing personalized content recommendation performance with Large Language Models (LLMs) through input augmentation.
Click for zooming up.	Retrieval Augmentation via User Interest Clustering Hanjia Lyu, Hanqing Zeng, Yinglong Xia, Ren Chen, Jiebo Luo arXiv, 2024 [PDF] We propose a novel approach that efficiently constructs user interest and facilitates low-cost inference by clustering engagement graphs and incorporating user-interest attention.
Click for zooming up.	Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, Zhongyu Wei WWW, 2024 [PDF] We present a Political Experts through Knowledge Graph Integration (PEG) framework.
Click for zooming up.	Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication Hanjia Lyu, Weihong Qi, Zhongyu Wei, Jiebo Luo ICWSM, 2024 [PDF] [Poster] We examine the behavior of GPT-4V in replicating human-like use of emojis.
Click for zooming up.	Computational Assessment of Hyperpartisanship in News Titles Hanjia Lyu, Jinsheng Pan, Zichen Wang, Jiebo Luo ICWSM*, 2024 [PDF] [Poster] [Data] We conduct a computational analysis to quantify the extent and dynamics of partisanship in news titles. While some aspects are as expected, our study reveals new or nuanced differences between the three media groups.
2023
Click for zooming up.	GPT-4V(ision) as A Social Media Analysis Engine Hanjia Lyu, Jinfa Huang, Daoan Zhang, Yongsheng Yu, Xinyi Mou, Jinsheng Pan, Zhengyuan Yang, Zhongyu Wei, Jiebo Luo arXiv*, 2023 [PDF] We explore GPT-4V(ision)'s capabilities for social multimedia analysis including sentiment analysis, hate speech detection, fake news identification, demographic inference, and political ideology detection.
Click for zooming up.	Understanding Divergent Framing of the Supreme Court Controversies: Social Media vs. News Outlets Jinsheng Pan, Zichen Wang, Weihong Qi, Hanjia Lyu, Jiebo Luo IEEE Big Data Conference, 2023 [PDF] We conduct a comprehensive investigation, focusing on the nuanced distinctions in the framing of social media and traditional media outlets concerning a series of American Supreme Court rulings on affirmative action, student loans, and abortion rights.
Click for zooming up.	A Fine-Grained Analysis of Public Opinion toward Chinese Technology Companies on Reddit Enting Zhou, Yurong Liu, Hanjia Lyu, Jiebo Luo IEEE Big Data Conference, 2023 [PDF] This study aims to fill in the gap of understanding the public opinion toward Chinese technology companies using Reddit data.
Click for zooming up.	Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking Mohammad Mahdi Kamani, Yuhang Yao, Hanjia Lyu, Zhongwei Cheng, Lin Chen, Liangju Li, Carlee Joe-Wong, Jiebo Luo NeurIPS, 2023 [PDF] We present Wyze Rule Dataset, a large-scale dataset designed specifically for smart home rule recommendation research.
Click for zooming up.	Dismantling Hate: Understanding Hate Speech Trends Against NBA Athletes Edinam Kofi Klutse, Samuel Nuamah-Amoabeng, Hanjia Lyu, Jiebo Luo SBP-BRiMS, 2023 [PDF] A deep learning classification model is implemented, effectively identifying tweets that genuinely exhibit hate against NBA players.
Click for zooming up.	Bias or Diversity? Unraveling Fine-Grained Thematic Discrepancy in U.S. News Headlines Jinsheng Pan, Weihong Qi, Zichen Wang, Hanjia Lyu, Jiebo Luo MEDIATE, 2023 [PDF] In this study, we collect a large dataset of 1.8 million news headlines from major U.S. media outlets spanning from 2014 to 2022 to thoroughly track and dissect the fine-grained thematic discrepancy in U.S. news media.
Click for zooming up.	Predicting Adverse Neonatal Outcomes for Preterm Neonates with Multi-Task Learning Jingyang Lin, Junyu Chen, Hanjia Lyu, Igor Khodak, Divya Chhabra, Colby L Day Richardson, Irina Prelipcean, Andrew M Dylag, Jiebo Luo ICDH, 2023 [PDF] We propose an MTL (multi-task learning) framework to jointly predict multiple adverse neonatal outcomes.
Click for zooming up.	Applying Machine Learning to Predict Esophageal Cancer Recurrence After Esophagectomy Hanjia Lyu, Kevin C. Kapcio, Kyle C. Purrman, Christian G. Peyre, Carolyn E. Jones, Michal J. Lada, Jiebo Luo ICDH, 2023 [PDF] We conducted a retrospective study of 260 consecutive patients who underwent esophagectomy for esophageal cancer from 2009 through 2018.
Click for zooming up.	Human Behavior in the Time of COVID-19: Learning from Big Data Hanjia Lyu, Arsal Imtiaz, Yufei Zhao, Jiebo Luo Frontiers in Big Data, 2023 [PDF] In this study, we present an overview of the existing studies on using big data techniques to study human behavior in the time of the COVID-19 pandemic.
Click for zooming up.	ML-SD Modeling: How Machine Learning Can Support Scientific Discovery Learning for K-12 STEM Education Xiaofei Zhou, Hanjia Lyu, Jiebo Luo, Zhen Bai AI4Edu, 2023 [PDF] [Video@YouTube] [Video@Bilibili] This work proposes research ideas and initial modeling results on the connection between Machine Learning components and young learners' scientific behaviors.
2022
Click for zooming up.	Doctors vs. Nurses: Understanding the Great Divide in Vaccine Hesitancy among Healthcare Workers Sajid Hussain Rafi Ahamed, Shahid Shakil, Hanjia Lyu, Xinping Zhang, Jiebo Luo IEEE Big Data Conference, 2022 [PDF] Reportedly, a considerably higher proportion of vaccine hesitancy is observed among nurses, compared to doctors. We intend to verify and study this phenomenon at a much larger scale and in fine grain using social media data.
Click for zooming up.	American Twitter Users Revealed Social Determinants-related Oral Health Disparities amid the COVID-19 Pandemic Yangxin Fan, Hanjia Lyu, Jin Xiao, Jiebo Luo Quintessence International, 2022 [PDF] By conducting logistic regression, we find that discussions about oral health vary across user characteristics. More importantly, we find social disparities in oral health during the pandemic.
Click for zooming up.	Understanding Political Polarization via Jointly Modeling Users, Connections and Multimodal Contents on Heterogeneous Graphs Hanjia Lyu, Jiebo Luo ACM MM, 2022 [PDF] [Video] [Poster] We adopt a heterogeneous graph neural network to jointly model user characteristics, multimodal post contents as well as user-item relations in a bipartite graph to learn a comprehensive and effective user embedding without requiring ideology labels.
Click for zooming up.	Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination Hanjia Lyu, Junda Wang, Wei Wu, Viet Duong, Xiyang Zhang, Timothy D. Dye, Jiebo Luo Intelligent Medicine, 2022 [PDF] We adopt a human-guided machine learning framework using more than six million tweets from almost two million unique Twitter users to capture public opinions on the vaccines for SARS-CoV-2.
Click for zooming up.	Taking sides: Public Opinion over the Israel-Palestine Conflict in 2021 Arsal Imtiaz, Danish Khan, Hanjia Lyu, Jiebo Luo SocialSens, 2022 [PDF] We devise an observational study to understand the friendliness of countries, agglomerated by the sentiments of tweets.
Click for zooming up.	Look behind the Censorship: Reposting-User Characterization and Muted-Topic Restoration Yichi Qian, Qiyi Shan, Hanjia Lyu, Jiebo Luo SocialSens, 2022 [PDF] We 1) create a web-scraping pipeline and collect a large dataset solely focusing on the reposts from Weibo; 2) discover the characteristics of users whose reposts contain censored information, in terms of gender, device, and account type; and 3) conduct a thematic analysis by extracting and analyzing topic information.
Click for zooming up.	Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis Wei Zhu, Zihe Zheng, Haitian Zheng, Hanjia Lyu, Jiebo Luo ICPR, 2022 [PDF] Our method relies on an external memory to aggregate and filter noisy labels during training and thus can prevent the model from overfitting the noisy cases.
Click for zooming up.	Challenges and Design Opportunities in Data Analysis for ML-Empowered Scientific Inquiry-Insights from a Teacher Professional Development Study Xiaofei Zhou, Jingwan Tang, Beilei Guo, Hanjia Lyu, Zhen Bai ICLS, 2022 [PDF] We investigated the pattern recognition and interpretation behaviors of 18 K-12 STEM teachers when engaged in ML-empowered scientific inquiry during a professional development workshop.
Click for zooming up.	Misinformation versus Facts: Understanding the Influence of News Regarding COVID-19 Vaccines on Vaccine Uptake Hanjia Lyu, Zihe Zheng, Jiebo Luo Health Data Science, 2022 [PDF] We conducted the Fama-MacBeth regression with the Newey-West adjustment to understand the influence of both misinformation and fact-based news on Twitter on the COVID-19 vaccine uptake.
2021
Click for zooming up.	Understanding Public Opinion Toward the #StopAsianHate Movement and the Relation With Racially Motivated Hate Crimes in the US Hanjia Lyu, Yangxin Fan, Ziyu Xiong, Mayya Komisarchik, Jiebo Luo IEEE TCSS, 2021 [PDF] We conduct a social media study of public opinion on the #StopAsianHate and #StopAAPIHate movement based on 46,058 Twitter users across 30 states in the United States ranging from March 18 to April 11, 2021.
Click for zooming up.	From Static to Dynamic Prediction: Wildfire Risk Assessment Based on Multiple Environmental Factors Tanqiu Jiang, Sidhant K. Bendre, Hanjia Lyu, Jiebo Luo IEEE Big Data Conference, 2021 [PDF] [Video] We propose static and dynamic prediction models to analyze and assess the areas with high wildfire risks in California.
Click for zooming up.	Understanding the Hoarding Behaviors during the COVID-19 Pandemic using Large Scale Social Media Data Xupin Zhang, Hanjia Lyu, Jiebo Luo IEEE Big Data Conference, 2021 [PDF] To investigate the hoarding behaviors in response to the pandemic, we propose a novel computational framework using large scale social media data.
Click for zooming up.	Social Media Opinions on Working From Home in the United States During the COVID-19 Pandemic: Observational Study Ziyu Xiong, Pin Li, Hanjia Lyu, Jiebo Luo JMIR: Medical Informatics, 2021 [PDF] We conducted a large-scale social media study using Twitter data to portray different groups of individuals who have positive or negative opinions on WFH.
Click for zooming up.	Characterizing Discourse about COVID-19 Vaccines: A Reddit Version of the Pandemic Story Wei Wu, Hanjia Lyu, Jiebo Luo Health Data Science, 2021 [PDF] This study aims to offer a clear understanding about different population groups’ underlying concerns when they talk about COVID-19 vaccines, particular those active on Reddit.
Click for zooming up.	What Contributes to a Crowdfunding Campaign’s Success? Evidence and Analyses from GoFundMe Data Xupin Zhang, Hanjia Lyu, Jiebo Luo IEEE JSC, 2021 [PDF] We focus on the performance of the crowdfunding campaigns on GoFundMe over a wide variety of funding categories.
Click for zooming up.	The Influence of COVID-19 on people’s Well-Being: Big Data Methods for Capturing Working Adults’ Well-being and Protective Factors Nationwide Xiyang Zhang, Yu Wang, Hanjia Lyu, Yipeng Zhang, Yubao Liu, Jiebo Luo Frontiers in Psychology, 2021 [PDF] We found that pandemic severity influenced working adults’ negative affect rather than positive affect.
Click for zooming up.	Fine-Grained Analysis of the Use of Neutral and Controversial Terms for COVID-19 on Social Media Long Chen, Hanjia Lyu, Tongyu Yang, Yu Wang, Jiebo Luo SBP-BRiMS, 2021 [PDF] To model the substantive difference of tweets with controversial terms and those with non-controversial terms with regard to COVID-19, we apply topic modeling and LIWC-based sentiment analysis.
Click for zooming up.	How Political is the Spread of COVID-19 in the United States? An Analysis using Transportation and Weather Data Karan Vombatkere, Hanjia Lyu, Jiebo Luo SBP-BRiMS, 2021 [PDF] We investigate the difference in the spread of COVID-19 between the states won by Donald Trump (Red) and the states won by Hillary Clinton (Blue) in the 2016 presidential election.
Click for zooming up.	Monitoring Depression Trend on Twitter during the COVID-19 Pandemic: Observational Study Yipeng Zhang, Hanjia Lyu, Yubao Liu, Xiyang Zhang, Yu Wang, Jiebo Luo JMIR: Infodemiology, 2021 [PDF] We create a fusion classifier that combines deep learning model scores with psychological text features and users’ demographic information and investigate these features’ relations to depression signals in the context of COVID-19.
2020
Click for zooming up.	InsurTech development: Evidence from Chinese media reports Siqing Cao, Hanjia Lyu, Xian Xu Technological Forecasting and Social Change, 2020 [PDF] This paper uses text mining technology and Python to analyze the word frequency and term frequency-inverse document frequency (TFIDF) of 25,662 InsurTech-related news reports from 2015 to 2019 in China.
Click for zooming up.	Sense and Sensibility: Characterizing Social Media Users Regarding the Use of Controversial Terms for COVID-19 Hanjia Lyu, Long Chen, Yu Wang, Jiebo Luo IEEE TBD, 2020 [PDF] We characterize the Twitter users who use controversial terms and those who use non-controversial terms for COVID-19.

Awards

Google PhD Fellowship, 2024
Graduate Student Supplemental Professional Development Funding Award, 2024
NSF Student Travel Award, CIKM 2024
Student Travel Award, MIPR 2024
Student Travel Award, ICWSM 2024
Student Travel Award, ICLR 2024
Donald M. and Janet C. Barnard Fellowship, 2023
Snap Research Fellowship Semi-Finalist, 2022
Graduate Student Scholarship, SBP-BRiMS 2021
Winner of the University of Rochester Biomedical Data Science Hackathon Summer 2020

Teaching

Guest Lecturer

University of Rochester, Nov. 2024
- CSC 511 Large Language Models: Adapting Large Language Models for Domain-Specific Applications
Fudan University, Nov. 2023
- MI620017 Insurance Study: Generative Artificial Intelligence in Finance and Insurance
University of Rochester, Sep. 2023
- CSC 440 Data Mining: Towards Data Intelligence: Unveiling Insights through Data Mining

Teaching Assitant

University of Rochester
- CSC 480 Computer Models & Limitations/CSC 486 Computational Complexity - Spring 2023
- CSC 440 Data Mining - Fall 2022
- CSC 446 Machine Learning - Spring 2022
- CSC 440 Data Mining - Fall 2021
- CSC 240/440 Data Mining - Fall 2020

Service

Organizing Committee

1st Multimodal Learning for Social Good (MML4SG) workshop at ICME 2024
1st Visual-Language Alignment in Text-Guided Multi-Modal Generation (TMMG) workshop at ICME 2024

Area Chair

EMNLP (2024)
NAACL (2025)
ACL Rolling Review (since June 2024)

Editor

Frontiers in Public Health

Program Committee/Reviewer

AAAI, ACL, ACML, ACMMM, AISTATS, ASONAM, BigData, CIKM, EMNLP, ICDM, ICLR, ICME, ICPR, ICWSM, KDD, NeurIPS, SDM, WSDM, WWW
TKDE, TMM, TPAMI

Internship

	Meta AI, Menlo Park, CA Research Scientist Intern (Part-Time) Oct 2024 - Jan 2025. Advisors: Yinglong Xia.
	Meta AI, Menlo Park, CA Research Scientist Intern (Full-Time) June - Oct 2024. Advisors: Yinglong Xia.
	Meta AI, Menlo Park, CA Research Scientist Intern (Full-Time) May - Aug 2023. Advisors: Hanqing Zeng, Yinglong Xia. Projects: Mixture of Weak & Strong Experts on Graphs (ICLR 2024) LLM-Rec: Personalized Recommendation via Prompting Large Language Models (NAACL 2024)
	Meta AI, Menlo Park, CA Research Scientist Intern (Full-Time) May - Sep 2022. Advisors: Fiona Tang, Ren Chen, Yinglong Xia. Project: Recommender systems.
	Wyze Labs, Kirkland, WA Research Scientist Intern (Part-Time) Jan - May 2022. Advisors: Zhongwei Cheng, Mohammad Mahdi Kamani, Lin Chen. Project: Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking (NeurIPS 2023)

Publications

* indicates co-first-authorship.

2024

Zhongyi Qiu, Kangyi Qiu, Hanjia Lyu, Wei Xiong, and Jiebo Luo, “Semantics Preserving Emoji Recommendation with Large Language Models,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Washington DC, USA, December 2024.
Yiyao Tao*, Hengyu Zhang*, Babli Dey, Selenge Tulga, Hanjia Lyu, and Jiebo Luo, “In the Eyes of the Bystander: Are the Stances on Different Conflicts Correlated?” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Washington DC, USA, December 2024.
Junda Wang, Weijian Li, Han Wang, Hanjia Lyu, Caroline Thirukumaran, Addisu Mesfin, Hong Yu, and Jiebo Luo, “CRTRE: Causal Rule Generation with Target Trial Emulation Framework,” IEEE Big Data Conference, Washington DC, USA, December 2024.
Hanjia Lyu, “Towards Advancing Text-Based User and Item Representation in Personalized Recommendation,” 33rd ACM International Conference on Information and Knowledge Management (CIKM), Boise, Idaho, October 2024.
Daoan Zhang*, Junming Yang*, Hanjia Lyu*, Zijian Jin, Yuan Yao, Mingkai Chen, and Jiebo Luo, “A Benchmark and Chain-of-Thought Prompting Strategy for Large Multimodal Models with Multiple Image Inputs,” International Conference on Pattern Recognition (ICPR), Kolkata, India, December 2024.
Weihong Qi, Jinsheng Pan, Hanjia Lyu, and Jiebo Luo, “Excitements and Concerns in the Post-ChatGPT Era: Deciphering Public Perception of AI through Social Media Analysis,” Telematics and Informatics, 2024.
Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, and Jiebo Luo, “Learning to Evaluate the Artness of AI-generated Images,” IEEE Transactions on Multimedia (TMM), 2024.
Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, and Jiebo Luo, “Holistic Visual-Textual Sentiment Analysis with Prior Models,” IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, USA, August 2024.
Xinnong Zhang*, Haoyu Kuang*, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, and Zhongyu Wei, “SoMeLVLM: A Large Vision Language Model for Social Media Processing,” Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
Hanjia Lyu*, Weihong Qi*, Zhongyu Wei, and Jiebo Luo, “Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication,” AAAI International Conference on Web and Social Media (ICWSM), Buffalo, NY, June 2024.
Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, and Jiebo Luo, “LLM-Rec: Personalized Recommendation via Prompting Large Language Models,” Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, USA, June 2024.
Xiaofei Zhou, Jingwan Tang, Hanjia Lyu, Xinyi Liu, Zhenhao Zhang, Lichen Qin, Fiona Au, Advait Sarkar, and Zhen Bai, “Creating An Authoring Tool for K-12 Teachers to Design ML-supported Scientific Inquiry Learning,” Late Breaking Papers: ACM Conference on Human Factors in Computing Systems (CHI), Honolulu, HI, USA, May 2024.
Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, and Zhongyu Wei, “Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs,” The ACM Web Conference (WWW), Singapore, May 2024.
Hanqing Zeng*, Hanjia Lyu*, Diyi Hu, Yinglong Xia, and Jiebo Luo, “Mixture of Weak & Strong Experts on Graphs,” Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
Hanjia Lyu*, Jinsheng Pan*, Zichen Wang*, and Jiebo Luo, “Computational Assessment of Hyperpartisanship in News Titles,” AAAI International Conference on Web and Social Media (ICWSM), Buffalo, NY, June 2024.

2023

Jinsheng Pan, Zichen Wang, Weihong Qi, Hanjia Lyu, and Jiebo Luo, “Understanding Divergent Framing of the Supreme Court Controversies: Social Media vs. News Outlets,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Sorrento, Italy, December 2023.
Enting Zhou*, Yurong Liu*, Hanjia Lyu, and Jiebo Luo, “A Fine-Grained Analysis of Public Opinion toward Chinese Technology Companies on Reddit,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Sorrento, Italy, December 2023.
Mohammad Mahdi Kamani, Yuhang Yao, Hanjia Lyu, Zhongwei Cheng, Lin Chen, Liangju Li, Carlee Joe-Wong, and Jiebo Luo, “Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking,” Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, December 2023.
Junyu Chen, Jie An, Hanjia Lyu, and Jiebo Luo, “How Art-like are AI-generated Images? An Exploratory Study,” International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice (McGE), ACM Multimedia Conference (ACM MM), Ottawa, Ontario, Canada, October 2023.
Edinam Kofi Klutse, Samuel Nuamah-Amoabeng, Hanjia Lyu, and Jiebo Luo, “Dismantling Hate: Understanding Hate Speech Trends Against NBA Athletes,” International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Pittsburgh, PA, September 2023.
Jingyang Lin, Junyu Chen, Hanjia Lyu, Igor Khodak, Divya Chhabra, Colby L Day Richardson, Irina Prelipcean, Andrew M Dylag, and Jiebo Luo, “Predicting Adverse Neonatal Outcomes for Preterm Neonates with Multi-Task Learning,” IEEE International Conference on Digital Health (ICDH), Chicago, IL, July 2023.
Hanjia Lyu*, Kevin C. Kapcio*, Kyle C. Purrman, Christian G. Peyre, Carolyn E. Jones, Michal J. Lada, and Jiebo Luo “Applying Machine Learning to Predict Esophageal Cancer Recurrence After Esophagectomy,” IEEE International Conference on Digital Health (ICDH), Chicago, IL, July 2023.
Jinsheng Pan*, Weihong Qi*, Zichen Wang, Hanjia Lyu, and Jiebo Luo, “Bias or Diversity? Unraveling Fine-Grained Thematic Discrepancy in U.S. News Headlines,” Workshop on News Media and Computational Journalism (MEDIATE), AAAI International Conference on Web and Social Media (ICWSM), Limassol, Cyprus, June 2023.
Hanjia Lyu, Arsal Imtiaz, Yufei Zhao, and Jiebo Luo, “Human Behavior in the Time of COVID-19: Learning from Big Data,” Frontiers in Big Data, 2023.
Xiaofei Zhou, Hanjia Lyu, Jiebo Luo, and Zhen Bai, “ML-SD Modeling: How Machine Learning Can Support Scientific Discovery Learning for K-12 STEM Education,” Artificial Intelligence for Education Workshop (AI4Edu), AAAI Conference on Artificial Intelligence (AAAI), Washington DC, February 2023.

2022

Sajid Hussain Rafi Ahamed, Shahid Shakil, Hanjia Lyu, Xinping Zhang, and Jiebo Luo, “Doctors vs. Nurses: Understanding the Great Divide in Vaccine Hesitancy among Healthcare Workers,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Osaka, Japan, December 2022.
Timothy Dye, Lisette Alcántara, Hanjia Lyu, Shazia Siddiqi, Saloni Sharma, Eva Pressman, and Jiebo Luo, “Oppression, COVID Vaccination, and Vaccine Sentiments in a Global Sample,” Annals of Epidemiology (ACE Abstracts), 2022.
Yangxin Fan, Hanjia Lyu, Jin Xiao, and Jiebo Luo, “American Twitter Users Revealed Social Determinants-related Oral Health Disparities amid the COVID-19 Pandemic,” Quintessence International, 2022.
Hanjia Lyu, and Jiebo Luo, “Understanding Political Polarization via Jointly Modeling Users, Connections and Multimodal Contents on Heterogeneous Graphs,” ACM Multimedia Conference (ACM MM), Lisboa, Portugal, October 2022.
Kevin C. Kapcio, Hanjia Lyu, Kyle C. Purrman, Christian G. Peyre, and Jiebo Luo, Carolyn E. Jones, and Michal J. Lada, “Applying Machine Learning to Predict Esophageal Cancer Recurrence After Esophagectomy,” Supplemental Issue of Journal of the American College of Surgeons, ACS Clinical Congress, San Diego, CA, October 2022.
Arsal Imtiaz, Danish Khan, Hanjia Lyu, and Jiebo Luo, “Taking sides: Public Opinion over the Israel-Palestine Conflict in 2021,” International Workshop on Social Sensing (SocialSens): Special Edition on Belief Dynamics, AAAI International Conference on Web and Social Media (ICWSM), Atlanta, GA and online, June 2022.
Yichi Qian, Qiyi Shan, Hanjia Lyu, and Jiebo Luo, “Look behind the Censorship: Reposting-User Characterization and Muted-Topic Restoration,” International Workshop on Social Sensing (SocialSens): Special Edition on Belief Dynamics, AAAI International Conference on Web and Social Media (ICWSM), Atlanta, GA and online, June 2022.
Wei Zhu, Zihe Zheng, Haitian Zheng, Hanjia Lyu, and Jiebo Luo, “Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis,” International Conference on Pattern Recognition (ICPR), Montréal, August 2022.
Xiaofei Zhou, Jingwan Tang, Beilei Guo, Hanjia Lyu, and Zhen Bai, “Challenges and Design Opportunities in Data Analysis for ML-Empowered Scientific Inquiry – Insights from a Teacher Professional Development Study,” International Conference of the Learning Sciences (ICLS), Virtual, June 2022.
Hanjia Lyu, Zihe Zheng, and Jiebo Luo, “Misinformation versus Facts: Understanding the Influence of News Regarding COVID-19 Vaccines on Vaccine Uptake,” Health Data Science, 2022.

2021

Hanjia Lyu, Yangxin Fan, Ziyu Xiong, Mayya Komisarchik, and Jiebo Luo, “Understanding Public Opinion toward the #StopAsianHate Movement and the Relation with Racially Motivated Hate Crimes in the US,” IEEE Transactions on Computational Social Systems (TCSS), 2021.
Tanqiu Jiang, Sidhant Bendre, Hanjia Lyu, and Jiebo Luo, “From Static to Dynamic Prediction: Wildfire Risk Assessment Based on Multiple Environmental Factors,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Virtual, December 2021.
Xupin Zhang, Hanjia Lyu, and Jiebo Luo, “Understanding the Hoarding Behaviors during the COVID-19 Pandemic using Large Scale Social Media Data,” Special Session on Intelligent Data Mining, IEEE Big Data Conference, Virtual, December 2021.
Hanjia Lyu, Junda Wang, Wei Wu, Viet Duong, Xiyang Zhang, Timothy D. Dye, and Jiebo Luo, “Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination,” Intelligent Medicine, 2021.
Ziyu Xiong, Pin Li, Hanjia Lyu, and Jiebo Luo, “Social Media Opinions on Working From Home in the United States During the COVID-19 Pandemic: Observational Study,” Journal of Medical Internet Research: Medical Informatics, 2021.
Wei Wu, Hanjia Lyu, and Jiebo Luo, “Characterizing Discourse about COVID-19 Vaccines: A Reddit Version of the Pandemic Story,” Health Data Science, 2021.
Xupin Zhang, Hanjia Lyu, and Jiebo Luo, “What Contributes to a Crowdfunding Campaign’s Success? Evidence and Analyses from GoFundMe Data,” IEEE Journal of Social Computing, 2021.
Xiyang Zhang, Yu Wang, Hanjia Lyu, Yipeng Zhang, Yubao Liu, and Jiebo Luo, “The Influence of COVID-19 on people’s Well-Being: Big Data Methods for Capturing Working Adults’ Well-being and Protective Factors Nationwide,” Frontiers in Psychology, 2021.
Long Chen, Hanjia Lyu, Tongyu Yang, Yu Wang, and Jiebo Luo, “Fine-Grained Analysis of the Use of Neutral and Controversial Terms for COVID-19 on Social Media,” International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Karan Vombatkere, Hanjia Lyu, and Jiebo Luo, “How Political is the Spread of COVID-19 in the United States? An Analysis using Transportation and Weather Data,” International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Yipeng Zhang, Hanjia Lyu, Yubao Liu, Xiyang Zhang, Yu Wang, and Jiebo Luo, “Monitoring Depression Trends on Twitter During the COVID-19 Pandemic: Observational Study,” Journal of Medical Internet Research: Infodemiology, 2021.

2020

Siqing Cao, Hanjia Lyu, and Xian Xu, “InsurTech development: Evidence from Chinese media reports,” Technological Forecasting and Social Change, 2020.
Hanjia Lyu, Long Chen, Yu Wang, and Jiebo Luo, “Sense and Sensibility: Characterizing social media users regarding the use of controversial terms for COVID-19,” IEEE Transactions on Big Data (TBD), 2020.

Other Fun Stuff

[10/10/2024] I was able to watch the magnificent aurora together with some members of VIStA at Webster Pier in Rochester, NY after missing a chance to see one in Vienna back in May.

[05/04/2023] For some reason, Maggie and I decided to order dinner to go. While we waited by the front door of a restaurant in Santa Clara, we caught sight of this beautiful rainbow. Even better, we were lucky to meet a lovely couple who offered to take a picture of us.

Template borrowed from Jon Barron. Thanks!