Tag Archives: visual representation

VISAGE: Designing AI Artifacts for Dynamic Self-Presentation in Matching Platforms

Kwon, Soonjae, Gene Moo Lee, Dongwon Lee, Sung-Hyuk Park (2024) “VISAGE: Designing AI Artifacts for Dynamic Self-Presentation in Matching Platforms,” Working Paper.

  • Previous title: Learning Faces to Predict Matching Probability in an Online Dating Market
  • Presentations: DS (2021), AIMLBA (2021), WITS (2021), ICIS (2022)
  • Preliminary version in ICIS 2022 Proceedings
  • Based on an industry collaboration

Online matching platforms constrain users to static profiles, producing a mismatch between the idealized self a user presents and the heterogeneous preferences of potential partners. Drawing on self-discrepancy theory, we conceptualize this mismatch as an interpersonal gap between one’s presented self and what each partner desires to see, with AI serving as a mediator to help address it. Following the computational design science perspective, we propose VISAGE, an AI system comprising two artifacts grounded in distinct human-AI collaboration principles. The augmentation artifact selects optimal images from users’ existing assets, whereas the assemblage artifact generates new images tailored to individual partner preferences. Using large-scale operational data from a major online dating platform, we evaluate VISAGE at both the user and platform levels. At the user level, model-predicted ratings suggest that both artifacts improve attractiveness ratings. Relative effectiveness varies with partner-preference heterogeneity and user impression management skill, consistent with theoretical predictions from the human-AI collaboration literature. At the platform level, agent-based simulations suggest that VISAGE can enhance matching efficiency and reduce inequality in matching opportunities, although optimal deployment strategies depend on the platform’s recommendation algorithm. The theoretical contribution of this study is to foreground the interpersonal gap as a key source of matching inefficiency and to illustrate how AI can address it at scale. The design contribution lies in actionable design knowledge, including when to deploy augmentation versus assemblage artifacts based on user and partner characteristics and how to align user-facing AI features with backend algorithmic infrastructure.

Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach (MISQ 2020)

Shin, Donghyuk, Shu He, Gene Moo Lee, Andrew B. Whinston, Suleyman Cetintas, Kuang-Chih Lee (2020) Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach, MIS Quarterly, 44(4), pp. 1459-1492. [SSRN]

  • Based on an industry collaboration with Yahoo! Research
  • The first MISQ methods article based on machine learning
  • Presented in WeB (Fort Worth, TX 2015), WITS (Dallas, TX 2015), UT Arlington (2016), Texas FreshAIR (San Antonio, TX 2016), SKKU (2016), Korea Univ. (2016), Hanyang (2016), Kyung Hee (2016), Chung-Ang (2016), Yonsei (2016), Seoul National Univ. (2016), Kyungpook National Univ. (2016), UKC (Dallas, TX 2016), UBC (2016), INFORMS CIST (Nashville, TN 2016), DSI (Austin, TX 2016), Univ. of North Texas (2017), Arizona State (2018), Simon Fraser (2019), Saarland (2021), Kyung Hee (2021), Tennessee Chattanooga (2021), Rochester (2021), KAIST (2021), Yonsei (2021), UBC (2022), Temple (2023)

This research methods article proposes a visual data analytics framework to enhance social media research using deep learning models. Drawing on the literature of information systems and marketing, complemented with data-driven methods, we propose a number of visual and textual content features including complexity, similarity, and consistency measures that can play important roles in the persuasiveness of social media content. We then employ state-of-the-art machine learning approaches such as deep learning and text mining to operationalize these new content features in a scalable and systematic manner. For the newly developed features, we validate them against human coders on Amazon Mechanical Turk. Furthermore, we conduct two case studies with a large social media dataset from Tumblr to show the effectiveness of the proposed content features. The first case study demonstrates that both theoretically motivated and data-driven features significantly improve the model’s power to predict the popularity of a post, and the second one highlights the relationships between content features and consumer evaluations of the corresponding posts. The proposed research framework illustrates how deep learning methods can enhance the analysis of unstructured visual and textual data for social media research.