Tag Archives: computer vision

Learning Faces to Predict Matching Probability in an Online Dating Market

Kwon, Soonjae, Sung-Hyuk Park, Gene Moo Lee, Dongwon Lee. “Learning Faces to Predict Matching Probability in an Online Dating Market”. Working Paper.

  • Presentations: DS (2021), AIMLBA (2021), WITS (2021), ICIS (2022)
  • Based on an industry collaboration

With the increasing use of online matching platforms, predicting matching probability between users is crucial for efficient market design. Although previous studies have constructed various visual features to predict matching probability, facial features, which are important in online matching, have not been widely used. We find that deep learning-enabled facial features can significantly enhance the prediction accuracy of a user’s partner preferences from the individual rating prediction analysis in an online dating market. We also build prediction models for each gender and use prior theories to explain different contributing factors of the models. Furthermore, we propose a novel method to visually interpret facial features using the generative adversarial network (GAN). Our work contributes to the literature by providing a framework to develop and interpret facial features to investigate underlying mechanisms in online matching markets. Moreover, matching platforms can predict matching probability more accurately for better market design and recommender systems.

IS / Marketing Papers on Visual Data Analytics (Image, Video)

Last update: June 9, 2022

With the advent of social media and mobile platforms, visual data are becoming the first citizen in big data analytics research. Compared to textual data that require significant cognitive efforts to comprehend, visual data (such as images and videos) can easily convey the message from the content creator to the general audience. To conduct large-scale studies on such data types, researchers need to use machine learning and computer vision approaches. In this post, I am trying to organize studies in Information Systems, Marketing, and other management disciplines that leverage large-scale analysis of image and video datasets. The papers are ordered randomly:

  1. Bharadwaj, N., Ballings, M., Naik, P. A., Moore, M, Arat, M. M. (2022) “A New Livestream Retail Analytics Framework to Assess the Sales Impact of Emotional Displays,” Journal of Marketing, 86(1): 24-47.
  2. Chen, Z., Liu, Y.-J., Meng, J., Wang, Z. (2022) “What’s in a Face? An Experiment on Facial Information and Loan-Approval Decision“, Management Science, forthcoming.
  3. Lu, T., Wang, A., Yuan, X., Zhang, X. (2020) “Visual Distortion Bias in Consumer Choices,” Management Science, forthcoming.
  4. Zhou, M., Chen, G. H., Ferreira, P., Smith, M. D. (2021) “Consumer Behavior in the Online Classroom: Using Video Analytics and Machine Learning to Understand the Consumption of Video Courseware,” Journal of Marketing Research 58(6): 1079-1100.
  5. Zhang, S., Lee, D., Singh, P. V., Srinivasan, K. (2021) “What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features,” Management Science, forthcoming.
  6. Gunarathne, P., Rui, H., Seidmann, A. (2021) “Racial Bias in Customer Service: Evidence from Twitter,” Information Systems Research 33(1): 43-54.
  7. Shin, D., He, S., Lee, G. M., Whinston, A. B., Cetintas, S., Lee, K.-C. (2020) “Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach,” MIS Quarterly 44(4): 1459-1492[Details]
  8. Li, Y., Xie, Y. (2020) “Is a Picture Worth a Thousand Words? An Empirical Study of Image Content and Social Media Engagement,” Journal of Marketing Research 57(2): 1-19.
  9. Zhang, Q., Wang, W., Chen, Y. (2020) “Frontiers: In-Consumption Social Listening with Moment-to-Moment Unstructured Data: The Case of Movie Appreciation and Live comments,” Marketing Science 39(2).
  10. Liu, L., Dzyabura, D., Mizik, N. (2020) “Visual Listening In: Extracting Brand Image Portrayed on Social Media,Marketing Science 39(4): 669-686.
  11. Peng, L., Cui, G., Chung, Y., Zheng, W. (2020) “The Faces of Success: Beauty and Ugliness Premiums in E-Commerce Platforms,” Journal of Marketing 84(4): 67-85.
  12. Liu, X., Zhang, B., Susarla, A., Padman, R. (2020) “Go to YouTube and Call Me in the Morning: Use of Social Media for Chronic Conditions,” MIS Quarterly 44(1b): 257-283.
  13. Zhao, K., Hu, Y., Hong, Y., Westland, J. C. (2020) “Understanding Characteristics of Popular Streamers in Live Streaming Platforms: Evidence from Twitch.tv,” Journal of the Association for Information Systems, Forthcoming.
  14. Ordenes, F. V., Zhang, S. (2019) “From words to pixels: Text and image mining methods for service research,” Journal of Service Management 30(5): 593-620.
  15. Wang, Q., Li, B., Singh, P. V. (2018) “Copycats vs. Original Mobile Apps: A Machine Learning Copycat-Detection Method and Empirical Analysis,” Information Systems Research 29(2): 273-291.
  16. Lu, S., Xiao, L., Ding, M. (2016) “A Video-Based Automated Recommender (VAR) System for Garments,” Marketing Science 35(3): 484-510.
  17. Xiao, L., Ding, M. (2014) “Just the Faces: Exploring the Effects of Facial Features in Print Advertising,” Marketing Science 33(3), 315-461.
  18. Suh, K.-S., Kim, H., Suh, E. K. (2011) “What If Your Avatar Looks Like You? Dual-Congruity Perspectives for Avatar Use,” MIS Quarterly 35(3), 711-729.
  19. Todorov, A., Porter, J. M. (2014) “Misleading First Impressions: Different for Different Facial Images of the Same Person“, Psychological Science 25(7): 1404-1417.
  20. Todorov, A., Madnisodza, A. N., Goren, A., Hall, C. C. (2005) “Inferences of Competence from Faces Predict Election Outcomes“, Science 308(5728): 1623-1626.
  21. Mueller. U., Mazur, A. (1996) “Facial Dominance of West Point Cadets as a Predictor of Later Military Rank“, Social Forces 74(3): 823-850.
  22. Lee, H, Nam, K. “When Machine Vision Meets Human Fashion: Effects of Human Intervention on the Efficiency of CNN-Driven Recommender Systems in Online Fashion Retail”, Working Paper.
  23. Lysyhakov M, Viswanathan S (2021) “Threatened by AI: Analyzing users’ responses to the introduction of AI in a crowd-sourcing,” Working Paper.
  24. Park, S., Lee, G. M., Shin, D., Han, S.-P. (2020) “Targeting Pre-Roll Ads using Video Analytics,” Working Paper.
  25. Choi, A., Ramaprasad, J., So, H. (2021) Does Authenticity of Influencers Matter? Examining the Impact on Purchase Decisions, Working Paper.
  26. Park, J., Kim, J., Cho, D., Lee, B. Pitching in Character: The Role of Video Pitch’s Personality Style in Online Crowdsourcing, Working Paper.
  27. Yang, J., Zhang, J., Zhang Y. (2021) First Law of Motion: Influencer Video Advertising on TikTok, Working Paper.
  28. Davila, A., Guasch (2021) Manager’s Body Expansiveness, Investor Perceptions, and Firm Forecast Errors and Valuation, Working Paper.
  29. Peng, L., Teoh, S. H., Wang, U., Yan, J. (2021) Face Value: Trait Inference, Performance Characteristics, and Market Outcomes for Financial Analysts, Working Paper.
  30. Zhang, S., Friedman, E., Zhang, X., Srinivasan, K., Dhar, R. (2020) Serving with a Smile on Airbnb: Analyzing the Economic Returns and Behavioral Underpinnings of the Host’s Smile,” Working Paper.
  31. Park, K., Lee, S., Tan, Y. (2020) “What Makes Online Review Videos Helpful? Evidence from Product Review Videos on YouTube,” UW Working Paper.
  32. Doosti, S., Lee, S., Tan, Y. (2020) “Social Media Sponsorship: Metrics for Finding the Right Content Creator-Sponsor Matches,” UW Working Paper.
  33. Koh, B., Cui, F. (2020) “Give a Gist: The Impact of Thumbnails on the View-Through of Videos,” KU Working Paper.
  34. Hou J.R., Zhang J., Zhang K. (2018) Can title images predict the emotions and the performance of crowdfunding projects? Workshop on e-Business.

Trustworthy Face? The Effect and Drivers of Comprehensive Trust in Online Job Market Platform

Kwon, Jun Bum, Donghyuk Shin, Gene Moo Lee, Jake An, Sam Hwang (2020) “Trustworthy Face? The Effect and Drivers of Comprehensive Trust in Online Job Market Platform”. Work-in-progress.

The abstract will appear here.

Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach (MISQ 2020)

Shin, Donghyuk, Shu He, Gene Moo Lee, Andrew B. Whinston, Suleyman Cetintas, Kuang-Chih Lee (2020) “Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach,” MIS Quarterly, 44(4), pp. 1459-1492. [SSRN]

  • Based on an industry collaboration with Yahoo! Research
  • The first MISQ methods article based on machine learning
  • Presented in WeB (Fort Worth, TX 2015), WITS (Dallas, TX 2015), UT Arlington (2016), Texas FreshAIR (San Antonio, TX 2016), SKKU (2016), Korea Univ. (2016), Hanyang (2016), Kyung Hee (2016), Chung-Ang (2016), Yonsei (2016), Seoul National Univ. (2016), Kyungpook National Univ. (2016), UKC (Dallas, TX 2016), UBC (2016), INFORMS CIST (Nashville, TN 2016), DSI (Austin, TX 2016), Univ. of North Texas (2017), Arizona State (2018), Simon Fraser (2019), Saarland (2021), Kyung Hee (2021), Tennessee Chattanooga (2021), Rochester (2021), KAIST (2021), Yonsei (2021), UBC (2022)

This research methods article proposes a visual data analytics framework to enhance social media research using deep learning models. Drawing on the literature of information systems and marketing, complemented with data-driven methods, we propose a number of visual and textual content features including complexity, similarity, and consistency measures that can play important roles in the persuasiveness of social media content. We then employ state-of-the-art machine learning approaches such as deep learning and text mining to operationalize these new content features in a scalable and systematic manner. For the newly developed features, we validate them against human coders on Amazon Mechanical Turk. Furthermore, we conduct two case studies with a large social media dataset from Tumblr to show the effectiveness of the proposed content features. The first case study demonstrates that both theoretically motivated and data-driven features significantly improve the model’s power to predict the popularity of a post, and the second one highlights the relationships between content features and consumer evaluations of the corresponding posts. The proposed research framework illustrates how deep learning methods can enhance the analysis of unstructured visual and textual data for social media research.