Tag Archives: computer vision

Learning Faces to Predict Matching Probability in an Online Dating Market

Kwon, Soonjae, Sung-Hyuk Park, Gene Moo Lee, Dongwon Lee (2021) “Learning Faces to Predict Matching Probability in an Online Dating Market”. Work-in-progress.

  • Under review for a conference presentation.
  • Based on an industry collaboration

With the increasing use of online matching markets, predicting the matching probability among users is crucial for better market design. Although previous studies have constructed visual features to predict the matching probability, facial features extracted by deep learning have not been widely used. By predicting user attractiveness in an online dating market, we find that deep learning-enabled facial features can significantly enhance prediction accuracy. We also predict the attractiveness at various evaluator groups and explain their different preferences based on the theory of evolutionary psychology. Furthermore, we propose a novel method to visually interpret deep learning-enabled facial features using the latest deep learning-based generative model. Our work contributes to IS researchers utilizing facial features using deep learning and interpreting them to investigate underlying mechanisms in online matching markets. From a practical perspective, matching platforms can predict matching probability more accurately for better market design and recommender systems for maximizing the matching outcome.

IS / Marketing Papers on Visual Data Analytics (Image, Video)

Last update: July 19, 2021

With the advent of social media and mobile platforms, visual data are becoming the first citizen in big data analytics research. Compared to textual data that require significant cognitive efforts to comprehend, visual data (such as image and video) can easily convey the message from the content creator to the general audience. To conduct large-scale studies on such data types, researchers need to use machine learning and computer vision approaches. In this post, I am trying to organize studies in Information Systems, Marketing, and other management disciplines that leverage large-scale analysis of image and video datasets. The papers are ordered randomly:

  1. Zhou, M., Chen, G. H., Ferreira, P., Smith, M. D. (2021) “Consumer Behavior in the Online Classroom: Using Video Analytics and Machine Learning to Understand the Consumption of Video Courseware,” Journal of Marketing Research, forthcoming.
  2. Shin, D., He, S., Lee, G. M., Whinston, A. B., Cetintas, S., Lee, K.-C. (2020) “Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach,” MIS Quarterly 44(4): 1459-1492[Details]
  3. Li, Y., Xie, Y. (2020) “Is a Picture Worth a Thousand Words? An Empirical Study of Image Content and Social Media Engagement,” Journal of Marketing Research 57(2): 1-19.
  4. Zhang, Q., Wang, W., Chen, Y. (2020) “Frontiers: In-Consumption Social Listening with Moment-to-Moment Unstructured Data: The Case of Movie Appreciation and Live comments,” Marketing Science 39(2).
  5. Liu, L., Dzyabura, D., Mizik, N. (2020) “Visual Listening In: Extracting Brand Image Portrayed on Social Media,Marketing Science 39(4): 669-686.
  6. Peng, L., Cui, G., Chung, Y., Zheng, W. (2020) “The Faces of Success: Beauty and Ugliness Premiums in E-Commerce Platforms,” Journal of Marketing 84(4): 67-85.
  7. Liu, X., Zhang, B., Susarla, A., Padman, R. (2020) “Go to YouTube and Call Me in the Morning: Use of Social Media for Chronic Conditions,” MIS Quarterly 44(1b): 257-283.
  8. Park, S., Lee, G. M., Shin, D., Han, S.-P. (2020) “Targeting Pre-Roll Ads using Video Analytics,” Working Paper.
  9. Zhao, K., Hu, Y., Hong, Y., Westland, J. C. (2020) “Understanding Characteristics of Popular Streamers in Live Streaming Platforms: Evidence from Twitch.tv,” Journal of the Association for Information Systems, Forthcoming.
  10. Ordenes, F. V., Zhang, S. (2019) “From words to pixels: Text and image mining methods for service research,” Journal of Service Management 30(5): 593-620.
  11. Wang, Q., Li, B., Singh, P. V. (2018) “Copycats vs. Original Mobile Apps: A Machine Learning Copycat-Detection Method and Empirical Analysis,” Information Systems Research 29(2): 273-291.
  12. Lu, S., Xiao, L., Ding, M. (2016) “A Video-Based Automated Recommender (VAR) System for Garments,” Marketing Science 35(3): 484-510.
  13. Xiao, L., Ding, M. (2014) “Just the Faces: Exploring the Effects of Facial Features in Print Advertising,” Marketing Science 33(3), 315-461.
  14. Zhang, S., Lee, D., Singh, P. V., Srinivasan, K. (2018) “How Much is an Image Worth? Airbnb Property Demand Estimation Leveraging Large Scale Image Analytics,” CMU Working Paper.
  15. Choi, A., Ramaprasad, J., So, H. (2021) Does Authenticity of Influencers Matter? Examining the Impact on Purchase Decisions, Working Paper.
  16. Park, J., Kim, J., Cho, D., Lee, B. Pitching in Character: The Role of Video Pitch’s Personality Style in Online Crowdsourcing, Working Paper.
  17. Yang, J., Zhang, J., Zhang Y. (2021) First Law of Motion: Influencer Video Advertising on TikTok, Working Paper.
  18. Davila, A., Guasch (2021) Manager’s Body Expansiveness, Investor Perceptions, and Firm Forecast Errors and Valuation, Working Paper.
  19. Peng, L., Teoh, S. H., Wang, U., Yan, J. (2021) Face Value: Trait Inference, Performance Characteristics, and Market Outcomes for Financial Analysts, Working Paper.
  20. Zhang, S., Friedman, E., Zhang, X., Srinivasan, K., Dhar, R. (2020) Serving with a Smile on Airbnb: Analyzing the Economic Returns and Behavioral Underpinnings of the Host’s Smile,” Working Paper.
  21. Park, K., Lee, S., Tan, Y. (2020) “What Makes Online Review Videos Helpful? Evidence from Product Review Videos on YouTube,” UW Working Paper.
  22. Doosti, S., Lee, S., Tan, Y. (2020) “Social Media Sponsorship: Metrics for Finding the Right Content Creator-Sponsor Matches,” UW Working Paper.
  23. Koh, B., Cui, F. (2020) “Give a Gist: The Impact of Thumbnails on the View-Through of Videos,” KU Working Paper.
  24. Gunarathne, P., Rui, H., Seidman, A. (2019) “Racial Discrimination in Social Media Customer Service: Evidence from a Popular Microblogging Platform,” Rochester Working Paper.
  25. Hou J.R., Zhang J., Zhang K. (2018) Can title images predict the emotions and the performance of crowdfunding projects? Workshop on e-Business.

Trustworthy Face? The Effect and Drivers of Comprehensive Trust in Online Job Market Platform

Kwon, Jun Bum, Donghyuk Shin, Gene Moo Lee, Jake An, Sam Hwang (2020) “Trustworthy Face? The Effect and Drivers of Comprehensive Trust in Online Job Market Platform”. Work-in-progress.

The abstract will appear here.

Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach (MISQ 2020)

Shin, Donghyuk, Shu He, Gene Moo Lee, Andrew B. Whinston, Suleyman Cetintas, Kuang-Chih Lee (2020) “Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach,” MIS Quarterly, 44(4), pp. 1459-1492. [SSRN]

  • Based on an industry collaboration with Yahoo! Research
  • The first MISQ methods article based on machine learning
  • Presented in WeB (Fort Worth, TX 2015), WITS (Dallas, TX 2015), UT Arlington (2016), Texas FreshAIR (San Antonio, TX 2016), SKKU (2016), Korea Univ. (2016), Hanyang (2016), Kyung Hee (2016), Chung-Ang (2016), Yonsei (2016), Seoul National Univ. (2016), Kyungpook National Univ. (2016), UKC (Dallas, TX 2016), UBC (2016), INFORMS CIST (Nashville, TN 2016), DSI (Austin, TX 2016), Univ. of North Texas (2017), Arizona State (2018), Simon Fraser (2019), Saarland (2021), Kyung Hee (2021)

This research methods article proposes a visual data analytics framework to enhance social media research using deep learning models. Drawing on the literature of information systems and marketing, complemented with data-driven methods, we propose a number of visual and textual content features including complexity, similarity, and consistency measures that can play important roles in the persuasiveness of social media content. We then employ state-of-the-art machine learning approaches such as deep learning and text mining to operationalize these new content features in a scalable and systematic manner. For the newly developed features, we validate them against human coders on Amazon Mechanical Turk. Furthermore, we conduct two case studies with a large social media dataset from Tumblr to show the effectiveness of the proposed content features. The first case study demonstrates that both theoretically motivated and data-driven features significantly improve the model’s power to predict the popularity of a post, and the second one highlights the relationships between content features and consumer evaluations of the corresponding posts. The proposed research framework illustrates how deep learning methods can enhance the analysis of unstructured visual and textual data for social media research.