Tag Archives: big data

Strategic Network Formation in a Location-Based Social Network: A Topic Modeling Approach (HICSS 2016)

Lee, G. M., Qiu, L., Whinston, A. B. (2016). Strategic Network Formation in a Location-Based Social Network: A Topic Modeling Approach, Proceedings of Hawaii International Conference on System Sciences (HICSS 2016), Kauai, Hawaii. Nominated for Best Paper Award

This paper studies strategic network formation in a location-based social network. We build a structural model of social link creation that incorporates individual characteristics and pairwise user similarities. Specifically, we define four user proximity measures from biography, geography, mobility, and short messages. To construct proximity from unstructured text information, we build topic models using latent Dirichlet allocation. Using Gowalla data with 385,306 users, three million locations, and 35 million check-in records, we empirically estimate the structural model to find evidence on the homophily effect in network formation.

AppPrint: Automatic Fingerprinting of Mobile Applications in Network Traffic (PAM 2015)

Miskovic, S., Lee, G. M., Liao, Y., and Baldi, M. (2015). AppPrint: Automatic Fingerprinting of Mobile Applications in Network Traffic, In Proceedings of Passive and Active Measurement Conference (PAM 2015), New York, New York.

  • Based on an industry collaboration with Narus (then Boeing subsidiary, now acquired by Symantec)
  • PAM is a premier conference in the network measurement area (h5-index: 24).

Increased adoption of mobile devices introduces a new spin to the Internet: mobile apps are becoming a key source of user traffic. Surprisingly, service providers and enterprises are largely unprepared for this change as they increasingly lose understanding of their traffic and fail to persistently identify individual apps. App traffic simply appears no different than any other HTTP data exchange. This raises a number of concerns for security and network management. In this paper, we propose AppPrint, a system that learns fingerprints of mobile apps via comprehensive traffic observations. We show that these fingerprints identify apps even in small traffic samples where app identity cannot be explicitly revealed in any individual traffic flows. This unique AppPrint feature is crucial because explicit app identifiers are extremely scarce, leading to a very limited characterization coverage of the existing approaches. In fact, our experiments on a nation-wide dataset from a major cellular provider show that AppPrint significantly outperforms any existing app identification. Moreover, the proposed system is robust to the lack of key app-identification sources, i.e., the traffic related to ads and analytic services commonly leveraged by the state-of-the-art identification methods.

Towards a Better Measure of Business Proximity: Topic Modeling for Analyzing M&As (EC 2014)

Shi, Z., Lee, G. M., Whinston, A. B. (2014). Towards a Better Measure of Business Proximity: Topic Modeling for Analyzing M&As, Proceedings of ACM Conference on Economics and Computation (EC 2014), Palo Alto, California

In this article, we propose a new data-analytic approach to measure firms’ dyadic business proximity. Specifically, our method analyzes the unstructured texts that describe firms’ businesses using the statistical learning technique of topic modeling, and constructs a novel business proximity measure based on the output. When compared with existent methods, our approach is scalable for large datasets and provides finer granularity on quantifying firms’ positions in the spaces of product, market, and technology. We then validate our business proximity measure in the context of industry intelligence and show the measure’s effectiveness in an empirical application of analyzing mergers and acquisitions in the U.S. high technology industry. Based on the research, we also build a cloud-based information system to facilitate competitive intelligence on the high technology industry.

Event Detection using Customer Care Calls (INFOCOM 2013)

Chen, Y., Lee, G. M., Duffield, N., Qiu, L., and Wang, J. (2013). Event Detection using Customer Care Calls. In Proceedings of IEEE International Conference on Computer Communications (INFOCOM 2013), Turin, Italy.

  • Based on an industry collaboration with AT&T Labs – Research.
  • INFOCOM is a top-tier conference in the networking area (h5-index: 72)

Customer care calls serve as a direct channel for a service provider to learn feedbacks from their customers. They reveal details about the nature and impact of major events and problems observed by customers. By analyzing customer care calls, a service provider can detect important events to speed up problem resolution. However, automating event detection based on customer care calls poses several significant challenges. First, the relationship between customers’ calls and network events is blurred because customers respond to an event in different ways. Second, customer care calls can be labeled inconsistently across agents and across call centers, and a given event naturally gives rise to calls spanning a number of categories. Third, many important events cannot be detected by looking at calls in one category. How to aggregate calls from different categories for event detection is important but challenging. Lastly, customer care call records have high dimensions (e.g., thousands of categories in our dataset). In this paper, we propose a systematic method for detecting events in a major cellular network using customer care call data. It consists of three main components: (i) using a regression approach that exploits temporal stability and low-rank properties to automatically learn the relationship between customer calls and major events, (ii) reducing the number of unknowns by clustering call categories and using L 1 norm minimization to identify important categories, and (iii) employing multiple classifiers to enhance the robustness against noise and different response time. For the detected events, we leverage Twitter social media to summarize them and to locate the impacted regions. We show the effectiveness of our approach using data from a large cellular service provider in the US.

Improving Sketch Reconstruction Accuracy Using Linear Least Square Method (IMC 2005)

Lee, G. M., Liu, H., Yoon, Y., and Zhang, Y. (2005). Improving Sketch Reconstruction Accuracy Using Linear Least Square Method, In Proceedings of Internet Measurement Conference (IMC 2005), Berkeley, California.

  • IMC is a premier conference in the network measurement area (h5-index: 37)

Sketch is a sublinear space data structure that allows one to approximately reconstruct the value associated with any given key in an input data stream. It is the basis for answering a number of fundamental queries on data streams, such as range queries, finding quantiles, frequent items, etc. In the networking context, sketch has been applied to identifying heavy hitters and changes, which is critical for traffic monitoring, accounting, and network anomaly detection.

In this paper, we propose a novel approach called lsquare to significantly improve the reconstruction accuracy of the sketch data structure. Given a sketch and a set of keys, we estimate the values associated with these keys by constructing a linear system and finding the optimal solution for the system using linear least squares method. We use a large amount of real Internet traffic data to evaluate lsquare against countmin, the state-of-the-art sketch scheme. Our results suggest that given the same memory requirement, lsquare achieves much better reconstruction accuracy than countmin. Alternatively, given the same reconstruction accuracy, lsquare requires significantly less memory. This clearly demonstrates the effectiveness of our approach.