We consider the clustering problem in single cell data with the existence of both (i) pure cells, each belongning to one single cluster, and (ii) mixed cells (transitional cells), that are transitioning between cell types. We propose SOUP, for Semi-sOft clUstering with Pure cells, which performs well at the hard-clustering problem for pure cell types and excels at identifying transitional cells with soft memberships. Notably, SOUP is derived under a generic additive noise model and can be versatile and applicable to a wider range of applications.

Recent advances of single cell sequencing yield valuable insights about individual cells. However, compared to traditional tissue-level bulk RNA-seq, analyzing single cell data is challenging because of high levels of technical noise. Here, we proposed a unified statistical framework for joint analysis of single cell and bulk RNA-seq data. Our hierarchical model borrows the strength from both data sources, identifies noisy entries in single cell data, and leads to a more accurate estimation of gene expression profiles.

At Uber, probabilistic time series forecasting is used for robust prediction of number of trips during special events, driver incentive allocation, as well as real-time anomaly detection across millions of metrics. Classical time series models are often hard to tune, scale, and add exogenous variables to. In this project, we proposed a novel end-to-end LSTM-based Bayesian deep model that provides time series prediction along with uncertainty estimation. This model has been successfully applied to large-scale time series anomaly detection at Uber. This work was completed during an internship.

Two-sample test for high-dimensional covariance matrices is a long-lasting and challenging statistical problem. Here, we proposed a novel perspective that utilizes the spectrum of the differential matrix, which is closely related with Sparse Principal Component Analysis. Our test remains powerful even when the signals are sparse and weak, and outperforms existing approaches under various practical scenarios. We successfully applied our method to the largest gene­ expression dataset comparing schizophrenia and control brains, and revealed intriguing patterns of gene-gene co­-expression change.

In this project, we study the problem of community recovery in stochastic block models. We show that a simple sample splitting trick can refine almost any approximately correct community recovery method to achieve exactly correct community recovery, when the expected node degrees are of order log(n) or higher. Our results simplify and extend some of the previous work on exact community recovery using sample splitting, and provide better theoretical guarantee for degree corrected stochastic block models.