Following breakthroughs in computer vision and natural language processing, widespread excitement and financial support have buoyed a rapidly-growing field of machine learning for healthcare. And yet, while most of machine learnings most impressive results concern point estimates under strict i.i.d. assumptions, medical decisionmaking often requires something more. Conditions shift (due to seasonality, changing prevalence of illnesses, and availability of tests), the quantities of interest are often counterfactual (causal) quantities, uncertainty quantification is often essential, and individuals are characterized by data from multiple modalities. In this talk, I will discuss recent breakthroughs in deep learning for healthcare, as well as my own group's work pioneering RNNs for multivariate clinical time series data and then focus on our more recent eforts to address aspects of decision-making that are fundamentally missing in the standard machine learning setup.