Deep learning has been successfully applied in many research fields, such as computer vision, speech recognition and natural language processing. Most of them are focused on single modality. On the other hand, multimodal information is more useful for practical applications. Multimodal deep learning has got a lot of attention and becomes an important issue in the field of artificial intelligence. Compared with traditional single-modal deep learning, there are following challenges in multimodal deep learning: development of multimodal dataset; multimodal representation; multimodal alignment; multimodal translation and multimodal co-learning. The propose of this talk is to introduce efficient and accurate multimodal deep learning methods and apply them to depression estimation.