Reinforcement learning (RL) has emerged as a powerful method for solving complex control tasks across various domains, from autonomous driving to maritime navigation. Work of my team in RL, particularly in value-based algorithms, addresses critical issues such as overestimation bias, proposing innovative solutions like the T-Estimator (TE) and K-Estimator (KE) for bias control and algorithmic robustness. Our advancements are validated through modifications to Q-Learning and the Bootstrapped Deep Q-Network (BDQN), demonstrating superior performance and convergence. Additionally, we have developed a spatial-temporal recurrent neural network architecture for autonomous ships, enhancing robustness in partial observability and compliance with maritime traffic rules. Our recent endeavors also include a modular framework for autonomous surface vehicles on inland waterways, utilizing DRL agents for path planning and following, significantly outperforming traditional control methods. Moreover, our work on dynamic obstacle avoidance environments for mobile robots and drones emphasizes the importance of controlled training difficulty for better generalization and robustness. This approach has been successfully applied across different platforms, reducing the simulation-to-reality (Sim2Real) gap and improving performance in real-world scenarios. Through these contributions, we aim to advance the practical application and reliability of reinforcement learning in diverse and dynamic environments.