Abstract: In the context of highway merging scenarios where ramp vehicles encounter rear vehicles on the main lane, a significant challenge arises due to the competition for the right of way, exacerbated by the stochastic nature of driving styles. This situation can lead to traffic congestion and even collisions if not managed effectively. To address these issues, this paper presents an optimal cooperative merging strategy based on Bayesian Nash Equilibrium for connected and automated vehicles. The approach begins by analyzing the inherent randomness in driving styles exhibited at on-ramps. Specifically, a Principal Component Analysis method is applied to extract key features with lower dimensions. These features are then used to estimate the probability distributions of driving styles for both ramp and mainline vehicles. Subsequently, a cooperative merging model is developed, taking into account the obtained probability distributions of driving styles. This model leverages the Markov Bayesian Game Decision Process framework to represent the decision-making interactions between mainline and ramp vehicles. Furthermore, a Deep Reinforcement Learning Framework integrated with Bayesian Game is proposed, to learn and derive the optimal cooperative merging strategy under stochastic driving styles. Simulation results indicate that the proposed model can make feasible and reasonable decisions at on-ramps, effectively avoiding collision accidents caused by stochastic driving styles.