Abstract: As intelligent vehicles usually have complex overtaking process, a safe and efficient automated overtaking system (AOS) is vital to avoid accidents caused by wrong operation of drivers. Existing AOSs rarely consider longitudinal reactions of the overtaken vehicle (OV) during overtaking. This paper proposed a novel AOS based on hierarchical reinforcement learning, where the longitudinal reaction is given by a data-driven social preference estimation. This AOS incorporates two modules that can function in different overtaking phases. The first module based on semi-Markov decision process and motion primitives is built for motion planning and control. The second module based on Markov decision process is designed to enable vehicles to make proper decisions according to the social preference of OV. Based on realistic overtaking data, the proposed AOS and its modules are verified experimentally. The results of the tests show that the proposed AOS can realize safe and effective overtaking in scenes built by realistic data, and has the ability to flexibly adjust lateral driving behavior and lane changing position when the OVs have different social preferences.