The government formally appointed veteran diplomat Shotaro Oshima as ambassador to South Korea on Friday.
Bilateral relations have been strained recently amid rekindled disputes over history and territory.
Oshima, 61, former ambassador in charge of inspecting the operations and funds of Japanese overseas missions, had served as ambassador to the Geneva-based International Organizations until April.
The appointment, endorsed Friday by the Cabinet, took effect the same day.
Oshima also has also served as deputy foreign minister, ambassador to Saudi Arabia, and the chief of the Foreign Ministry's Economic Affairs Bureau since joining the ministry in 1968.
A new post for former Ambassador to South Korea Toshiyuki Takano has yet to be announced.
The plan to replace Takano with Oshima was reported in June, after Takano's remarks on disputed islets in the Sea of Japan -- known here as Takeshima and in South Korea as Tok-do -- drew criticism in South Korea, which controls the islets.
But Japanese government sources denied that the change was related to Takano's remarks, saying it formed part of a routine personnel reshuffle as he had held the post for more than 2 1/2years.