Japan calls the 4 islands the Northern Territories and maintains they are an inherent part of its territory. Russia says they became part of their country as a consequence of war.
In the final year of World War II, the former Soviet Union declared war on Japan and occupied the islands. The Japanese government says it was a violation of a neutrality pact.
Soviet leader Joseph Stalin then annexed the islands and deported 17,000 Japanese.
A decade later, Japan and Russia resumed diplomatic relations, but the islands remained a sticking point and leaders agreed to continue talks on signing a peace treaty.
In the years since, Moscow has been building infrastructure for the Russians who call the islands home. Currently, about 16,000 people live there.
And as the leaders of both countries changed, both sides continued their discussions over the dispute off and on, but neither has budged on its stance.
In 2013, Japanese Prime Minister Shinzo Abe and Russian President Vladimir Putin issued a joint statement saying "the two governments aim to finally resolve the issue."
Since then, they've held 11 summits.
"We agreed to accelerate our talks to draw up solutions acceptable for both sides," Abe said.
He proposed what he calls "a new approach" to Putin.
"If we continue on like this, this very same discussion will continue for yet more decades to come," Abe said. "Vladimir, shall our generation have the courage to fulfil our responsibilities?"
Putin welcomes stronger economic ties but he has been cautious about referring to the territorial issue.
"Both countries are hoping and making efforts to settle the issues, but I can't yet say when or how or even if we can achieve it at all," Putin said.
And he has said nothing new when it comes to the islands.
"As the result of the war, the islands are under Russia's sovereignty," Putin said.
Abe too knows the road ahead is not a smooth one.
"The path toward achieving the goal is now in sight, but the 'mountain' has to be climbed one step at a time," Abe said.
The big question now is whether these two leaders can achieve something their predecessors couldn't.