Japan's Maritime Self-Defense Force detected a foreign submarine navigating in the contiguous zone, just outside the country's territorial waters east of Amami-Oshima Island in Kagoshima Prefecture on Friday.
The SDF's destroyer and patrol aircraft followed the submarine, which was underwater and heading northwest.
The submarine had left the contiguous zone without entering Japan's territorial waters by Sunday morning, and was seen navigating west in the East China Sea.
Defense Ministry officials say the submarine is believed to belong to the Chinese navy, as its destroyer was seen navigating near the sub.
It's the first time a foreign submarine has been detected in the contiguous zone without ascending since June last year, when what is believed to be a Chinese submarine was navigating near the same island.
Under international law, submarines navigating within the territorial waters of another country have to ascend and display their national flags, but there are no such rules for contiguous zones.
The Defense Ministry is analyzing the objective of the submarine's navigation.
Defense Minister Kishi Nobuo, who is currently visiting Vietnam, has instructed officials to collect information and fully implement surveillance with a sense of urgency.