Fradkov is in Tokyo on a two-day visit that will include talks with his Japanese counterpart Shinzo Abe.
Abe earlier this month renewed Tokyo's demand that Russia return islands -- known to Russia as the Southern Kuriles and to Japan as the Northern Territories -- to Japan.
Fradkov told reporters that Russia is prepared to discuss the issue when overall relations with Japan improve.
"We should work to improve the quality of our relations [with Japan] and when that goal has been achieved -- and all the conditions are there, although we should work hard to do it -- then this [territorial] issue will be considered. It is too early to speak about it now," Fradkov said.
The Soviet Union seized the islands at the end of World War II.
Fradkov met today with former Prime Minister Yoshiro Mori, House of Representatives Speaker Yohei Kono, and other officials. He is scheduled to meet Abe on February 28.