Russian news agency Interfax reported that Georgian troops had entered the besieged town of Tskhinvali after intense battles overnight.
But Georgia denied the Interfax report.
Interior Ministry spokesman Shota Utiashvili said Georgian forces had not yet entered Tskhinvali, but were engaged in a battle with two convoys, carrying "mercenaries", which had entered South Ossetia from Russia and were trying to reach the town.
"We want to give time to the remaining civilians to leave [the town]," Utiashvili said. Asked if Georgian forces planned to enter the capital, he said, "If the need arises."
Georgian big guns shelled Tskhinvali, where government and separatists envoys had been due to meet for Russia-mediated peace talks later on August 8. Many houses were ablaze.
Russian peacekeepers said three of their men were wounded and their headquarters damaged during shelling of the town, Interfax reported.
In Tbilisi, Georgian Prime Minister Lado Gurgenidze said the military operation would continue until a "durable peace" had been reached.
Russia, main backer of the separatists who have controlled the region since a war in the early 1990s, accused Georgia of treachery and urged the world community to avert "massive bloodshed."
The Kremlin said Russian President Dmitry Medvedev was holding consultations with aides on Moscow's strategy, "aimed at restoring peace in South Ossetia, defending the local civilian population within the peacekeeping mandate we have."
Russian news agencies said he later summoned his top security advisers to discuss the crisis.
At an emergency session of the United Nations, Russia failed to push through a statement that would have called on the two sides to immediately halt all bloodshed.
Council diplomats said a phrase calling on all sides to "renounce the use of force" had been unacceptable to the Georgians, backed by the United States and the Europeans.
The crisis has fuelled fears of full-blown war in the wider Caucasus region, which is emerging as a vital energy transit route and where Russia and the West are vying for influence.
Georgia said the operation, launched after a week of clashes between separatists and the troops in which nearly 20 people were killed, was aimed at ending South Ossetia's effective independence won in a 1991-92 war.
"We are forced to restore constitutional order in the whole region," the commander of Georgian peacekeepers in South Ossetia, Mamuka Kurashvili, told Georgian television.
In Tskhinvali, thousands of people took refuge from the shelling in makeshift shelters in the cellars of their homes. Russian television showed pictures of houses on fire.
"It is scary. We don't know whether Georgians are advancing or what," one woman said, nervously trying to fix a candle in the dark vault. "We even did not have time to take radios."
A Vesti-24 correspondent in Tskhinvali, Andrei Chistyakov, said at least 15 civilians had been killed in the town. "These are the people whose bodies were seen in their yards and in the streets," he said by telephone.
Georgian President Mikheil Saakashvili, who wants to take his small Caucasus nation into NATO, has set a priority on winning back control over South Ossetia and Abkhazia, another breakaway region on the Black Sea.
The issue has bedevilled Georgia's relations with Russia, which is angered by Tbilisi's moves towards the Western fold and its pursuit of NATO membership.
On August 7, Saakashvili announced a unilateral truce and gave a go-ahead for peace talks in Tskhinvali the next day. But just a few hours later, Tbilisi accused separatists of shelling Georgian-populated villages and set troops in motion.
A Reuters correspondent said the roar of warplanes and explosions of heavy shells was deafening more than three kilometers away from the town.
Georgian National Security Council Secretary Kakha Lomaia told Reuters that several Soviet-designed Su-25 planes took part in a strike on the village of Tkverneti.
"These planes can be used again if the needs comes to hit foreign mercenaries arriving in the region," he said. "But they will not be used in Tskhinvali."
The Russian Foreign Ministry appealed for international efforts to prevent "massive bloodshed."
Factbox -- South Ossetia
Status: The region broke away from Georgia in a 1991-92 war. A peacekeeping force with 500 peacekeepers each from Russia, Georgia, and North Ossetia monitors a 1992 truce.
Population: Approximately 70,000 (according to the 1989 census, about two-thirds Ossetian, one-third Georgian)
Languages: Ossetian, Georgian, Russian
Religion: Orthodox Christianity
South Ossetia: Timeline Of A Crisis