Chechens were able to salvage some of these gravestones after returning from their forced exile to Central Asia and Siberia in the late 1950s.
Almost four decades later, Chechnya's first post-Soviet leader, Dzhokhar Dudayev, ordered the gravestones to be gathered and placed in neat lines in a leafy square in Grozny's city center.
But this week, the memorial commemorating the Great Terror and the deportation of Chechens vanished from its spot after workers turned up without warning and began pulling out the gravestones.
Dzhalal Kadiyev, the head of Chechnya's architecture committee and Grozny's chief architect, says the gravestones will be moved to a more convenient location outside the city center.
"There was very little space in the old spot; the new location has enough space for the construction of a rest area and a parking lot," he says. "It will be a very beautiful place, close to university and the Kirov road. This is a big territory that can host both this memorial and a rest area. It will more comfortable there for people in every sense, to reflect, to relax."
Who ordered the surprise removal, however, remains a mystery. Kadiyev himself admits he is in the dark.
Some suspect developers keen to get their hands on a prime plot of land in the heart of the capital.
Others believe Chechnya's Kremlin-anointed president, Ramzan Kadyrov, decided to move the memorial out of consideration for visiting Russian officials, who must drive past the tombstones on their way from Grozny's airport.
Kremlin envoys are not likely to have appreciated the monument standing amid the graves -- a stone first clenching a sword, engraved with the words "We won't fall apart, we won't cry, and we won't forget."
Whatever the motives, Grozny residents are furious.
"It's difficult for me now to talk, to express my opinion about it," says Shaaman Akbulatov, from the Memorial human rights group in Grozny "This place is in the city center, it's close to the mosque, visitors drive past it. It didn't cause inconvenience to anyone. Why does it have to be moved to the city's outskirts? Who will see it now? By moving the memorial out of sight, I think they want to erase it from people's memory."
The 1944 deportation is one of the most painful chapters of Chechnya's history.
In February of that year, nearly half a million Chechens and Ingush were rounded up and sent to Kazakhstan, Kyrgyzstan, and Siberia on freight trains. About half the deportees are estimated to have died either during the journey or within the year that followed their deportation, succumbing to cold, hunger, and disease.
It was not until 1956, after the death Josef Stalin -- who had accused the two peoples of collaborating with the Nazi German Army -- that Chechens were permitted to return to their homeland.
Zulay Bagalova, a well-known Chechen actress, says the original memorial to the victims could easily have been expanded and improved. Its removal from central Grozny, she says, is just one more affront to the long-suffering Chechen people.
"I went to see it yesterday. There isn't a single whole gravestone left, just a few stumps," she says. "If you decide to move gravestones from one place to another, the least you can do is treat them with care. They were already removed and desecrated when we were stripped of our homeland and deported. They were desecrated to show us that we weren't regarded as human beings. And now this is being intensified."