The watchdog said on October 4 most appear to have been targeted because of their non-Slavic appearance.
The detentions come less than five months before Russia hosts the 2014 Winter Olympics in Sochi.
In a statement, Jane Buchanan, HRW's associate director of the Europe and Central Asia division, said it was "outrageous for the migrant workers who helped to build Sochi's shiny new Olympic venues to be herded into detention and deported."
HRW says the raids began after Krasnodar Governor Aleksandr Tkachev on September 3 called for "raid brigades" to scour Sochi's streets to "clean them up."
The New York-based NGO said the raids continued during the International Olympic Committee's final inspection visit in late September.
It said many of those rounded up were released after several hours but that some have been held for more than a week.
In some cases, they were denied access to a lawyer, and police denied they were being held in custody. Others have been expelled from the country, after perfunctory court hearings, without lawyers present.
Yulia Gorbunova, an HRW researcher, told RFE/RL the rights group has also recorded exploitation of workers at Sochi construction sites.
"Many of those migrant workers are actually owed wages from their employers," Gorbunova said. "Many of them have been working on the Olympic construction sites and have not been paid in full or at all by their employers, and some of them, as we have been told, are not even able to purchase tickets to go back to their countries."
HRW called on the International Olympic Committee to send a "clear message" that such actions are "completely unacceptable" for an Olympic host city.
During the final inspection visit last week, the IOC said preparations for the February 7-23 Sochi games were "really magnificent."