Federation Council members rejected the agreement -- which would have regulated the separation of powers between Tatarstan and the federal center -- by a vote of 93-13.
In opposing the agreement, Federation Council Speaker Sergei Mironov said it could set a precedent for other regions.
"The approval of this bill and this agreement would create a dangerous political precedent in violation of the constitutional principle of equality of Russian Federation constituents," Mironov said. "In effect, this would be a departure from the course of federal reform that has been pursued by Russian President Vladimir Putin for the past seven years."
Putin and Tatarstan President Mintimer Shaimiyev originally signed the agreement in October 2005.
It passed the State Duma, the lower house of Russia's parliament, this month.