The Federation Council voted by 140 votes to zero to accept his request after the issue was unexpectedly put on the agenda of the day.
Ustinov had held the post since 2000, after being nominated by President Vladimir Putin. However, his departure follows criticism from Putin that Russia is failing to combat corruption.
Ustinov led the legal case against the oil giant Yukos and its former owner, Mikhail Khodorkovsky -- critic of Vladimir Putin and once the richest man in Russia.
Inevitably, though, speculation in Moscow is rife as to why Ustinov stepped down. Rumors had long been circulating that his days were numbered but Mironov maintained that there is no subtext -- Ustinov had simply had enough.
"The president called me to his office yesterday and handed me a note on the dismissal of Vladimir Vasilyevich Ustinov. From our conversation I understood that it was a technical decision, if one could say so. There is no politics here," Federation Council Chairman Sergei Mironov said.
But in Moscow today not many seem to be buying that. The politologists and other Kremlin watchers are linking his departure to President Putin's state-of-the-nation address last month in which he singled out the failure of the law enforcement agencies to root out corruption.
Putin criticized the work of law enforcement agencies during his state-of-the-nation address in May
"The situation here regarding the oversight of the law enforcement agencies and with crime, for which the prosecutor-general is responsible, leaves much to be desired. And under Ustinov things have most certainly not changed for the better," said Vladimir Pribylovsky, the director of the Moscow-based Panorama think tank.
But while Pribylovsky concedes that Ustinov's failings in the struggle against crime and corruption may have been a factor in his apparent fall from grace, he believes the real reason lies in the endless struggle for influence within the Kremlin.
"It would appear the balance of power within the coalition of groups around Putin has changed -- the so-called Putin team. Something must have tipped the balance and the president finally decided to tell Ustinov he had to go. There are a lot of people after that job," Pribylovsky said.
In Ustinov's absence, the position of prosecutor-general will be filled temporarily by his deputy, Yury Biryukov, but the battle for the succession is already under way.
Top contenders for one of the most powerful jobs in Russia are Dmitry Kozak, the president's special envoy to the Southern Federal District, and Aleksandr Konovalov, the president's special envoy to the Volga region.
Vladimir Pribylovsky (RFE/RL)
RFE/RL's Russian Service asked Moscow-based political analyst VLADIMIR PRIBYLOVSKY
to comment on the June 2 dismissal of Prosecutor-General Vladimir Ustinov. Pribylovsky is the director of the Panorama think tank.
RFE/RL: Were you surprised by Ustinov's removal?
Vladimir Pribylovsky: We have the kind of prosecutor-general about whom it should be asked why he wasn't dismissed earlier. Generally, I think this dismissal is connected with the clan struggle and signifies, most likely, a tactical or strategic defeat for one of the St. Petersburg groups, specifically the one associated with [deputy presidential administration head] Igor Sechin, who is the prosecutor-general's [relative]. [Sehchin's daughter] Inga Sechina is the wife of [Ustinov's son] Dmitry Ustinov.
RFE/RL: And how can this defeat be explained?
Pribylovsky: It would appear that the power within the coalition of groups around [President Vladimir] Putin has changed -- the so-called Putin's team. Some weight must have tipped the balance and the president finally decided that it was time to advise Ustinov to leave. In general, there are a lot of people seeking that post.
RFE/RL: Now there are reports that [presidential envoy to the Southern Federal District] Dmitry Kozak will be appointed prosecutor-general and Vladimir Ustinov, most likely, will be sent as ambassador to the European Union.
Pribylovsky: In theory that is possible. Naturally that would be an honorable exile for Ustinov. As far as Kozak is concerned -- he is one of the two most likely candidates for the post. The other is Aleksandr Konovalov, the presidential envoy to the Volga Federal District. His name came up fairly recently as a possible prosecutor-general -- came up in informal conversations, naturally. But Konovalov is a friend of Nikolai Vinnichenko, and Nikolai Vinnichenko is a friend and former classmate of [First Deputy Prime Minister] Dmitry Medvedev. Therefore, such an appointment is possible. Although the appointment of Kozak is also possible. Both of them are associated with the so-called group of Petersburg lawyers, although Kozak is more or less on the periphery of that grouping. It can't be said that he is Medvedev's vassal, as it can be of Konovalov or Vinnichenko.
RFE/RL: You said that it is surprising that Ustinov wasn't fired earlier. Why?
Pribylovsky: The situation here regarding the oversight of the law enforcement agencies, regarding crime, for which the prosecutor-general is responsible, leaves much to be desired. And Ustinov's tenure has not improved a thing.
SUBSCRIBE For news and analysis on Russia by e-mail, subscribe to "RFE/RL Russia Report."