Arzumanian was released on September 6 after spending four months in prison on charges of money laundering and illegally receiving finances from abroad.
"Until Friday, I was a political prisoner; now I am a politically persecuted person," Arzumanian said. "That ludicrous, fabricated charge has not been dropped, the political order is still there, and nobody has apologized for the four months I spent" in jail.
Armenian authorities have accused Arzumanian of illegally receiving large sums of money from a fugitive Russian businessman of Armenian descent.
Armenian law-enforcement authorities say the charges have not been dropped, and that Arzumanian may still stand trial.
Arzumanian's release came as a surprise to many, as one week earlier a Yerevan court had agreed to extend his arrest for another two months.
But Prosecutor-General Aghvan Hovsepian said he personally ordered the release, as he did not think there were sufficient reasons for keeping him in jail.
Arzumanian served as Armenia's foreign minister between 1996-1998, and is seen as one of the most loyal associates of former President Levon Ter-Petrosyan.
Armenian Prime Minister Serzh Sarkisian, speaking to RFE/RL's Armenian Service today at the opening of the fall parliamentary session, said he was neither happy nor unhappy about Arzumanian's release, and that he did not know who had made the decision.