The group, Human Rights Watch, and eight Uzbeks last year filed the complaint against Zokirjon Almatov with Germany's federal prosecutor.
Almatov was thought to be in Germany for medical treatment at the time the complaint was filed.
The prosecutor, Kay Nehm, says he chose not to launch the probe because he did not expect cooperation from Uzbekistan and the case would likely fail as a result.
In a statement, HRW says it will challenge the decision. It also criticized Germany for allowing Almatov to enter the country when the German government was "clearly aware" of his probable role in the Andijon crackdown.
German law allows for the prosecution of crimes against humanity regardless of where the alleged crimes were committed, and does not require that the accused or the victims be in Germany.