Despite economic stagnation and rising unemployment, pre-vote opinion polls put the pro-Western Tadic ahead of Nikolic among Serbia's more than 6 million registered voters.
One pollster, the Center for Free Elections and Democracy (CESID), suggested in one of the first projections to come out that Nikolic was leading the voting with 49.4 percent of the vote to Tadic's 47.4, according to RFE/RL's Balkan Service.
But those figures emerged early in the vote count and could well change.
The election appeared to be marked by a low turnout, with just 37 percent of eligible voters having cast ballots by late in the voting day, far below the 58 percent participation in the first round two weeks ago.
"Today is a crucial day when we will decide which direction we will take in the next five years and even the next decade," Tadic said after voting at a polling station in the heart of Belgrade's upmarket old town, where he arrived with his daughter and pushing his elderly father in a wheelchair. "And the world is watching to see our democratic maturity."
Tadic finished just ahead of Nikolic in the first round of voting on May 6. After that vote, Tadic secured the backing of the Socialist Party, the third-largest bloc in the Serbian parliament.
If elected, Tadic has promised to use his third term as president to start EU membership talks by the end of the year, with the aim of joining in five years.
Nikolic claims that vote was marred by fraud and has warned he will call supporters to the streets over alleged ballot rigging. But election authorities and foreign monitors found no evidence of the 500,000 votes Nikolic says were forged in the parliamentary and first-round presidential polls.
Nikolic's Serbian Progressive Party said its monitors will confiscate ballot boxes and close polling stations if they observe irregularities.
Nikolic cast his ballot in New Belgrade -- a socialist-era high-rise apartment district of the Serbia capital. Unlike his opponent, Nikolic has said he would pursue EU membership but not at any cost -- suggesting that he would maintain Serbia's claim on the former UN-administered province of Kosovo, which declared independence in 2008.
"I am convinced that I am the best choice for Serbia, but it is up to the citizens of Serbia now," Nikolic, who finished second to Tadic in presidential elections in 2004 and 2008. "And I will congratulate the winner."
Nikolic has accused Tadic of overseeing a creeping culture of cronyism, increasing the government's influence over the media, and fuelling an economic downturn that has seen unemployment reach 24 percent.
He alleged vote fraud in the first round of voting, but a week ago backed off his threat to boycott the runoff.
Tadic's Democratic Party and Nikolic's Progressive Party finished neck-and-neck in the parliamentary voting that accompanied the first round of the presidential poll, at 24 and 22.3 percent, respectively.
The European Union made Serbia, which has a population of more than 7 million, an official candidate for membership in March. The union has said that a date for talks could be set early next year if Belgrade takes steps to improve relations with Kosovo, which it does not recognize as an independent state.
Early figures from Serbia's roughly 8,500 polling stations were expected to trickle out within a few hours of the end of voting, and final preliminary results were likely to emerge on May 21 or 22.