An experienced Russia reporter, he was working in the city of Yekaterinburg at the time of his detention.
The White House has condemned his detention "in the strongest terms".
The Kremlin claimed he had been caught "red-handed" but the Wall Street Journal vehemently denied the allegations against him.
Mr Gershkovich, 31, is well known among foreign correspondents in Moscow and BBC Russia Editor Steve Rosenberg describes him as an excellent reporter and a highly principled journalist.
US Secretary of State Antony Blinken echoed the Wall Street Journal in saying he was "deeply concerned" by the arrest. US officials said they had immediately sought access to Mr Gershkovich but had not had any response.
The WSJ said its reporter had dropped out of contact with his editors while working in Yekaterinburg, about 1,600km (1,000 miles) east of Moscow, on Wednesday afternoon.
US officials said Mr Gershkovich's driver had dropped him off at a restaurant and two hours later his phone had been turned off. The newspaper said it had hired a lawyer to try to find him at the FSB headquarters in the city, but they told the lawyer they had no information.
Russia's FSB security service claimed that it had halted "illegal activities". The journalist had been detained "acting on US instructions", it added, alleging that he had "collected information classified as a state secret about the activities of a Russian defence enterprise".
It said its investigation department had launched a criminal espionage case and one source told Russian media it was classed as "top secret".
FSB agents took him to Lefortovo district court in Moscow on Friday, where he was formally arrested and ordered to remain in detention until 29 May.
Russian media said the court had already been cleared of staff and visitors and Mr Gershkovich's lawyer said he had not been allowed into the courtroom.
Espionage in Russia carries a maximum jail term of 20 years. Tass news agency reported that the journalist had denied the charge. He was then seen being escorted from the building before being driven away.
In his most recent WSJ piece, published this week, Evan Gershkovich reported on Russia's declining economy and how the Kremlin was having to deal with "ballooning military expenditures" while maintaining social spending.
Press freedom watchdog Reporters Without Borders said he had gone to Yekaterinburg to cover Russian mercenary group Wagner, which has taken part in some of the heaviest fighting in eastern Ukraine.
He has covered Russia for the Wall Street Journal for more than a year, having worked there previously for the AFP news agency and the Moscow Times. He began his career in the US.
In a statement, the Wall Street Journal said it stood in solidarity with the reporter and his family: "The Wall Street Journal vehemently denies the allegations from the FSB and seeks the immediate release of our trusted and dedicated reporter, Evan Gershkovich."