The headline index fell 341.72 points to 15,407.94 while the Topix index of all first-section shares was down 1.71 percent, or 21.55 points, to 1,240.99.
The dollar bought 102.56 yen in afternoon forex trade, little changed from 102.48 yen in New York Tuesday but well down from 103.25 yen in Asia.
A stronger yen is negative for Japanese exporters as it erodes their competitiveness abroad and shrinks repatriated profits.
Tokyo's fall accelerated as investors looked for opportunities to adjust their positions on the back of recent gains, but the outlook for Japanese shares remains upbeat, Takuya Takahashi, senior strategist at Daiwa Securities, said.
"Stocks had surged in the past weeks, and today is an excellent day to lock in profits," he added.
Investors were keeping a close eye on a string of US data that could provide clues about when the Federal Reserve will reduce its stimulus programme.
The ADP report on private-sector job creation, the Institute for Supply Management (ISM) services sector data and the Federal Reserve's Beige Book report on economic conditions are all due later Wednesday.
Upbeat figures would likely ramp up speculation that the Federal Reserve will start pulling back on its monetary easing scheme after its mid-December policy meeting.
"Investors are now worried that stronger-than-expected US jobs data could lead to a December start of scaling back of Federal Reserve's monetary easing," said Kenichi Hirano, operating officer at Tachibana Securities.
The stimulus has been credited with buoying global equity markets.
Shares in Seven & I holdings fell 0.40 percent to 3,750 yen after the 7-Eleven convenience store operator said it was buying 49.9 percent of luxury retailer Barneys New York's business in Japan for 6.0 billion yen.
Sony fell 3.40 percent to 1,845 yen. The leading Nikkei business daily said Wednesday the company was looking to buy a Japanese chip fabrication plant from struggling Renesas Electronics for 10 billion yen.
Wall Street fell for a second day in a row Tuesday on speculation over the Fed's stimulus. The Dow fell 0.59 percent. (AFP)