The Dow Jones Industrial Average surged nearly 150 points, rallying above its all-time closing high of 15,409.39. All 30 components traded in positive territory, boosted by Caterpillar and Disney.
The blue-chip index's point gain for 2013 is already greater than any year on record.
The S&P 500 was higher for a sixth-consecutive session, briefly crossing above its record closing level of 1,669.16. The S&P 500 is now on pace for its biggest weekly gain since January.
The Nasdaq also jumped, while the Russell 2000 index hit a new all-time high for the fourth-straight session.
The CBOE Volatility Index (VIX), widely considered the best gauge of fear in the market, slid near 14.
All key S&P sectors were in positive territory, lifted by materials and industrials.
Robert Barbera, Johns Hopkins Center for Financial Economics, discusses the latest buzz from the Fed and how its policies are impacting the markets.
Bernanke said monetary policy would remain accommodative for the foreseeable future, even if the unemployment rate hit the Fed's target of 6.5 percent. Speaking at a conference after the market close Wednesday, Bernanke's remarks came after the release of minutes from the Fed's latest policy meeting, which showed that policymakers wanted further reassurances about the strength of the jobs market before pulling back on stimulus measures.
On the economic front, weekly jobless claims rose by 16,000 last week to a seasonally adjusted 360,000, according to the Labor Department, above expectations for a reading of 340,000. The four-week moving average of new claims increased by a more modest 6,000 to 351,750.
Meanwhile, import and export prices declined for the fourth-straight month in June, according to the Labor Department.
The Treasury will sell $13 billion of 30-year bonds with the results available shortly after 1pm ET.
Among earnings, Yum Brands edged higher after the parent company of KFC and Pizza Hut edged past earnings expectations. But sales were slightly shy of consensus and the company's sales in the important China market fell 20 percent. Still, the fast-food chain said it expects positive same-store sales in China by the fourth quarter.
Financial giants JPMorgan and Wells Fargo are slated to post quarterly results on Friday.
Analysts expect S&P 500 earnings to have increased 2.6 percent in the second quarter from a year ago, with revenue up 1.5 percent, according to the latest data from Thomson Reuters.
Chris McNickle, global head of institutional business at Fidelity Worldwide Investment, discusses Fed tapering and says U.S. equities remain a "better place to invest in" than most.
Microsoft edged higher after the software giant announced a long-awaited restructuring. As part of the reorganization, Microsoft announced the president of its Microsoft Office division, Kurt DelBene, will be retiring.
In other news, gasoline prices are forecast to jump between 10 and 20 cents per gallon within the next few days, driven by rising oil prices and peak driving season. Oil prices have risen in recent weeks on geopolitical concerns, along with declining inventories.
In Asia, the Shanghai Composite rose above the key 2,000 level for a second straight session on Thursday, on hopes that Wednesday's dismal trade data will lead the Chinese central bank to ease monetary policy in an effort to boost growth.
Meanwhile, the yen strengthened further against the dollar on Thursday after the Bank of Japan held its monetary policy steady, but upgraded the country's economic outlook.
—By CNBC's JeeYeon Park (Follow JeeYeon on Twitter: @JeeYeonParkCNBC)