U.S. stocks retreated on Thursday, led by financial stocks, while Caterpillar shares dropped following news that federal officials searched its Illinois facilities.

Caterpillar (CAT.N), down 4.3 percent at $94.36, was the biggest drag on the Dow and among the biggest negatives for the S&P 500. In a statement issued after the closing bell, Caterpillar said the search may be related to an Internal Revenue Service investigation on profits earned by a Swiss subsidiary.

Financials led the decline among sectors in the S&P 500, which had its biggest daily percentage decline since Jan. 30. The S&P financial index .SPSY fell 1.5 percent in its biggest daily drop since mid-January.

Bank stocks had surged on Wednesday on increased expectations that the Federal Reserve will hike interest rates this month. The S&P 500 and the Nasdaq had their best day since the November election after U.S. President Donald Trump's measured tone in his first speech to Congress lifted optimism.

"It was an awfully strong rally yesterday without necessarily a lot of real news to justify it, so I think you're just getting some profit taking today," said Rick Meckler, president of investment firm LibertyView Capital Management in Jersey City, New Jersey.

He said investors also appeared to be rotating into sectors that have not done as well in the post-election rally, including S&P utilities .SPLRCU, up 0.7 percent.

Meckler said that since the election, though, any selling has soon been met by greater buying. "It's going to take a more material selloff to break people's view that they don't have much to lose by buying stocks."

Shares of Snap Inc (SNAP.N), the parent company of messaging app Snapchat, rose 44 percent to $24.48 on their first day of trading. The stock opened at $24, above the IPO price of $17, and nearly 217 million shares traded on the debut.

The Dow Jones Industrial Average .DJI fell 112.58 points, or 0.53 percent, to end at 21,002.97, the S&P 500 .SPX lost 14.04 points, or 0.59 percent, to 2,381.92 and the Nasdaq Composite .IXIC dropped 42.81 points, or 0.73 percent, to 5,861.22.

The S&P 500 is up 11.3 percent since the Nov. 8 election.

Several Fed officials this week have stoked expectations of an interest rate hike this month as the economy strengthens.

Fed Chair Janet Yellen is set to speak on Friday and could provide the strongest indication about a move in coming weeks. The Fed's next policy-setting meeting is set for March 14-15.

Traders have priced in about a 74-percent chance of a rate hike this month, up from roughly 30 percent at the start of the week, according to Thomson Reuters data.

Kroger (KR.N) fell 4.3 percent after reporting a surprise decline in fourth-quarter same-store sales as competition intensified in the U.S. grocery industry.

Declining issues outnumbered advancing ones on the NYSE by a 2.85-to-1 ratio; on Nasdaq, a 2.23-to-1 ratio favored decliners.

The S&P 500 posted 40 new 52-week highs and three new lows; the Nasdaq Composite recorded 126 new highs and 39 new lows.

About 7.4 billion shares changed hands on U.S. exchanges, compared with the 6.9 billion daily average for the past 20 trading days, according to Thomson Reuters data.