Is it true that Rutherford B. Hayes won by one vote?

It is indeed, yes. The hotly-disputed Presidential election of 1876 between the Republican Rutherford B. Hayes and his Democratic opponent, Samuel J. Tilden, initially ended in victory for Tilden, but because of widespread voter fraud and intimidation, results in four states—Louisiana, Oregon, Florida, and South Carolina—were re-examined by a specially-appointed Electoral Commission.
In order to avert a major constitutional crisis, and with Inauguration Day fast approaching, the Commission set to work quickly. In a relatively short period of time, it duly arrived at its decision. By an 8 to 7 majority, the Commission decided to award all twenty disputed Electoral College votes to Hayes, thus giving him a wafer-thin margin of victory: 185 to 184. Two days after the Commission adjourned, Hayes was sworn in as President.
As one can imagine, Hayes's Democratic opponents were far from pleased at the outcome. Not only had Tilden won a plurality of the popular vote—50.9% to 47.9%—he had (initially, at least) gained a majority in the Electoral College as well. The bitter pill was somewhat sugared, however, by a backstairs deal that promised the withdrawal of Federal troops from South Carolina and Florida, the last two still-occupied Southern states.