Why does the narrator refer to the bust of Hippocrates?

The bust of Hippocrates helps set the scene in Dr. Heidegger's study and establish the nature of Dr. Heidegger.
When Nathaniel Hawthorne describes the study, he mentions the bust of Hippocrates and says that Dr. Heidegger has a tendency to consult with it when he's facing difficulties. Hippocrates, the father of medicine, is someone whom a doctor would naturally admire and look up to.
The other time the bust is mentioned, it's in reference to a book of magic in the study. When a chambermaid lifts it, the bronze bust itself frowns and tells her to "Forbear." This might be foreshadowing for Dr. Heidegger himself, who forbears in the matter of drinking from the potion of rejuvenation—remaining the doctor and not becoming the subject of his own study. It also helps establish the book as depicting actual magic, which helps create the required setting for the events of the story.

In the story, the bust of Hippocrates serves two different functions. The first pertains generally to the doctor's profession. The second relates to the story's fantasy or Gothic genre.
Hippocrates (ca. 460 - 375 B.C.E.) was a Greek physician who wrote and/or compiled writings on medicine. Often called the "father of modern medicine," he is still well known today for the Hippocratic Oath (although probably not really its author) in which doctors pledge to do no harm.
A bust of Hippocrates would have been an entirely typical accessory for a 19th-century physician to have in his study. Hawthorne uses it ironically because Dr. Heidegger is ignoring his duty to do no harm through his reckless behavior.
In addition, the fantastic element is emphasized through this bust. When the maid moves the magic books, they affect several items. The bust smiles into the mirror—very out of keeping with the gravity associated with the person and the oath.
https://www.britannica.com/biography/Hippocrates