As an administrator of a penal institution, what rights would you grant to an inmate with regard to his or her religious preference?

The question largely depends on the jurisdiction in which the penal institution is located. In the United States, prison wardens do not have the authority to grant or withhold a prisoners' rights, as rights are—by definition—inalienable. While persons under a judicial sanction may forfeit rights, such forfeiture is defined in constitutional and statute law and not by executive fiat.
However, a warden can grant or withhold privileges or determine the method by which rights can be safely exercised in the prison environment. And, the warden generally has the authority to impose a "neutral" rule—one that is not specific to any particular religion—and apply it throughout the prison even if, to do so might interfere with the ability of specific prisoners to practice a specific religion (see Oregon v. Smith). For example, a warden can require invasive, bodily searches of prisoners for contraband even if the religious beliefs of some prisoners might be transgressed by such a search.
That said, prisoners are permitted to practice a "sincerely held religious belief." To be "sincerely held," a belief system has to be genuinely subscribed to by its adherent who practices and follows its precepts consistently. For instance, in Vaughn v. Garrison, the Supreme Court determined that a prison could deny a prisoner's request for accessories typically associated with the practice of Islam—such as a prayer rug—if they did not first request a pork-free diet.