Why did Vladimir Nabokov give his protagonist such an unusual name?

It may have been because the author was searching for a name for his character which nobody else in the world could have.
Humbert Humbert was guilty of such terrible multiple crimes that any man who happened to share his name would be stigmatized by association. Humbert is not only guilty of statutory rape, but he is guilty of violating the Mann Act for what must be a record number of times. The Mann Act made it a federal crime to transport "any woman or girl [across state lines] for the purpose of prostitution or debauchery, or for any other immoral purpose."
Humbert blithely takes Lolita into most of the states in the U.S., and the purpose is certainly debauchery. And finally, Humbert Humbert is guilty of murdering Clare Quilty. The original penalty for violation of the Mann Act was not more than five years in prison, so Humbert Humbert was eligible for perhaps 200 years in prison, plus numerous statutory rape convictions in all the states involved in the crazy journey, and the murder charge. Fortunately for Humbert, he dies of heart failure in prison.