What effect did the veil have when the congregation first saw it?

In the short story "A Minister's Black Veil," some of the words that author Nathaniel Hawthorne uses to describe the reaction of the people include:
astonishment
wonder
wonder-struck
amazement
It is no surprise that these reactions would take place. The man behind the black veil is not known for being an eccentric, or even a complex person at all.

Mr. Hooper had the reputation of a good preacher, but not an energetic one: he strove to win his people heavenward by mild, persuasive influences rather than to drive them thither by the thunders of the word.

Imagine a man of this temperament suddenly showing up with a thick piece of black cloth covering his face, for no rational reason. This would invariably stir some commotion among his followers.
Like the story describes, the people turned their heads toward the door as he arrived. He made an effect on the children, who "clambered upon the seats" causing "a terrible racket."
As far as the men and women of the congregation, the story describes a "rustling of the women's gowns and shuffling of the men's feet." However, none of these noises seemed to distract or perturb Mr. Hooper.
Still, people started making conjectures as to the meaning of the veil. It was the "not knowing" and "not understanding" that led many to feel fear.

Such was the effect of this simple piece of crape that more than one woman of delicate nerves was forced to leave the meeting-house. Yet perhaps the pale-faced congregation was almost as fearful a sight to the minister as his black veil to them.

Therefore, it is arguable to say that the parishioners, and the meaning that they each gave to the veil, definitely caused fear and concern.

When the people first see Mr. Hooper wearing the strange veil, some doubt that it is even him. One man asks the sexton if he is sure that the figure is actually their own parson. The sexton assures him that it must be, but he later admits that he has a hard time believing that it is their "good" minister's face behind the veil. One woman declares that he "'has changed himself into something awful'" simply by hiding his face, and one man even believes that Mr. Hooper "'has gone mad.'" Even those who do voice such notable opinions are utterly "wonder-struck" and unable to respond even to Mr. Hooper's greetings. Then, rather than meeting with a "hushed repose" in the meetinghouse, Mr. Hooper is met with a "general bustle" and "perturbation." Rumors seem to precede him, setting all of the parishioners "astir." They all turn to look at him, and some even stand up to gawk.