In what way is this story humorous?

The absurdity of the quirks the narrator's family members possess help to create the humor in Thurber's story. The nightly rituals his two aunts and cousin perform to guard themselves against burglars and sudden death are amusing rather than sympathy-inducing.
The grandfather's disappearances and reappearances combined with his news and opinions on the long-past Civil War are likewise, played for laughs instead of pathos.
Another aspect of the story's humor is the comedy of errors that ensues after the narrator's cot tips over in the night. His mother's panic, the yelling, his cousin nearly drowning himself in camphor, Rex's barking, and the misunderstandings that grip the family all create a tone that is chaotic and funny.
There is an amusing irony that the mother's irrational fear that the bed in the attic will fall on her husband is not what creates the disturbance in the night; instead, it is the narrator's cot that collapses.