What is the origin of Roderick's gloom in "The Fall of the House of Usher"?

I think that "The Fall of the House of Usher" is one of those stories which is open to some degree of interpretation as to the nature of Roderick's affliction and what caused it. My own reading of it runs something as follows.
The first thing that stands out to me is the house itself: the house is described as oppressive, and I think that living under these kinds of conditions (both from the oppressive gloominess of the house as well as from the isolation) would exact a heavy toll.
Exacerbating an already miserable situation is the mysterious illness of his sister, Madeline, who is described within the text as "his sole companion for these long years—his last and only relative on earth." The progression of her illness has had a powerful impact on Roderick's own mental health.
I think it's in the combination of these two factors (the oppressive isolation of life within the house, along with the experience of his sister's illness) that is responsible for Roderick's own deterioration.