What does Antonio think of Prospero's library in Shakespeare's The Tempest?

For Caliban, Prospero's library full of magic books is the source of his power. But for Antonio, Prospero's scheming brother, it's the exact opposite: it's a source of weakness. Prospero spends so much time in his library that he neglects his duties as Duke of Milan. This allows Antonio to get involved in a dastardly scheme to usurp Prospero and have him banished to a remote island.
It's somewhat ironic that Prospero's spending all that time with his magic books didn't ultimately save him from being sent into exile. Surely he could've rustled up a magic spell or two to send his greedy, scheming brother and his co-conspirators packing? In any case, Prospero will put the fruits of his learning to good effect later on when he'll use his magic powers to whip up a gigantic storm to bring Antonio and his partners in crime to his remote island home.