Does Eliza improve her self-confidence in Pygmalion?

Yes, Eliza does markedly improve her self-confidence. This comes about as she learns the speech and manners of a lady. Her confidence increases as she gains the acceptance and approval of upper-middle-class people, the love of Freddy, and finally, the acceptance of the nobility, who embrace her as one of them on the basis of her accent, clothing, and demeanor.
We know she has gained self-confidence because by the end of the play, she is standing up to the bullying Professor Higgins and asserting her worth as a human being. She tells him in an impassioned speech that it is wrong to make her a lady, unfit for her old life, and then falsely threaten to throw her back on the street:

Oh, you are a cruel tyrant. I can't talk to you: you turn everything against me: I'm always in the wrong. . . . you're nothing but a bully. You know I can't go back to the gutter, as you call it, and that I have no real friends in the world but you and the Colonel.

In other words, she confronts Higgins, tells him what he is, and says he treats her cruelly only because he knows she has nowhere else to turn. She informs him if he can't treat her kindly that she will pursue "independence." All of this is a far cry from the young woman who was willing to put up with his insults earlier in the play.