Who knocked over the “colored women bathroom” sign and why?

Al Harrison, the director of the Space Task Group, knocked over the Colored Ladies Bathroom sign. He did it to end bathroom segregation at NASA.
Harrison asks Katherine why she's so often away from the group. After Katherine points out that she has to walk half a mile to the bathroom, he's forced to stop and think about the fact that every time she goes to the bathroom, it takes her an hour to walk to the colored toilets and back. Harrison has to reconsider his attitude toward her and the other people of color working at NASA.
Ultimately, he decides that the sign designating the colored restroom needs to be taken down. He walks the half mile to the bathroom, takes a crowbar and knocks it down himself, declaring that from now on, there will be only bathrooms. The toilets at NASA will no longer be segregated.
This scene was not included in the book; it was only in the film. In the book, the author explains that she used the white bathrooms and just never went into the colored bathrooms.