What factors were behind the “scramble for Africa,” and what impact did it have on the continent? How did the subject peoples respond to colonialism, and what role did nationalism and Social Darwinism played in their response?

You've asked a few questions here, so I'll focus on your first question: the factors behind the scramble for Africa and its effects on the African continent.
European nations had been colonizing other lands for centuries, of course. By the 1880s, however though, there was a new push for imperial claims. This came to a head in the Berlin Conference in 1884, when the leading European nations carved up the African continent into easily digestible colonial holdings, or "spheres of influence." The factors that influenced their desire to claim these colonial holdings can be divided into four categories: political, economic, industrial, and cultural.
Politically, rising nationalism was spurring on competition between European nation-states, all of whom were eager to demonstrate their power by claiming new territories. Increasing conflict between these nations also prompted a surge of interest in expanding the armed forces and securing military bases. Having military outposts in colonial nations and securing access to those outposts for troops and navy ships, was particularly important.
Economically, Europe was booming thanks to industrialization. In order to continue to produce at high levels, though, factories needed access to raw materials—which could be acquired more cheaply in colonial nations than on the European mainland. The other side of the coin, of course, was that the markets needed to expand if they were to keep producing. Selling goods to people in these colonial holdings was a good method for keeping business booming.
The boom in industry also made colonization safer and more profitable, thanks to new tools and innovations. In the nineteenth century, the invention of quinine allowed European explorers to survive tropical diseases. Advanced weapons, like machine guns, also helped Europeans convince indigenous peoples to follow their directives. The development of steamboats, telegraphs, and railroads increased the speed with which people could move themselves and information between colonies and continental Europe.
There were also cultural influences on the scramble for Africa, often couched in terms of "morality." Religious groups had long entered foreign nations in the hopes of converting the local people to their faith; by the 1880s, even non-religious groups argued in the "white man's burden" (as Rudyard Kipling called it) to "civilize" the "savages." Social Darwinism offered a semi-scientific rationale for embarking on these colonial missions—and provided imperialists a "humanitarian" rationale for their actions.
The result of imperialism on the people of Africa differed from colony to colony. On the whole, the African people were expected to assimilate to their European colonizers. The Europeans, based on their views about Africans' inferiority, believed that the African people should adopt European languages, culture, and customs. Often, this assimilation was forced through the education system. In other words, the schools set up by European colonizers was intended to indoctrinate, not to teach.
Many Africans did learn to speak the language of the colonizers, as this allowed them some control over their economic mobility. Many began to enter into leadership positions in European systems of power. While this benefited some Africans financially and socially, it not only helped lead to the disintegration of indigenous culture, but it fomented conflicts between African groups, which would often lead to conflict in the long run.
Resistance to colonization was nearly impossible: Africa was not only incredibly large and diverse, but it was also colonized by a number of different European nations, all of whom used different tactics and desired different outcomes. Part of the problem was how effectively European colonizers used the long-time conflicts between African communities to their advantage. Africa was incredibly diverse, with no common faith or culture across the continent as a whole, or even across newly divided "nations." That meant that in battles for tribal sovereignty, African communities were often fighting both European and rival African soldiers at the same time.
Ultimately, when the European colonizers finally withdrew from Africa over the course of the next century, they left no system in place to help the African people transition into power and self-governance. Often, this caused a jockeying for power between African tribal groups that led to civil wars—the damage of which we can still see in the continent today.