It's not talked about a lot, but the makeup and history of the South following the Civil War, the Reconstruction era, is largely skipped over for a lot of people, but it didn't go very well for a lot of people. In fact, it was a disaster. Almost every southern state terrorized free blacks, and the Klu Klux Klan had an outsized influence on the politics and were known to muddle in the election processes with other "Redeemer" Democrat paramilitary organizations.
After the insanely close election in 1876, an informal deal (The Compromise of 1877) was reached, in which Northern Republicans essentially ceded their last political holdouts to the Southern Dems in exchange for the Presidency. Left in charge, the Redeemers, composed largely of the former white overseer wealthy class, took control of nearly all the former Confederacy and immediately began passing legislation to disenfranchise blacks and terrorize the ones who still held political office. By the 1900s, voting had been gutted in the South and discrimination was policy thanks to Jim Crow. It was then that the commission of statues of the former Confederate leaders began to spring up.
Again, not a lot of people talk about the overt terror tactics that took during Reconstruction, particularly against the poor whites and blacks, and its failures are often put upon greedy Northerners. As always, the truth is a little muddled, but the fact remains that there was a hard political movement meant to regain power and disenfranchise the 4 million freed slaves after the Civil War, and only after they achieved total power were those statues created by the state legislatures, who, again, had kept it from being a public issue thanks to mass voter disenfranchisement.
Aside from Blacks' obvious objections to it, even most poor whites would have agreed that traitorous soldiers against the United States would not be appropriate iconography in a still-United States. But they didn't really get a say.
