Optimization of consumer behavior through endless A/B testing optimizes behavior in favor of the corporations, not the society or the individual. So one of my bigger worries is that these optimization strategies will lead society into a dead end local maximum where everyone is perfectly conditioned to consume for the good of the corporation, when we could have taken a different path resulting in a more egalitarian outcome.
Another worry is that through subtle manipulation of search result ordering, page load delays, news feed filtering, etc., Facebook, Google, et al. could influence politics and culture in counterproductive ways, even if only by algorithmic accident.
This is discussion that has happened in economics for a long time. I am still utterly inconvinced of the pure power of advertsing. Sure it has a big influence, but its not like google is so good that everybody is just a mindless zombi perfectly conditioned to buy whatever they show me. Even if that were the case, Im not clear why this would lead to less egalitarian outcome.
Your second point is in conflict with the first. If they want me to consume as much as possible, they have to show me what I want. If they want to influence broad culture they have to show me what they want me to see. If it is by algorithmic accident, then its like all cultural devlopment. Its all accidents, nobody planes the evolution of culuture, google, facebook and reddit have already changed culture and will continue to do so.
You points are a small worry, compared to something like the prospect of 1984.
Another worry is that through subtle manipulation of search result ordering, page load delays, news feed filtering, etc., Facebook, Google, et al. could influence politics and culture in counterproductive ways, even if only by algorithmic accident.