Are we being subdued?

Long post alert!

Over the past few years, the large-scale commercialization of AI chatbots has quietly reshaped how we work, and I would argue to a degree, how we think.

Companies jumping on the bandwagon, less because it solved a real problem but because it enhanced a valuation, because it is in demand. Ordinary users followed, outsourcing more and more of their thinking to a chatbot. What worries me isn't this dependence, it's a given that every new creation leads to some dependency some way or the other, it's more of a subtler pattern I keep running into during my work - people seem to be getting less sharp, even as their output looks more impressive than ever.

I work in a sort of an economic consulting domain, dealing with clients across various sectors. Meet some of them in person, and I sometimes think there couldn't be a bigger dumbass than this person in front of me. But open their emails, and suddenly I'm dealing with Shakespeare who majored in supply chain logistics. Every sentence is crisp, structured, and academically aligned, and the best part, it is almost entirely useless.

It reads well but isn't applicable to the real world. The demands raised are so textbook perfect which would never work with actual implementation process. Read enough of these and you'll realise which chatbot they are using (Claude, GPT, copilot).

There's another issue which bothers me even more. As chatbots is/have creating/created an industry of experts - any person with a prompt can sound like the appropriate figure on anything from a macro policy to supply chain risk to geopolitical issues. But how is any of that authority earned?

Every tool highlights a disclaimer that it can get things wrong and we know it, and yet we treat its output these days as the gospel. The worst part is we don't even know how these models are trained, what are the data sources, the filtering, how do they provide weightage to stuff, LLM itself is a black box and we sit behind a wall of 'trust them'.

Now take this thought a step further - what happens when a piece of wrong information gets published somewhere, gets scrapped, and becomes training data? The model repeats it. That repetition gets picked up and republished elsewhere, and eventually becoming a training data for future models. One mistakes leads to another and you have a feedback loop, a kind of a butterfly effect for misinformation. One bad input is amplified and alters an entire generation.

In my opinion, we're being subdued not only by the companies pushing it for profit (they have a minor role is this) but majorly by ourselves.

What do you guys think?