AI, demon-hunters and the lost art of critical thinking
Thirty years ago, scientist and author Carl Sagan penned Demon -Haunted World, a riposte to the hype and mumbo-jumbo of the times. The book delivers invaluable lessons for today’s leaders.
5-minute read – 100% written by humans
A participant at one of our workshops recently emailed to say that critical thinking had helped her to see through all the hype and in her words, it felt ‘like breathing fresh air again.’
Sagan wrote the book – a paean to critical thinking – against a backdrop of rising new-age pseudoscience, cranky management fads and an ever-inflating dot.com bubble. Thirty years on, the internet has now transformed the way we live, but we shouldn’t forget that the bubble burst, billions were wiped out and many start-ups went to the wall. AI has the potential to help humanity, but to navigate its hidden depth-charges, leaders need to apply critical thinking.
Compensatory strategies
Sagan would point out that when we hear startling or inflated claims, we should be suspicious. These are often what psychologists call ‘compensatory strategies.’ Put simply, the grander the claim the more probable that there’s a hidden motivation behind it. There are so many from the AI industry that it’s difficult to know where to start, but here are a couple of my favourites:
In February 2022, Ilya Sutskever a co-founder and senior leader at Open AI posted on X:
“It may be that today’s large neural networks are slightly conscious.”
The response from Murray Shanahan, an AI researcher from Imperial College London, cut to the quick:
“… in the same sense that a large field of wheat is slightly pasta.”
Marc Andreessen in his 2023 essay, The Techno-Optimist Manifesto, wrote: “We believe Artificial Intelligence is our alchemy, our Philosopher’s Stone – we are literally making sand think.”
Except that we’re not, are we. The silicon chip (the sand) may be capable of encoding mind-bogglingly vast amounts of information into unfathomably small spaces, but it’s not thinking; at least no more so than the Gutenberg Press made paper think.
Real world data and the contrarian viewpoint
This brings us on to one of Sagan’s main tenets. When we hear a claim, we should be clear whether the person making the claim has a vested interest in our believing it. If they do, it doesn’t necessarily nullify the claim, but it does mean that we should both seek out contrarian viewpoints and look at what the real-world data are telling us.
If we look at the big and bold claims from the tech industry over recent years three key themes emerge:
- Inevitability: The primacy and mass adoption of the technology are inevitable. Therefore, the debate is over.
- Universal benefit to humanity: The future of humanity is in the hands of the AI industry and they are the guys wearing the white hats.
- We should get with the programme… or be dumped in the loser pile.
Going back to Sagan’s principles, do they have a vested interest in our believing these claims? And if so, are there alternative viewpoints and what do the real-world data tell us?
Get your terms precise
The first step is to define our terms. What do we mean when we talk about AI? Artificial Intelligence was always a catch-all term. It was developed by researchers in 1956 in the belief it would more readily attract funding.
Today we face a situation where the phenomenal increase in processing power may lead to the sort of breakthrough in life sciences that will free humanity from Alzheimer’s. The essential analysis that would take 100 PhD students 100 years can be done in a micro-sliver of that time.
That’s very different from that digital colleague LLM chatbot who can take your place on the customer services team. It makes sense for the industry that’s pushing LLM-driven avatars for profit to hitch its product onto the wagon of life-saving research.
The same underlying technology may ultimately drive them both, but it’s like comparing Chat-GPT with an MRI scanner.
In the commercial world, the main promise is that the technology will maximise profits by boosting productivity. That means replacing what Standard Chartered CEO Bill Winters called “low-value human capital” when announcing plans to eliminate roughly 7,800 roles as part of a major AI automation.
Is there a falsifiable prediction?
To be meaningful, any claim about the future requires a falsifiable prediction. If AI is about to turbo charge productivity across the economy, by when and by how much? What are the best, worst and middle-case scenarios?
Sure, there are multiple variables to consider, but statistical methods are quite capable of controlling these and pinpointing what percentage change is explained by AI. Without a falsifiable prediction, a claim about the future is wishful thinking.
So, what does the data tell us and what’s the alternative perspective? When you look at the real-world data that we have to date, the picture is quite mixed.
What the real-world data show
A study by Mert Demirer and colleagues from MIT looked at productivity gains among software developers, at different stages in the cycle. The impact at the top of the funnel shows a staggering 300% more files created. But when you then look at the actual pieces of work submitted for review, the productivity increase halves to 150%. Then it gets more interesting. At the stage of software releases the number falls to 30%. Now, a 30% uplift is not to be sniffed at, but when the researchers looked at actual changes in consumption, they discovered that the benefits of AI disappeared altogether.
Completeness and complexity
The 300% increase figure is real. But any statement that is incomplete and ignores complexity is misleading.
Here’s an example of what we’re talking about from another domain: As a measure to combat climate change, new buildings above a certain size in the city of London were required to derive a certain proportion of their energy from biomass as opposed to fossil fuels. This meant that individual buildings emitted less carbon. That statement is true, but it ignores complexity. The biomass had to be delivered by lorry through the London traffic which meant the overall carbon footprint of the building was far greater.
In the mental health domain, chatbots may mean that we can scale up the number of people benefiting from talking therapy. But the two primary determinants of mental health – meaningful work and social contact – are both threatened as AI replaces workers.
The statement that AI will generate unprecedented increases in economic value may be true but it’s incomplete and fails to acknowledge complexities. How will the value be distributed? To what extent will it increase inequality? What will be the effects on the ranks of workers in the developing world who are already having to do mind-numbingly tedious work with very low long hours and low pay to provide the human backup that the system currently needs? (Often this work is emotionally damaging as it requires long periods of looking at disturbing images.)
The response of the AI industry to the charge of environmental degradation is that we shouldn’t worry. AI is so smart that it’s bound to crack the problem of climate change anyway. But the critical thinker would point out that without a clear description of mechanism and pathway the reassurance is an empty one.
And what of the industry’s claim that the AI-driven future is inevitable? AI boosters like to talk in metaphors such as ‘tsunami.’
Of course, inevitability means that the time for debate and scrutiny is over. The ship’s sailed. It’s time to get with the programme or be on the wrong side of history.
The real-world tells us that we have choices
However real-world experience tells us that we have choices. Take the example of so-called ‘designer babies.’ The technology for such gene-editing exists. Parents can theoretically select for desirable traits. But all countries effectively prohibit this kind of reproductive use. Human societies have invariably taken the view that although the technology exists, we will not use it until we understand it’s impact more fully.
In 1996, the US company Monsanto announced that it had created biogenetically modified maize and soya beans that were herbicide resistant. The assumption was that GMO foods would become ubiquitous in our food chain. In reality, they now represent a miniscule proportion. As a society we made a decision about what we want.
Beware the false dichotomy
There is no tsunami. We are free to make choices about the sort of future that we want. And that choice is not between an AI future and a not-AI future. That’s precisely the sort of false dichotomy that critical thinking encourages us to challenge. It’s not a binary choice. We should resist the idea that we must lineup with Team Pro or Team Anti. We’re free to choose. And that choice may well be to be a thinking adopter.
Thirty years on, critical thinking remains the best prophylactic against doing dumb stuff.
To find out how we can help leaders in your organisation to be more impactful, influential and persuasive visit www.threshold.co.uk
