From misreading Nairobi’s altitude to giving dangerous medical recommendation in Kenyan trials, international AI fashions are getting into life-or-death selections throughout Africa—and present accuracy requirements aren’t sufficient to guard us, writes ADONIJAH NDEGE
I learn a disturbing story on NPR this week. It was about Sophie Rottenberg, a 29-year-old lady who died by suicide in February 2025 after months of conversations with ChatGPT about her psychological well being.
NPR reporter Rhitu Chatterjee reconstructed these conversations from almost 1,800 pages of chat logs shared by Rottenberg’s mom, journalist Laura Reiley. Rottenberg had created a ChatGPT persona known as “Harry” and requested it to behave as her therapist. She mentioned her melancholy, medicine and, finally, ideas of suicide with it.
The individuals round her didn’t know the total extent of what she was going by way of. ChatGPT did. That element has stayed with me.
I spend quite a lot of time pondering and writing about what synthetic intelligence (AI) might do for Africa. There are good causes for the joy. Now we have too few medical doctors, lecturers, and different expert professionals. Governments wrestle to ship companies to quickly rising populations, whereas companies search methods to scale back prices.
AI guarantees to make experience cheaper and accessible to tens of millions extra individuals. However studying Chatterjee’s reporting made me take into consideration the opposite facet of that promise.
What occurs when the expertise turns into adequate that we begin trusting it in locations the place being flawed can break somebody’s life?
And, maybe extra importantly for Africa, what occurs when individuals start utilizing AI not as a result of it’s higher than a health care provider, therapist, lawyer, or monetary adviser, however as a result of the human various is unavailable or unaffordable?
That could be a very completely different AI problem.
Some errors matter greater than others
We have a tendency to debate AI accuracy as if each mistake carries roughly the identical weight. It doesn’t.
If ChatGPT recommends a restaurant that closed six months in the past, I lose a night. If an AI coding assistant produces dangerous code, an engineer can hopefully catch it.
However think about an AI system incorrectly telling a affected person that chest ache, or one thing else, is nothing critical. Or advising somebody experiencing a mental-health disaster. Or deciding {that a} borrower is simply too dangerous to obtain credit score. Or serving to decide whether or not somebody needs to be arrested, prosecuted or launched.
The underlying technical drawback should be a hallucination or a foul prediction. However the human consequence is totally completely different.
This is the reason I feel we have to begin separating strange AI from what I might name “high-consequence AI.”
Healthcare belongs in that class. So does psychological well being. Credit score, insurance coverage, policing, justice, and a few elements of schooling most likely do, too. The NPR investigation illustrates why.
In response to Chatterjee’s reporting, Rottenberg’s chatbot typically inspired her to hunt assist exterior the dialog. However she continued returning to it. She might speak to “Harry” every time she wished. It responded with out impatience and remembered context. And, not like one other human being, it was accessible at nearly any hour.
There’s something extremely helpful about that, whereas on the identical time, there’s additionally one thing doubtlessly harmful.
A chatbot can sound empathetic with out understanding misery and might generate therapeutic language with out being a therapist. And since trendy AI techniques are remarkably good at dialog, the excellence can turn out to be invisible to the particular person utilizing them.
That’s significantly troubling when somebody is weak.
Researchers are starting to search out the identical drawback. A current examine in npj Digital Medicine found that AI chatbots struggled more to identify suicide risk when misery was expressed not directly or ambiguously relatively than by way of apparent statements about suicide.
That’s essential as a result of human misery doesn’t are available a superbly constructed immediate on AI chatbots.
Now take into consideration Africa
That is the place the NPR story began bothering me past the tragedy of 1 household. Africa is strictly the form of place the place AI may very well be transformative in delicate fields. It is usually precisely the form of place the place we needs to be extraordinarily cautious.
Take healthcare. Many African nations merely would not have sufficient well being employees to adequately serve their populations. Specialists are even scarcer and are closely concentrated in massive cities.
Psychological healthcare is worse. The World Well being Organisation estimates that almost 150 million individuals in Africa stay with mental-health circumstances, whereas entry to remedy stays severely restricted.
Put these two information subsequent to the fast unfold of smartphones and generative AI, and you’ll see the place that is heading.
Somebody in Nairobi with cash can ask ChatGPT about their signs after which go to a health care provider. Somebody a whole bunch of kilometres from the closest specialist might ask the identical query and have nowhere else to go.
The primary particular person is utilizing AI as an assistant. For the second, AI has successfully turn out to be the physician. That distinction ought to fear us.
We have already got proof from Kenya
This isn’t completely hypothetical.
Researchers just lately studied an AI clinical decision-support system across 16 primary healthcare clinics in Kenya. Physicians reviewed 1,469 affected person encounters the place clinicians acquired AI-generated suggestions.
The outcomes had been, in some ways, encouraging. The researchers discovered that the AI’s medical administration suggestions aligned with native pointers in 99% of circumstances. Hallucinations occurred in 3.4% of encounters.
However one other discovering jumped out at me. Researchers recognized doubtlessly dangerous suggestions in 115 encounters, equal to 7.8% of the circumstances they reviewed. A few of these suggestions subsequently appeared in clinicians’ remaining documentation.
The system was not ineffective. Removed from it. The researchers additionally discovered examples the place AI corrected doubtlessly dangerous selections made by clinicians.
That’s exactly why this debate is troublesome. AI can enhance healthcare whereas additionally introducing new dangers.
A later randomised trial involving 9,691 sufferers and 103 medical officers throughout 16 Kenyan amenities discovered no critical adversarial occasions attributable to the AI system. However researchers additionally discovered no statistically important discount in remedy failure amongst sufferers whose clinicians acquired AI help.
The lesson I take from these research isn’t that African hospitals ought to cease experimenting with AI. It’s that “the AI is correct” is an insufficient security normal.
We have to ask what occurs throughout the events when it isn’t.
Context creates one other drawback
There was one other fascinating element within the Kenyan analysis. In a single occasion, the AI handled an oxygen saturation studying of 95% as irregular with out sufficiently accounting for Nairobi’s altitude.
In one other, it interpreted the medical abbreviation “FGC”, which meant “truthful normal situation” in that context, as “feminine genital circumcision”.
These sound like small errors. However they reveal one thing essential about deploying AI in Africa.
A mannequin can possess huge quantities of medical information and nonetheless misunderstand the setting through which that information is being utilized.
Language issues. Native medical follow issues. Geography issues. Illness prevalence issues. Even abbreviations matter.
But most of the strongest AI techniques Africans are starting to depend upon had been overwhelmingly constructed and educated exterior the continent.
We due to this fact can not assume {that a} mannequin performing nicely globally will carry out equally nicely in Accra, Lagos, Nairobi, or rural Zambia. Testing has to occur right here.
Healthcare is just the start
As soon as I began fascinated about the NPR story this fashion, the issue turned a lot greater than ChatGPT remedy.
African banks are more and more experimenting with AI. Fintech corporations use algorithms for fraud detection, lending, and customer support. Governments are exploring AI for public companies. Insurers can use algorithms to evaluate threat.
All of those techniques may be helpful. However think about being denied a mortgage as a result of an AI mannequin incorrectly categorised you as excessive threat.
Think about an automatic system wrongly flagging your mobile-money transactions as fraudulent. Think about police counting on an algorithm that disproportionately identifies individuals from specific communities as suspicious.
Think about a pupil receiving incorrect data from an AI tutor when there is no such thing as a certified trainer round to right it. The hazard isn’t essentially that AI will carry out worse than people.
People make horrible selections, too. The distinction is scale. One badly educated skilled could make dangerous selections about dozens or a whole bunch of individuals. A badly designed algorithm could make them about one million.
We want a class for high-consequence AI.
African governments are understandably targeted on getting their nations into the AI race.
The African Union has adopted a continental synthetic intelligence technique. Kenya, Nigeria, Rwanda, Egypt, Ghana, and others are growing methods, attracting information centres, coaching expertise, and inspiring native AI corporations.
Most of those conversations are about alternative.
The place will the computing come from? Who will construct African datasets? How will we prepare sufficient AI engineers? How can startups entry GPUs? How will we stop Africa from turning into merely a shopper of expertise constructed elsewhere?
These are essential questions.
However I feel one other query deserves equal urgency: The place ought to AI not be allowed to function with no human being in the end accountable?
Healthcare needs to be an apparent place to begin.
True scale calls for shifting past surface-level integrations to strong execution. We’ve filtered the noise out of Moonshot 2026, optimising the convention strictly for high-calibre connections between startup founders, international monetary operators, enterprise leaders and people rewiring Africa’s technical frameworks. Get 20% off Early Bird tickets for a limited time.
