Artificial Intelligence is no longer something that belongs in science fiction, technology conferences or distant global debates. It is already here. South Africans are using AI to write emails, draft documents, prepare content, analyse information, generate images, automate work, support customer service and make business decisions.
But the recent withdrawal of South Africa’s Draft National Artificial Intelligence Policy has shown us something important: AI may be powerful, but it is not automatically reliable. The draft policy was withdrawn after fictitious sources were found in its reference list. That detail matters. It is not just a technical mistake. It is a public example of the very risk that AI regulation is supposed to address: when technology produces information that looks credible, sounds convincing, but has not been properly verified.
This is exactly why South Africans need to understand AI, not fear it blindly, but also not trust it blindly.
What is AI?
Artificial Intelligence (AI), refers to technology that can perform tasks that usually require human intelligence. This may include writing, analysing data, recognising patterns, generating images, summarising information, answering questions or assisting with decision-making.
Many people now use AI tools in ordinary daily life without always realising the legal consequences. A business may use AI to draft a client email. A student may use it to summarise research. A company may use it to screen applicants. A professional may use it to prepare a report. A content creator may use it to generate ideas or captions.
The problem is not that AI exists. The problem is when people use AI without understanding what it can get wrong. AI can produce false information. It can misunderstand context. It can reproduce bias. It can generate content that infringes on someone else’s rights. It can expose personal or confidential information. It can also create a dangerous sense of certainty because the answer often appears polished, confident and professional.
That is why AI must be treated as a tool, not an authority.
What South Africa’s withdrawn AI policy teaches us
South Africa’s withdrawn draft AI policy is a clear warning.
If a national policy document dealing with Artificial Intelligence can contain fictitious sources, then ordinary users, businesses and professionals must be even more careful. This is not because AI is useless. It is because AI output can look correct even when it is not. That is one of the biggest risks of AI: it does not always fail obviously.
A bad AI answer may not look messy. It may look sophisticated. It may use legal language. It may cite sources. It may sound logical. But if the information has not been verified, the risk remains. For businesses and professionals, this can create serious consequences. If AI is used to prepare advice, policies, marketing claims, contracts, reports or decisions, and that information is wrong, the person or business using it may still be responsible.
You cannot simply blame the tool.
What should South Africans be careful of?
The first major concern is personal information.
If you insert a client’s personal details, employee information, medical information, financial data, identity numbers, contracts or confidential documents into an AI tool, you may be processing personal information in a way that triggers legal obligations. Under POPIA, organisations must be careful about how personal information is collected, used, stored, shared and protected.
The second concern is confidentiality.
Professionals and businesses must be especially careful before entering confidential client information into AI platforms. Once sensitive information is placed into a system, you may not fully control where it goes, how it is stored, or whether it can be accessed later. This is particularly important for lawyers, consultants, employers, accountants, healthcare professionals and anyone dealing with private information.
The third concern is accuracy.
AI can generate information that is false, outdated or completely fabricated. This is sometimes referred to as an AI “hallucination”. In simple terms, the tool may produce an answer that sounds real, but is not actually true.
The fourth concern is bias.
AI systems are trained on existing information. If that information contains bias, incomplete data or unfair assumptions, the output may repeat or amplify those problems. This becomes especially dangerous where AI is used in recruitment, lending, insurance, education, policing, disciplinary processes or access to services.
The fifth concern is accountability.
If a business uses AI to make or support a decision that affects another person, it must be able to explain and justify that decision. A company cannot hide behind a machine when a person’s rights, employment, finances, privacy or dignity are affected.
What should people prevent?
South Africans should avoid using AI as a replacement for professional judgment.
AI should not be used to make serious legal, financial, employment, medical or business decisions without human review. It should not be used to process sensitive information without understanding the privacy risks. It should not be used to generate public claims, legal content or professional advice without checking the sources.
Businesses should also avoid presenting AI-generated content as fact without verification. This is where reputational damage begins. Once false information is published, used in a report or sent to a client, the excuse that “AI wrote it” is weak. The legal and professional responsibility often remains with the person or organisation who used it.
AI should assist the process. It should not replace accountability.
What is good about AI?
AI is not the enemy.
Used correctly, AI can be incredibly useful. It can save time, simplify complex information, help small businesses work more efficiently, support research, improve access to information and assist with communication.
For a country like South Africa, AI also has potential to support innovation, education, entrepreneurship and service delivery. It can help people who do not have easy access to expensive professional services. It can make information more understandable. It can support administrative efficiency and help businesses compete in a digital economy.
In the legal space, AI may assist with summaries, document organisation, research support and public education. For consumers, it may help people understand basic concepts before seeking proper advice. For businesses, it may help with planning, drafting and internal systems.
But the benefit only exists when the tool is used responsibly.
What is dangerous about AI?
The danger of AI is not only that it can be wrong. The danger is that it can be wrong, confidently so. That is what makes AI different from an obvious mistake. A poorly written answer is easy to question. A polished answer feels trustworthy. This is why AI can mislead people so quickly.
There is also a deeper legal concern: AI may affect rights before the law has fully caught up. South Africa does not yet have a final AI-specific Act or binding national AI policy. However, this does not mean AI exists in a legal vacuum. Existing laws may already apply, including laws dealing with privacy, cybercrime, consumer protection, intellectual property, employment, equality and constitutional rights.
This means users should not wait for a future AI Act before acting responsibly. The legal risks already exist.
What should businesses do now?
Businesses should begin by creating clear internal rules for AI use.
Employees should know what information may and may not be entered into AI tools. Sensitive information, confidential documents and personal data should be handled with extreme caution. AI-generated content should be checked before it is published, relied on or sent to clients.
Businesses should also keep human oversight in place. Where AI assists with decisions, a human being should still review the outcome, especially where the decision affects another person’s rights, opportunities or access to services.
The question should not be, “Can AI do this?” The better question is, “Can we justify how AI was used, and can we prove that the final result was properly checked?” That is where responsible AI use begins.
Conclusion
South Africa’s withdrawn AI policy is more than a government embarrassment. It is a national lesson.
AI can help us move faster, but speed without verification is dangerous. AI can help us work smarter, but convenience without accountability is reckless. AI can assist with information, but it cannot replace human judgment, legal responsibility or ethical decision-making.
The future will not belong to people who reject AI completely. It will also not belong to people who trust it blindly. The future will belong to those who know how to use AI carefully, critically and responsibly. At The Law Box, we believe legal awareness must move with the times. As technology changes, so do the risks, the responsibilities and the questions South Africans need to ask. AI is here. The law is catching up. But responsibility starts now
Yours sincerely,
Sharné Montgomery
Founder, The Law Box
