South Africans who use AI tools from popular platforms like ChatGPT, Claude, Grok, and Google Gemini at work should be aware of the complex legal challenges that may arise.

This is especially the case when these AI tools are used in decision-making that involves human work and may impact their employment or gauge their performance.

Any usage of AI tools in human resources work in South Africa can have dangerous implications under the Protection of Personal Information Act (POPIA) and the Employment Equity Act (EEA).

Bowmans, a top legal firm in South Africa, explained that since South Africa does not have a dedicated AI Act, the regulation of AI in the employment context falls to existing legislation of general application.

AI chatbots from third-party companies can be used for automated CV screening, chatbot-led interviews, and algorithmic performance monitoring and predictive analytics of the workplace.

“Employers are increasingly deploying AI systems across the employment lifecycle. These tools promise efficiency gains and data-driven objectivity,” Bowmans said.

“These tools promise efficiency gains and data-driven objectivity. However, they also present significant legal risks, particularly in the areas of privacy, non-discrimination and fair labour practices.”

However, these activities may infringe on POPIA. The Act states that data subjects may not be subject to a decision which affects them to a substantial extent through the automated processing of their data.

This takes place when “an AI system generates a performance rating, identifies an employee for promotion or demotion, or recommends that a candidate be rejected for a role.”

Bowmans said that, to avoid being in breach of Section 71, employers must ensure they take measures that allow the data subject to make a representation.

Employers must also ensure that data subjects are provided with sufficient information about the automated processing and what will happen to their data.

“The responsible party must therefore have sufficient understanding of how the automated decision-making process came to its result,” Bowmans said.

Understanding this automated decision-making can be difficult because many third-party AI tools are not necessarily easy to understand, nor do they disclose details on how they arrive at their outputs.

AI-only recruitment can breach POPIA

Bowmans said that using AI in the recruitment process could also lead to breaches of POPIA if companies do not take the necessary measures.

If AI tools are used in the recruitment or promotion process where the tools are used to shortlist a candidate, the affected employee or applicant must be given the opportunity to challenge the outcome.

“Affected employee or applicant must be afforded the opportunity to be heard and to challenge the outcome of the automated decision-making,” Bowmans said.

These affected applicants or employees must also be given the opportunity to engage with a human decision-maker and insight into how the AI system reached its conclusion.

“It would be important for recruitment agencies or HR personnel relying on AI to shortlist candidates to consider whether the persons excluded have been excluded on grounds that are rational,” the firm said.

This meant that the recruiting and selecting potential employees in South Africa solely using AI models is technically in breach of Section 71 of POPIA.

“Even if section 71 of POPIA is not triggered, the principles of lawful processing and other obligations in POPIA will still apply,” Bowmans said.

Employees warned against giving personal information to ChatGPT or Gemini

Another significant risk arises where employees use AI systems in the course of their day-to-day work and, in doing so, upload personal information to them.

Employees who input personal data, whether belonging to colleagues, customers or third parties, into a publicly accessible AI tool like ChatGPT or Gemini, will see their companies held accountable.

“This is so because the employer determines the purpose and means for processing in the employment context,” Bowmans said.

“The employer must secure the integrity and confidentiality of personal information under its control.”

Bowmans said that to mitigate the risk, employers should implement acceptable AI use policies that clearly define which uses of an AI tool are allowed and which are not.

“Establish guardrails on what categories of personal information may be uploaded to AI systems (if any),” it said.

“Training and awareness programmes should accompany these measures to ensure that employees understand their obligations under POPIA when interacting with AI systems.”

Source: https://mybroadband.co.za/news/ai/668816-lawyers-send-warning-to-people-who-use-chatgpt-gemini-and-claude-at-work-in-south-africa.html