ChatGPT warning for businesses in South Africa

While generative AI tools like ChatGPT and Copilot may boost workplace productivity, experts warn that they create new cybersecurity risks, as attackers can exploit employees’ personal AI interactions.

Integrity360 CTO Richard Ford explained that AI has become a go-to assistant for many office workers, helping them with everything from drafting emails and summarising reports to brainstorming creative ideas.

“This widespread adoption is an undeniable productivity boon, but it also creates a new and often-overlooked cybersecurity risk – one that exists in the ‘unsupervised’ digital lives of our employees,” Ford said.

Recently, some Microsoft SharePoint servers were hacked by Chinese “threat actors” who targeted the data of the businesses using them.

In response, Microsoft released security updates and has advised all on-premises SharePoint server customers to install them.

The BBC reported that the tech giant cautioned that it had “high confidence” hackers would continue targeting systems that had not installed its security updates.

According to Ford, this breach serves as a powerful reminder that even the most robust digital ecosystems are not invulnerable.

“A compromise at this level can expose a vast amount of seemingly benign data to malicious actors – perhaps even the treasure trove of personal AI data,” he said.

He explained that the core of this emerging potential threat lies in the subtle, continuous stream of data that employees share with generative AI tools for personal, non-work tasks.

“They may ask an AI to help plan a family holiday, summarise a personal document, or even generate a social media post. Many are even, disconcertingly, starting to use AI as therapists,” he said.

While it may seem innocuous, Ford said each interaction adds a fragment of information to a larger digital profile.

“Over time, these fragments, including personal interests, communication styles, and even details about their routines, accumulate. This is the ‘digital exhaust’ of personal AI use,” he explained.

“Traditional corporate security measures are blind to this. They are not designed to monitor an employee’s personal laptop or phone, nor should they be.”

Ford warned that this creates a significant blind spot for organisations that only specialised cybersecurity providers can help fix.

“Cybercriminals, however, see a goldmine – even though they’re probably still only in the prospecting phase,” he said.

“If they got access to this treasure trove, they could leverage the aggregated personal data, maybe through a breach at a major AI provider, to craft hyper-effective and deeply personalised social engineering attacks.”

Personal information compromises security

According to Ford, the problem isn’t just about an employee accidentally inputting a company secret into ChatGPT. “That’s a known risk,” he said.

An attacker who succeeds in scraping an employee’s personal AI chats could learn deeply personal information about them.

This includes, for example, that they are planning a holiday to Cape Town next month, have a child who attends a specific school, and are frustrated with a particular internal software system, or even their state of mind.

“When combined with other publicly available information from social media or professional networks, this level of advanced profiling can become even more comprehensive and convincing to unparalleled degrees,” he said.

Ford warned that this detailed insight can then be used in several ways. This includes hyper-personalised phishing.

“An attacker could send a phishing email disguised as a travel agency reaching out about the employee’s upcoming trip to Cape Town, with a malicious attachment or a fake login page,” he warned.

Bad actors could also use this information in a targeted social engineering scheme, impersonating the employee in a message to a colleague or manager.

They may reference the employee’s recent frustration with the internal software to build rapport and trust before making a malicious, socially engineered request.

This personal information also allows hackers to understand an employee’s personal life, struggles, and interests.

As a result, an attacker can more effectively bypass their critical thinking and exploit emotional triggers to get what they want, not to mention the blackmailing possibilities.

How businesses can stay protected

Ford said that the Microsoft hack is just the latest example of how the game is changing because it proves the risk of compromised AI databases isn’t hypothetical.

“It underscores that even the most well-resourced technology companies with good reputations for security can be breached,” he said.

“If a major provider of generative AI tools can be compromised, it means that the vast repository of personal and professional data that users have entrusted to these platforms could potentially also be exposed.”

Ford said this data can then be used or sold on cybercriminal marketplaces to fuel a new wave of highly sophisticated, AI-driven attacks.

“The vulnerabilities exploited in the Microsoft attack are the type that need to be fortified, but by not resting on our laurels, these hatches can and have been battened down,” he said.

“It’s long been a race to innovate faster than malicious actors; now it’s also about staying so far ahead that you buy time for more solutions to be developed.”

He stressed that an organisation’s cybersecurity can no longer be contained within office walls. Holistic security must now encompass the wider digital footprint of employees, including the so-called “digital exhaust”.

There are a number of measures companies can take to manage these cyber blind spots.

“First, the solution is not to ban AI tools, but to provide clarity and education. Develop clear internal guidelines and policies for both professional and personal AI use,” he explained.

Ford noted that these policies should emphasise what information should never be shared, even informally.

“Second, enhance employee training. Security awareness training needs to evolve beyond simply spotting malicious URLs,” he said.

“It must now include a focus on data privacy and the risks of personal data aggregation, teaching employees to be mindful of the digital exhaust they generate.”

Finally, he said that while traditional tools may not see this risk, advanced threat intelligence and analytics can help detect anomalies.

These irregularities might signal a compromise of defences, including those that could stem from AI-attacks leveraging digital exhausts.

“This means a focus on a ‘security by design’ approach that considers all potential data exposure points, including external ones,” he said.