Unstructured Privacy Data Risks: AI Can Help

As per Gartner, 65% of world population’s data will be impacted due to privacy regulations by 2023. In fact, it might happen sooner as most countries wish to provide economic nationalism by restricting cross country data transfers and data rationing by global technology businesses.

Another Independent trend coupled with the rise of tighter privacy regulations is the volume of unstructured data being collected. it is estimated that about 60-80% of the overall data stored today is unstructured. Combined, both structured & unstructured data are projected to grow at the rate of 7-12% on an annual basis.

The Cambrian Explosion ... of Data — Source: Cprime Big Data Article

Technological advances along with ever falling storage prices have made it quite easy to collect unstructured data from the customers. And businesses are mining this data at far greater levels than before to gain competitive advantage. With the rise of social media, messaging and multiple modes of communication there is a significant rise in the collection of personal data about users’ preferences, buying behaviors and activities.

The Explosion of Data — Source: Why you should study big data

Sources with unstructured data

The sprawls of unstructured data includes customer transactions, chat, online communities, service notes, social media, voice transcriptions, call center notes, ratings/reviews and email. These sources integrate and have pointers to records stored in structured databases today and it becomes quite complex and overwhelming to detect, manage and monitor. It is fairly common where CRM systems would ingest unstructured text from messaging systems such as Slack or Facebook feeds for improving customer service.

Within these sources, the kind of information is quite varied from textual emails to FedEx PDF receipts, image formats, medical images, binary genome sequences, Twitter feeds and the list continues. In terms of variety of data sources, privacy data is scattered across in documents, messaging systems like Slack and Microsoft Teams, service tickets in Servicenow, Salesforce & Zendesk. One of the most common and largest sources would be emails containing not only customer data but employee resumes, payroll information along with backup & archival storage for these. Imagine a situation where there is a data deletion request against an employee data which is typically quite widespread in multiple sources.

Even with structured sources, data copy or deletion requests for personal data for small and medium businesses take 172 hours to fulfil vs 1,300 hours for larger enterprises. Besides, finding data manually is error-prone. Businesses might miss personal information for privacy data requests. Here automations and AI would greatly help. If we go beyond the fulfillment of privacy data requests and focus on the data security and compliance perspective, the risks and costs associated are much higher.