Big Data, Data Breaches, and the GDPR: We’ve Hit a Tipping Point

In business, your biggest commodity is your company data. And in your personal life, your biggest commodity is your personal data – although the average person on the street is only just beginning to realise this. We’ve now reached a tipping point in time. A time when the amount of data that’s being stored and transmitted is exploding, and what we are now able to discern from the resulting big data is staggering.

According to Statistica, in 2017, there were 8190 exabytes (that’s 1 billion gigabytes) of cloud centre IP traffic, along with 897 exabytes of traditional data centre traffic. According to Cisco Systems, it’s expected that by the end of 2023, cloud centre traffic will reach 19,509 exabytes. That’s per year.

This explosion in data can be attributed to the increasing use of digital technologies across various sectors, from healthcare and finance to retail and entertainment. As a result, data is now being generated at an unprecedented scale, with a plethora of information ranging from mundane interactions to highly sensitive personal data.

The convergence of emerging technologies, such as cloud computing, Internet of Things (IoT) devices, and artificial intelligence (AI), has given rise to big data – a term that encompasses the massive datasets from which valuable insights can be extracted. However, this tremendous growth in data generation has also brought about significant challenges related to data breaches and privacy concerns.

Data vs metadata vs big data

Just what is all this data going across networks? There’s plenty of business content, videos, and social networking traffic – that’s regular data.

But there’s also data such as your GPS location from your phone for Google Maps, how long you hovered over that ‘BUY’ button on the David Jones website, how often you search for medical conditions, and whether you share political posts on Facebook. The extrapolation of meaning behind each of these data points is a piece of metadata – it explains your daily movements, your buyer behaviours, if you’re a possible hypochondriac, and your political leanings. Metadata refers to the contextual information associated with data, including GPS location, browsing behaviour, and social media activities.

Big data refers to finding patterns across all of these pieces of metadata. This can be used to classify you as an X type of person, in simple terms, although it’s far more complex than that. If you’re known as an X type of person, you’ll be able to be subtly influenced according to the rules of behaviour and thought that an X type of person generally follows.

Data: The Building Blocks of Information

At its core, data represents raw and unprocessed information. It constitutes the foundation of all digital interactions and is the basis upon which complex insights and patterns are built. Within the context of networks, data can encompass a multitude of forms, such as text, images, audio, and video. For instance, when you send a text message, upload a photo on social media, or purchase a product online, you are generating data points that form part of the vast digital universe.

Metadata: Unveiling the Contextual Insights

While data provides the essential content of an interaction, metadata adds layers of context and meaning to that content. Metadata refers to the supplementary information that accompanies the data, offering insights into how, when, and where the data was generated or interacted with. It acts as a supporting framework that helps organise, describe, and interpret data in a meaningful way.

Consider the following examples of metadata:

GPS Location: When you use a navigation app like Google Maps, your GPS location data provides critical metadata about your geographical movements and preferences.

Online Behavior: The time spent hovering over a ‘BUY’ button on an e-commerce website provides valuable metadata on your buying behavior and interests.

Search History: The frequency and types of medical condition searches you perform reveal insights into your health concerns and interests.

Social Media Activity: The content you share on platforms like Facebook reflects your political leanings and preferences.

Big Data: Unleashing the Power of Patterns and Insights

Big data refers to the massive collection, storage, and analysis of large datasets, including both raw data and metadata. The volume, velocity, and variety of big data pose unique challenges for traditional data processing methods. To harness the full potential of big data, sophisticated analytical techniques and technologies, such as artificial intelligence and machine learning, are employed to uncover hidden patterns, correlations, and trends across multiple data sources.

In the context of data privacy and personalisation, big data can be leveraged to create detailed profiles of individuals, classifying them into different categories or personas based on their metadata-rich interactions. These personas are representations of user behavior, preferences, and tendencies, enabling businesses and organisations to tailor their services, products, or marketing strategies to specific audience segments.

For example, if you are identified as an environmentally conscious consumer through metadata analysis, you might receive advertisements for eco-friendly products or services. Similarly, if you are classified as an avid reader based on your reading habits and interactions, you may receive book recommendations that align with your interests.

However, the use of big data for personalisation raises ethical considerations, as it can also be used for subtle behavioral manipulation and targeted advertising. This was exemplified by the Cambridge Analytica scandal, where big data was exploited to influence political preferences based on personalised profiles.

In the interconnected digital world, data, metadata, and big data form the foundation of our online interactions. Data represents the raw content of our digital footprint, while metadata adds context and insights. Big data, on the other hand, encompasses the vast and complex web of information from which patterns and trends can be extracted.

The Cambridge Analytica scandal is just the tip of the iceberg

The Cambridge Analytica scandal involved harvesting of around 87 million users’ personal data from Facebook via an initial quiz app given to approximately 270,000 people, which was then used to create psychological profiles of each user and all their friend networks (and theirs, and theirs…), then on-sold to politicians, who were able to create advertising based on what would appeal to voters. In essence, they determined the X, Y, and Z people, and then could create advertising based on each of these personas.

Cambridge Analytica was able to determine key personality traits such as openness, neuroticism, political views, sensational interests, IQ, and more (according to leaked emails). What’s worse is that the firm’s behaviour was not illegal, it was within the terms of Facebook at the time, and participants agreed to the conditions – albeit without realising the underlying reason behind the collection of the information they were providing.

While it’s not a typical data breach, where sensitive data is stolen by hackers through exploiting vulnerabilities in organisational web applications, it amounts to treading in very murky waters – highly unethical, but scraping by on a technicality.

It’s issues like this that are starting to be addressed in new privacy regulations like the GDPR.

The GDPR and what it means for the way personal data is handled

In the wake of such data breaches and privacy concerns, the European Union introduced the General Data Protection Regulation (GDPR), which became enforceable on May 25, 2018. It’s a legally binding requirement ensuring European citizens are protected from data breaches and privacy concerns online.

The GDPR aims to protect the data and privacy rights of European citizens and applies to any organisation processing the data of EU residents, regardless of the company’s location.

The regulations set out a number of conditions that all companies collecting data must adhere to if they have European customers or visitors to their site/app and violators can be fined “up to 4% of annual global turnover or €20 Million (whichever is greater)”.

•        Must ask for expressed permission to collect data
•        Must store data securely, notifying people within 72 hours if there has been a data breach

[Note: We already have the Australian Mandatory Data Breach Notification regime in effect here, however, one of the conditions to trigger a mandatory breach notification (within 30 days) is “The breach is likely to result in serious harm to one or more of the affected individuals”]

•        Must inform the person what the company is doing with the collected data
•        Must give a person access to the data if requested
•        People may request erasure of the data

This regulation, while extremely important, will be difficult to police for companies residing outside the EU. However, companies doing regular trade with the EU, have offices in the EU, or process data from individuals in the EU need to ensure compliance to avoid trade, travel, or other disrupted services, and warning or fine notices.

The GDPR’s impact on data protection has been significant, driving businesses to reevaluate their data practices and adopt stringent privacy measures. While the implementation of GDPR has posed challenges for companies, it has undoubtedly been a step in the right direction for safeguarding individuals’ data and privacy rights. The regulation has raised awareness about the value of personal data and the need for responsible data handling.

How can Australian SMBs adopt rigorous customer data protection principles?

In Australia, data privacy has also been a growing concern. Whether you’re a B2B sales organisation, a small accounting firm, or run a nationwide chain of gyms, you need to be ensuring that your customer data is kept safe, private, and only used within the guidelines set out under Australian policy.

The Australian government introduced the Mandatory Data Breach Notification scheme in 2018, which requires organisations to report eligible data breaches to affected individuals and the Office of the Australian Information Commissioner (OAIC).

Develop a Comprehensive Privacy Policy: If you are collecting customer data, whether that be their address, how many calls they logged with you that quarter, or profiling each one for a new product you’re launching, then you need to have a Privacy Policy. Your Privacy Policy outlines what you are doing with your customers’ data and each customer must agree to it. Ensure you have a good understanding of the Australian Privacy Principles guidelines or seek the assistance of someone who does for further clarification.

Microsoft 365 compliance center (Microsoft Purview compliance portal) offers robust tools that assist in developing and implementing privacy policies that align with industry standards and regulatory requirements.

Obtain Informed Consent: Consent is a critical aspect of customer data protection. Australian businesses must seek explicit, informed consent from customers before collecting their data. This consent should be obtained for each specific purpose of data processing, as different activities may require separate permissions. Consent must be freely given, specific, and unambiguous, and customers should be provided with an option to withdraw their consent at any time.

Azure Active Directory B2C (Business to Consumer) offers a solution to efficiently manage customer identities and enable seamless consent management. Using Azure AD B2C, businesses can create customizable user experiences that include obtaining consent for data processing activities. Customers can provide explicit consent for specific purposes, helping SMBs ensure compliance with the Australian Privacy Principles and other relevant regulations.

Educate Employees and Staff: Data protection is not solely a responsibility of the management or IT department; it requires a collective effort from all employees and staff who handle customer data. Australian SMBs must invest in regular training sessions to educate employees about the importance of data privacy, the company’s privacy policy, and the implications of non-compliance. Staff members should be made aware of potential risks, such as phishing attacks, social engineering, and the use of secure data storage practices.

Implement Robust Data Security Measures: Protecting customer data from unauthorised access or breaches necessitates the implementation of strong data security measures. Encryption should be employed to secure sensitive data both in transit and at rest. Regularly updating and patching software and systems can help address vulnerabilities and protect against cyber threats.

Microsoft Azure provides a comprehensive suite of cloud services designed to enhance data security and compliance. Azure offers advanced encryption capabilities to secure sensitive data at rest and in transit. Azure also provides tools for continuous monitoring and threat detection, helping SMBs identify and address potential vulnerabilities.

Keeping data safe from breaches involves sophisticated systems and infrastructure setup. Unless you have a dedicated and experienced team on site, then this can be almost impossible to manage internally. That’s why Australian businesses are choosing to use Managed Service Providers, much like us here at A1 Technologies. Managed Service Providers take care of your infrastructure and IT management remotely, so you can concentrate on your core business, without having to spin up and manage a dedicated IT team.

With secure data storage solutions and management, safe networking configuration, monitoring and optimisation, let us help your business grow and thrive.

As for your personal data, be aware that it is being collected. Little quizzes like the ones you see on Facebook could well be a data harvester and big data consumer profiler masquerading as innocuous fun.

Rob Rattray

Rob Rattray is the Sales Director of A1 Technologies, a Microsoft consultancy based in Australia. Rob's focus is settings the strategic and cultural direction for A1 Technologies, while also overseeing Sales and Marketing. Rob is passionate about business and helping good people do good things through technology.