There is no viable way to prevent data from being collected about us in the current age of computing. But if institutions insist on knowing our financial status, purchasing habits, health information, political preferences, and so on, they have a responsibility to keep this data—known as personally identifiable information (PII)—from leaking to unauthorized recipients. At the 2017 Strata data conference in London, Steve Touw presented a session on privacy-enhancing technologies. In a fast-paced 40 minutes he covered the EU regulations about privacy, the most popular technical measures used to protect PII, and some pointed opinions about what works and what should be thrown into the dustbin.

To jump straight to Touw's conclusions: we need to maintain much tighter control over data that we share. Like most who have studied the question of PII, Touw finds flaws in current forms of de-identification, which is the technique we rely on most often for protecting PII. He suggests combining de-identification techniques with a combination of restrictions on the frequency and types of queries executed against data sets, along with a context-based approach to data protection that is much more sophisticated than current access controls.

No one would have enough time to explain thoroughly all the issues in protecting PII. Touw focused on European legal requirements (which made sense for a conference held in London), technical difficulties in de-identifying data, and good organizational practices for protecting privacy. This article fills out some of the background underlying these issues as well.

Common constraints on data collection

Although people viscerally fear the collection of personal data, and alternatives such as Vendor Relationship Management have been suggested for leaving control over data in the hands of the individual, there are few barriers in the way of organizations that collect this data. The EU has regulated data collection for decades, and its General Data Protection Regulation (GDPR), which is supposed to come into force on May 25, 2018, requires limitations that are familiar to those in the privacy field. These include minimization, data retention limits, and restrictions on use to the original purpose for collecting the data. I'll offer a brief overview of these key concepts.

Minimization means collecting as little data as you can to meet your purpose. If you need to know whether someone is old enough to drive, you can record that as a binary field without recording the person's age. If you need to know how many cars pass down a street each day in order to plan traffic flow, you don't need to record the license plates of the cars.

Data retention limits are a form of minimization. Most data's value diminishes greatly after a few months. For instance, a person's income may change, so income information collected a year ago may no longer be useful for marketing. Therefore, without much of a sacrifice in accuracy, an organization can protect privacy by discarding data after a certain time interval.

Restricting use to the original purpose of data collection is an even stricter criterion. Supposedly it would mean that a retailer who collects your information in order to charge your credit card should not use that information to improve its marketing campaigns.

Governments in the US impose restrictions only on specific classes of information, such as data collected by health care providers. Fair Information Practices, which cover some broad issues such as transparency and the right to correct errors, are widely praised but not required by law. They also go nowhere near as far as EU laws in granting rights to individuals for their data.

Although the GDPR does not require organizations to obtain consent for data collection, Touw advised them always to do so. Otherwise, the organizations may be asked to demonstrate in court that they had a "legitimate interest" in the data, which is a subjective judgment. Touw did not go into the problems of consent forms, so his advice was really aimed at protecting the company doing the collection, not the individuals.

The dilemma of data sharing

Organizations can realize a large income stream from selling the data, which can then be used for purposes ranging from benign to ill. Governments collect and share data that is supposed to be for the public benefit (e.g. race and gender, incidences of communicable diseases). Public agencies, and even some companies, believe their data could contribute to initiatives in health, anti-corruption efforts, and other areas. Some institutions also anticipate that they might benefit from tools developed by others. Thus, Netflix released data on who viewed its video content for the Netflix prize of 2009, hoping to get a better algorithm for video recommendations from experts in the field.

When data is shared publicly, the organization tries to strip direct identifiers, such as names and social security numbers, and tries to reduce the risk that indirect identifiers such as postal codes can be used to re-identify individuals. Even when organizations sell their data privately, they often try to de-identify it in similar ways. The GDPR gives organizations pretty much free rein to use and release data, so long as it is correctly de-identified.

Problems with de-identification

The bulk of Touw's talk was devoted to the risks of de-identification, also known as anonymization. His skepticism about de-identification is shared by most experts in computing who have examined the field. In particular, he looked at techniques for pseudonymity and K-anonymity, claiming that they can't prevent re-identification unless they're pursued so far that they render the output data useless.

Touw predicted that organizations will stop releasing free, de-identified data sets, because de-identification has too often proven insufficient and too many embarrassing breaches have been publicized. Besides the Netflix prize mentioned earlier, where researchers re-identified Netflix users from the data [PDF], Touw mentioned some other open data sets and spent a good deal of time on New York City taxi data.

All these re-identification attacks depended on the mosaic effect, or finding other publicly available sources and joining them with the released data set. (Touw called this a "link attack.") In the case of the New York taxi data, most of us would have nothing to fear, but celebrities who are sighted at the beginning or end of their rides could potentially be re-identified. Touw claimed that New York City could not have prevented the re-identification by fuzzing or removing fields from the data, a point also made by the researcher who originally performed the re-identification attack. I believe Touw moved the goalposts a bit by adding new sources of information to fuel possible attacks as he removed existing information. Still, he made a case that the only way to protect celebrities would be to remove everything of value from the data.

Pseudonymization is the easiest way to de-identify data. It consists of putting a meaningless value in place of a personally identifying field. People may still be re-identified, though, if they possess unique values for other fields. For instance, if someone is the only Hispanic person in a particular apartment building, a combination of race and address can identify them. If someone suffers from a rare disease, a hospital listing with diagnoses may reveal sensitive information to someone who knows they have that disease.

K-anonymity addresses the problem of unique values, known also as high cardinality values. The technique makes sure there are enough duplicate values in different rows of data so that no individual is identified by a particular combination of fields. K-anonymity works by making values in fields more general: a common example is offering just the first three digits of a five-digit ZIP code. Because the digits are hierarchical (the code 200 is a single contiguous geographic area that contains 20001, 20002, etc.), generalizing the ZIP code exposes data that is still useful but is less specific.

Touw briefly mentioned two enhancements to K-anonymity, known as L-diversity and T-closeness, that are more restrictive. L-diversity [PDF] restricts the number of unique values in information by taking into account the probability that an attacker can guess something about the target (such as their address). T-closeness [PDF] tries to prevent re-identification by making sure that each division in the data (such as ZIP code) contains sensitive values with about the same frequency as the general population. Touw claimed L-diversity and T-closeness are more trouble than they're worth, and that all these techniques leave people at risk of re-identification unless the data is generalized to the point where it's worthless.

When you listen to data scientists like Touw who have investigated the limitations of anonymization, you come away feeling that there's no point to doing it. But let's step back and consider whether this is a constructive conclusion. Nearly all published examples of re-identification took advantage of poor de-identification techniques. Done right, according to proponents, de-identification is still safe. On the other hand, it's easy for proponents of de-identification to say that a technique was flawed after the fact.

To resolve the dilemma, one can look at de-identification like encryption. We can be fairly certain that, within a few decades, increased computing power and new algorithms will allow attackers to break our encryption. We keep increasing key sizes over the decades to compensate for this certainty. And yet we keep using encryption, because nothing better exists. De-identification is still worth using too. But Touw has some alternative ways to carry it out.

Proposed remedies

In addition to advising that organizations obtain consent for data collection, Touw offered two practices that are more effective than the previous methods of data protection: restricting data requests to a safe set of queries and using context-based restrictions. Neither practice is in common use now, but models exist for their use.

If an organization does not release data in the open, it can achieve some of the organizational and social benefits of open data by offering a limited set of queries to third parties. Touw promoted the concept of differential privacy, which is a complex technology understood by relatively few data experts. The concept has been attributed [PDF] to Cynthia Dwork, who co-authored a key paper [PDF] laying out the theory. She explains differential privacy there (on page 6) by saying, "it ensures that any sequence of outputs (responses to queries) is 'essentially' equally likely to occur, independent of the presence or absence of any individual." It never reveals any specific fields in the underlying data, but provides a set of aggregate queries—such as sums or averages—that mathematical analysis of the data set have shown to be privacy-preserving.

Touw demonstrated how a specific value for a specific person might be obtained by asking the same question—or to disguise the attack, many questions that differ slightly—over and over. Each question produces a slightly different result in the field you're interested in, but if you take the average of these results you can get very close to the original value. So some form of rate-limiting must be imposed on queries.

Touw's other major recommendation involves context-based or purpose-based restrictions, which he called "the future of privacy controls". They go far beyond individual or group access controls used by most sites.

One example of context-based restrictions is time-based access. A conventional employer might allow access by its employees from 9:00 AM to 5:00 PM. In a more flexible environment, such as a hospital where nurses' shifts have irregular beginnings and ends, the hospital may allow each nurse access to data when their schedule indicates they are on duty.

Another type of context-based restriction is based on granting users limited access to data based on a license that spells out what they want to do (say, cancer research) and how they can use data. If the user starts issuing requests for certain combinations of rows or columns that don't seem to fulfill the basis for which the license was granted, access can be denied.

Touw advises organizations not to try to combine all their data in a single data lake—or worse still, to copy data into a new repository in order to perform access controls. Maintaining two copies of data is always cumbersome and error-prone. In addition, you now offer attackers twice the opportunities to break into the data. Instead, he suggests an organizational set up what he calls a "data control plane". It implements all the policies defined by the organization and covers all data stores. The control plane should expose easy ways to create rules, make sure new policies take effect immediately, recognize the types of context mentioned earlier, and maintain audits that show what the data was used for. Organizations must also exercise governance over data so they know who owns it, who has access to it and under what circumstances, and how to manage the data's lifecycle (acquiring, storing, selling, purging). They can't just rely on the IT department to define and implement policies.

Few if any commercial vendors offer the advanced privacy-protecting technologies recommended by Touw. So at this point, attackers run ahead of most organizations that maintain data on us. Still, Touw's talk opens up a valuable debate about what real privacy protection looks like in 2017.