Understanding privacy policies

Abstract

Their is growing recognition that users of web-based systems want to understand, if not control, what customer’s data is stored by whom, for what purpose, for what duration, and with whom it is shared. We inform current language-based privacy efforts with an empirical study of P3P—the W3C domain-specific language for privacy policies. We use methods of software language engineering to study usage profiles, correctness of policies, metrics, cloning, and language extensions. The study supports the conclusion that P3P’s approach to policy validation is too weak to ensure correct use of the language. The study also discovers common, dominating policies, which may suggest a simpler approach to web privacy. Further, the study investigates a range of metrics for policies in an attempt to discover particularly interesting or complex policies. Finally, the study also attempts to discover symptoms of the need for extending the P3P language, but the found results are not conclusive here.

Appendices

Appendix

1.1 A.1 Additional Information on ODP

The catalog of Open Directory Project (ODP) is subdivided into categories and subcategories through multiple levels. There are these top-level categories: Arts, Business, Computers, Games, Health, Home, Kids and Teens, News, Recreation, Reference, Regional, Science, Shopping, Society, Sports,World. There are two special top-level categories: World and Regional. The following discussion of these two special categories should be helpful when trying to understand ODP in general and its overall coverage of the WWW.

Consider the World category: these are “sites in languages other than English” where languages actually serve as subcategories of World.²⁴ Also: “If a site’s content is available in more than one language, the site may be listed in more than one language category. For example, if a site is in English, German, and French, the site may be listed in an English-only category, World/Deutsch, and World/Francais”.²⁵ The term ‘English-only category’ refers to the top-level categories other than World and Regional.

Consider the Regional category: this “category lists sites specific to a particular geographic area. The Regional category as a whole organizes sites according to their geographic focus and relevance to a particular regional population. To this end, individual Regional categories become mini-web directories in their own right, while remaining functionally part of the larger Open Directory.”²⁶

There may be multiple occurrences of a URL within the directory. That is, a URL may have more than one associated category.²⁷ However, ODP’s rules seem to stipulate preference of fitting each site into one category, if there is a best match. Special reasons for overlapping apply to the categories World and Regional. That is, World can overlap with the rest of the directory when a site has an English version, and Regional can overlap with the rest of the directory when “sites are relevant to a subject category and a specific local geographic area”.

A.2 Additional Information on Corpus Diversity

Table 22 breaks down all policies in terms of the top-level domain of the policy URL (i.e., the URL found in the policy reference file). The larger part of policies resides in the com domain. The domains gov, org and net are popular, too. The seven first top-level domains cover about 90% of all policies. We show such information here merely as a short indication of the corpus’ diversity, as it was obtained from ODP. This information must not be misunderstood as being part of the study, which is not concerned with P3P adoption by domain, country, or other means of breakdown; see Section 6.

Table 22 Top-level domains in policies URLs

Figure 30 shows the distribution of policies over ODP’s website categories. Nearly half of all policies are (also) regional policies. Figure 31 shows the distribution of policies in terms of the numbers of associated categories; 81.20% of all policies are associated with only one category. Table 23 shows the distribution of policies over subcategories of World—that is, over languages. Please note the category English is not included here because it is not a valid subcategory of World in ODP’s sense. Table also shows number of websites and percentage of policies per subcategory.

Fig. 30

Number of policies per ODP category

Fig. 31

Number of ODP categories per policy

Table 23 Policies by subcategories of ODP’s world category (raw data)

A.3 Additional Information on Disappeared Policies

The corpus of the paper was originally downloaded in Dec 2009–Sep 2010. We verified the availability of the corpus’ policies in Jan 2012. We found that 1,578 policies (out of 6,182) cannot be obtained anymore. Those disappeared policies come from 1,395 different websites. We also checked if the sites themselves still exist; it turned out that 173 sites disappeared. Such information may be useful in making claims about the potential decline of P3P. Such claims are not central though to the contribution of the present paper.

We analyzed the disappeared policies while only considering those policies whose sites still exist. We looked into clone group information and syntactical size. Figure 32 shows the results of the analysis as follows:

• Subfigure (a) shows how disappeared policies are distributed over textual clone groups where the x axis shows the clone group’s cardinality and the y axis shows the percentage of disappeared policies. Large (red) dots show positions of top-ten textual clone groups. We observe that top-ten groups were mostly only slightly affected (losing up to 20% of their clones). There is one top-ten clone group that was eliminated almost completely. This effect can be associated with a particular hosting service.

• Subfigure (b) applies to syntactical clone groups instead of textual ones.

Fig. 32

Disappeared policies: a Textual cloning; b Syntactical cloning

A.4 Additional Information on ‘${{\leq_{\mathit{sem}}}}$’

1.1 A.4.1 ‘${{\leq_{\mathit{sem}}}}$’ for Retention Levels

The value ‘no retention’ (i.e., not storing data at all) implies the most privacy; the value ‘indefinite retention’ (i.e., storing data forever) implies the least privacy. All other values are hard to differentiate in terms of privacy. Hence, we group them between top and bottom.

1.2 A.4.2 ‘${{\leq_{\mathit{sem}}}}$’ for Recipients

The value ‘ours’ (i.e., essentially the corresponding system itself) implies the most privacy if we assume that adding any further recipient decreases privacy. In fact, P3P anticipates recipients that use the ‘same’ policy, and hence we group ‘same’ and ‘ours’ together at the bottom.²⁸ The value ‘public’ clearly implies the least privacy. All the other recipients are hard to differentiate in terms of privacy since we simply do not know anything about their privacy policies. Hence, we group them between top and bottom.

1.3 A.4.3 ‘${{\leq_{\mathit{sem}}}}$’ for Purposes

The value ‘current’ implies the most privacy since it models use of the system for its primary purpose. We treat the values ‘admin’ and ‘develop’ as equal; these are activities internal to the corresponding system. The value ‘tailoring’ further decreases privacy in that the system is adapted for the user based on data from the current session. For instance, the website’s content or design may be adapted. There are several purposes for ‘analysis’ and ‘decision’ that we all consider to maintain less privacy than ‘tailoring’. A purpose with analysis in the name targets at “research, analysis and reporting” whereas a purpose with decision in the name targets at “a decision that directly affects that individual” (W3C 2006).²⁹ We contend that ‘analysis’ implies less exposure than ‘decision’, and the prefix ‘pseudo’ provides implies less exposure than prefix ‘individual’. At the top, we have purposes ‘contact’, ‘historical’, and ‘telemarketing’, as they are typically the least related to the original purpose of the system.

A.5 Additional Information on Semantically Distinct Policies

We are left with 1,385 semantically distinct policies of 4,869 semantically valid policies of a total of 6,182 policies in the corpus. This remaining diversity can be further characterized on the grounds of the partial order for the degree of exposure; see Section 3.4. That is, all the semantically distinct policies combined with ‘${{\leq_{\mathit{sem}}}}$’ define a Hasse diagram with the ‘full privacy’ policy as bottom element. Alas, the diagram is too large for inclusion, but we can discuss it in an abstract manner.

• Number of nodes = 1,385 (= number of semantically distinct policies)

• Number of edges = 2,102

• Number of maximal elements = 983

• Longest chain length = 12

As an illustration, Fig. 33 shows the chains for the policy with the ‘greatest height’ in the Hasse diagram. (This is the policy with the longest chain length = 12; there happens to be only one such policy.) The nodes either refer to clone groups (specified by number and cardinality) or to specific (say, unique) policies. The edges are labeled with the relation between smaller and greater element in compliance with ‘${{\leq_{\mathit{sem}}}}$’. We propose that an inspection of the Hasse diagram, as exercised here, may be helpful in deciding replacements of uncommon policies by common ones. This may be useful for privacy approaches that stipulate a smaller number of ‘common privacy scenarios’.

Fig. 33

The longest partially ordered chain(s) of semantically distinct policies

Figure 34 shows the distribution of longest paths from the bottom element no non-bottom elements. About half of all semantically distinct policies are ‘alone’; they are not approximated by any policies but the bottom element; they do not approximate any policies. One may expect a top element in the Hasse diagram—something like ‘no privacy’ (as opposed to ‘full privacy’). This top element is not exercised and this may be impractical because of P3P’s variable-category data elements. The number of edges is an indicator that many policies serve as joins because the lowest possible number of edges in a partially ordered set with a bottom element equals the number of elements − 1. This means that if uncommon policies are close (in terms of ‘${{\leq_{\mathit{sem}}}}$’) to a common policy, then one could decide to adopt the common policy in favor of the uncommon one. Future work on the diversity of P3P policies is needed.

Fig. 34

Distribution of longest paths for semantically distinct policies

A.6 Additional Information on Diversity of Extensions

To understand the diversity of policies that use extensions other than ‘group-info’, we examined the domains of the underlying websites. All these sites use distinct domains. Table 24 lists frequency of top-level domains. Hence, most cases of non-trivial usage of the extension mechanism come from websites of the Republic of Korea and from commercial websites.

Table 24 Top-level domains of policies using extensions aside ‘group-info’

We further examined the domains. We observed a few cases of shared subdomains: (cii.samsung.co.kr, shi.samsung.co.kr), (cops.usdoj.gov, usdoj.gov), (hoam.samsungfoundation.org, kids.samsungfoundation.org), (ec21.com, ec21.net). Also, we observed the case of a company, Samsung, with several websites using domains without shared subdomains: cii.samsung.co.kr, hoam.samsungfoundation.org, kids.samsungfoundation.org, samsungengineering.co.kr, samsungengineering.com, samsungtechwin.com, sem.samsung.com, shi.samsung.co.kr. Further screening of all domains with top-level domains .kr and .com confirmed that indeed diverse entities appear in those lists. A noticeable number of commercial sites concerned Korean companies. Hence, we found that most non-trivial extension usage concentrates on Korea while exercising various companies and non-commercial entities. At the time of writing, we do not know the reasons for such non-proportional use of extensions in Korea. One possible reason could be a particular, national legislation.