Coming a bit late to the party as I am, I think the other essays on DPI capture most of the issues that I would want to talk about. So I won’t, especially since I agree with most of the essayists on the issues surrounding network neutrality, spying and privacy.

However, there’s one critical aspect missing from all of the other essays which at first surprised me completely. On second thought, perhaps it wasn’t so surprising, because none of the other writers seem to be in the front line of Internet Security with a handle on current and ongoing threats.

This issue is that of malware, spambots, viruses, phishing, trojans, keyloggers, denial of service, malicious downloaders, “DNS attacks” and so on.

On a daily basis we track tens of millions of infected computers (mostly home computers) participating in the sending of billions of email spams per day, resulting in distributed denial of service attacks, identity/credit card/credential theft, money laundering, keystroke logging and so on. Also we see legitimate web sites and other services being “hacked” so as to leave malicious software to drop on the unwary.

Attacks on DNS (the name service that maps the name of where you want to go to its Internet location) is one of the newer and most dangerous threats. You think you’re on your bank’s site managing your account? No, you’re not, you have someone eavesdropping (man-in-the-middle attack via perversion of your DNS lookups) on your conversation, and they will deplete your account shortly thereafter. Encryption (eg: https/SSL) can help, but not always because there are attacks that can subvert that or confuse the user too.

The sheer magnitude of the problem is staggering – and getting worse. This isn’t visible to people not specializing in the field because for the most part organized crime is very good at hiding (some can fool even the experts at times), and ISPs have been struggling to shield their users from it.

Still, it is becoming increasingly dangerous to your bank balance and your privacy to use the Internet. The criminals are getting better at their attacks with new tools and techniques, and network security has to keep pace.

The reality is also that law enforcement’s efforts to catch and prosecute such criminals has been spotty at best, and at least for the medium term, it’s an ineffective weapon for dealing with this. We’re doing our best, and we do have successes, but the overall effects have been minimal so far.

Another unpleasant reality is that anti-virus/spyware packages are becoming increasingly ineffective. Less than 23% of all new infectors are caught by any anti-virus solution, and are seldom useful in preventing current infectors taking hold.

When it really comes down to it, discussions about privacy, network neutrality and the other issues brought up in the other essays here won’t mean anything if users can no longer trust the services they use, nor indeed even their own computers. Even full encryption isn’t a panacea. As more and more people distrust the Internet, the Internet will suffer, and perhaps even die with catastrophic economic consequences.

It’s true that many ISPs are looking into Deep Packet Inspection (DPI) in ways that we may not like (non-network-neutral bandwidth shaping decisions, “phorm-like” marketing intelligence gathering, or even outright “spying” et cetera). Those were possible without DPI and will remain so, whether or not DPI exists.

However, perhaps the biggest incentive for DPI within Internet providers and businesses is the detection and interception of malicious traffic undesirable by any user, and identifying which user has these infections so as to assist them getting the infection removed.

In other words, providers are trying to protect their customers from organized crime attacking them.

DPI can detect when the popular social networking site you just visited had been hacked and tried to download a virus onto your computer, or when an email sent to you contains something malicious and stop it. It can detect when the virus activates and tries to operate. It can detect where the attacks originate from. And so on.

DPI can be misused. So can a hammer. We don’t ban hammers. We do ban the bad things you can do with a hammer.

We need to consider DPI as just another tool. DPI is a very powerful one that can be miss-used, but it’s still just a tool.

Rather than talk about DPI in terms of the things we don’t want DPI to do, we as a society have to decide what things we do/don’t want done, regardless of what technology is used to do it. If we want network neutrality, than that’s what we should regulate, not a particular tool that may or may not be used for it.

Professor Larry Lessig writes in Code and Other Laws of Cyberspace[1] that four forces regulate behaviour on-line: code, law, norms and markets. In the case of deep-packet inspection and other forms of Internet surveillance, code is currently no impediment at all. Most Internet communications take place in “plain text” — unencrypted data that is as easy to read as a postcard sent through the postal system. These unprotected data packets are passed through dozens of computers, any of which could peer into its contents. Deep-packet inspection is merely a matter of one machine’s diverting this flood of data, and doing what computers do best: analyzing their contents.

A single line of code, run on a standard PC running Linux or MacOS with generally-available software, can conduct “deep packet inspection” across everyone communicating over your local network, and search for a keyword in all users’ communications.

# tcpdump -A -s0 -i eth0 | grep privacy

Can existing law defend users’ privacy? Many national laws provide strong protections for the privacy of communications — but in a world of plain-text traffic, enforcement of such laws is a constant challenge.

It’s also a constant temptation to stretch, bend, or circumvent these rules. Apart from encrypted traffic, surveillance on the present Internet is not only easy, but nigh undetectable. Reading email and web traffic requires no steamed-open envelopes. Often, the inspection of Internet traffic can be revealed only by human whistle-blowers like Mark Klein, a retired AT&T employee who provided details of a secret surveillance system installed in the telephone company’s facilities in San Francisco.[2]

Markets can provide incentives to protect customer privacy — but can also incentivize prying. Many ISPs are now mulling the financial benefits that might come from various applications of deep packet inspection to their own customers’ communications. Companies like Phorm[3] in the United Kingdom have proposed that ISPs scan the private traffic of their users to create marketing “profiles”, which can then be used to more precisely target advertising to them. Naturally, the more information that is collated on an Internet user, the more valuable that data is.

In practice, a remarkable part of the burden of discouraging mass surveillance online relies on ISPs’ internal cultural norms. Because the techniques are so simple, the data so valuable, and the extent of the privacy violations unbounded, intermediaries themselves are forced to impose a bright line themselves to avoid the temptation to investigate every packet that passes through them.

Unwritten norms like this are most effective when human oversight exists in Internet surveillance. The more customers and ISPs know, the more reticent they are to conduct or condone such behaviour.

Ironically, the aspect of deep-packet-inspection that reassures many may also embody its profoundest risk. In the case of Phorm’s ad targeting, dragnet government surveillance, and automated ISP filtering for particular content, the argument is often made that the surveillance is acceptable because “no humans see the intercepted data” – that it’s just a machine watching.

It may be easier to feel uneasy about a human being looking over one’s shoulder than an appliance in a remote server room crunching out statistics. But to the extent that humans are taken out of the loop, it is harder detect or report abuses, and harder still to resist “mission creep”. Without careful oversight, the subtlest and most apparently reasonable deep packet inspection can turn into a tool for widespread privacy violation with just a few more lines of code. The packets are there; the data is present; the machines are flexible. After all, if we spy on all data for intellectual property infringement, should we not inspect all private data for potential terrorist attacks, a far more pressing social threat? And if our automatic IP filters work so well without human intervention, perhaps we are happy to run our “bad politics” filters with a similar lack of oversight?

Much of what has protected our privacies online thus far is the ISP world’s thin cultural norm that your private communications really are private to you and those you address. If deep packet inspection replaces ISPs’ bright line of ignoring the data passing under their eyes, the Internet may truly become lawless; with ineffective privacy laws, a culture within intermediaries of consequence-free surveillance, and an emergent new marketplace of private communications, sold to the highest bidder.

[1] http://www.code-is-law.org/
[2] http://www.eff.org/issues/nsa-spying
[3] http://en.wikipedia.org/wiki/Phorm

“Neither rain, nor snow, nor heat, nor gloom of night stays these couriers from the swift completion of their appointed rounds.” So wrote Herodotus of the fifth-century-BCE packet delivery service used by Xerxes, king of the Persians. This famous passage is inscribed on the general post office in New York City. Less familiar are the next words of the text: “The first courier transfers the message to the second, the second to the third, and thence it passes from one to the next.”

Though the technologies have changed, the principles have not. Break the delivery chain into segments; provide fast service on each link; make your best effort to complete the handoff at each stage; and don’t try to do anything else with the message except to deliver it.

In the Internet, “deep packet inspection” (DPI) is usually described as the practice by Internet Service Providers (ISPs) of looking at the contents of packets, not just their addresses, before deciding how to deliver them. In fact, DPI is more than that: “inspection” is a euphemism. As actually used, DPI may involve introducing forged packets into the data stream—packets apparently created by a sender, but in fact created by the ISP to alter the recipient’s experience. Comcast used this method to “manage” communications by slowing certain data streams (mostly video), and drew a stinging rebuke from the U.S. Federal Communications Commission.

Some ISPs consider DPI to be a useful tool in their quest to provide high-quality service and rational allocation of limited bandwidth. In their view, regulation of DPI would hobble innovation in their business practices. Some have even suggested that anti-DPI legislation would be a precedent for government regulation of Internet speech itself.

In fact, DPI should be banned for two reasons. The first is privacy. DPI violates the universal expectation that delivery services won’t read the messages they are delivering. Second is “generativity,” to use the term coined by Harvard Law School professor Jonathan Zittrain to describe technologies on which users can build in unanticipated ways. Reliability of the delivery service is the mother of creativity at the endpoints.

Privacy first. Users do not expect service providers to examine packets en route, any more than they expect the phone company to decide by listening in whether a call merits a high-quality line. The Internet by design connects peers to peers. For example, “distributors” and “consumers” of movies streamed over the Internet are architecturally on an equal footing with email in and out of African Internet cafes. The real threat of censorship comes not from government guarantees of content neutrality, but from carriers discriminating on the basis of content, source, and destination—probably in favor of the powerful and against the weak. It has happened before, as when Western Union cut a deal with the Associated Press in 1867 to exclude other news services from its telegraph wires, and when Verizon denied a pro-choice group access to text messaging in 2007 on the basis that its agenda was “controversial or unsavory.”

Analysis of packet protocols (“he’s been downloading a lot of video lately”) and origins (“those videos are from YouTube, not Comcast”) is intrusive. Indeed, the presumption of privacy, and of neutral treatment of all data types and sources, is so strong that DPI might be self-defeating. Were it widely known that ISPs could lawfully exploit information they glean from peeking inside packets, Internet users might encrypt their communications to defeat the ISPs’ payload analysis.

Generativity second. As Internet pioneer David Reed explained to the U.S. Congress, creative software engineers at the edge of the network gave us countless useful applications for which the Internet was not designed. Internet telephone protocols, for example, changed the international phone call from an expensive luxury into a routine part of millions of daily lives. Such creativity will continue into the future only if the functioning of the core of the Internet remains documented, consistent, and predictable.

The market won’t sort out this conflict because necessary competitive conditions don’t exist. When many areas have only one choice for broadband services, and few have more than two, service providers find it more profitable to sustain and manage scarcity than to build toward reducing it.

The Internet is a public good owned by private businesses, which enjoy monopoly or duopoly powers almost everywhere. Though any regulation must judiciously avoid hobbling future technological innovation, broad legal guarantees of the Internet’s secure and transparent operation will serve the public interest.

The end-to-end principle for the Internet, where the intelligence is at the edges of the network, not within its core infrastructure, is supported by three types of arguments:

• Technical Simplicity: Because of the layered protocol stack, the sub-networks are only connected through the TCP/IP protocol suite and a shared address space. Therefore, they are highly open to new transportation methods as well as new applications.
• Political Freedom: Because the payloads at the application layer are encapsulated for the lower transport layers, the users have uncensored and uncontrolled end-to-end communication channels.
• Economic Openness: Because of the openness for new applications, the Internet does not discriminate traffic based on its source, therefore treating all innovations equally and giving them a fair chance to succeed at the market.

Lawrence Lessig in his 1999 book “Code and other Laws of Cyberspace” used a nice illustration for the end-to-end model: “Like a daydreaming postal worker, the network simply moves the data and leaves interpretation of the data to the applications at either end.” Now, imagine a postal worker who is not just daydreaming and moving packets from one point to another in the transportation chain. Imagine the postal worker opens up all packets and letters; inspects and even reads the content; checks it against databases of illegal material and if finding a match, sends a copy to the police authorities; destroys letters he finds having prohibited or immoral content; sends packets with content from those mail-order companies which pay extra to the postal service to a special and very fast delivery truck, while the ones from the competitors go to an extra-slow and cheap sub-contractor. Such a postal system would infringe on the values embodied by the internet as described above:

• Political Freedom: The postal system would now invade the privacy of communications and introduce censorship, potentially leading to “lost” letters from trade unions or political dissidents.
• Technical Simplicity: Such an inspection system would create an additional overhead that would slow down postal delivery and place a significant responsibility on the postal worker. The letters and packets would also be damaged when being opened. And, most importantly, the postal service would assume functions it never was founded for.
• Economic Openness: The differential treatment of content from different senders and companies basically means blackmailing content companies like mail-order stores into signing additional and costly high-speed contracts. New business models that solely rely on innovative content being delivered through the normal postal system would have to negotiate specialized fees with the postal service for their products.

Now, imagine a postal worker could all do this without significant delays compared to his (former, now fired) daydreaming colleague. This is what deep packet inspection technology is designed for.

Many of the functions provided by DPI have been available before. Internet traffic could be intercepted and logged with tools like TCPDump or Wireshark, copyright was enforced with digital rights management (DRM) and watermarks, scarce bandwidth was prioritized by the TCP congestion management and quality of service protocols, user behaviour was tracked and used for advertising with cookies, and so on. The potentially paradigm-changing characteristic of DPI is the fact that it integrates these diverse functions into one hard-coded and extremely fast piece of equipment. It thereby also integrates the interests of a diverse set of actors, who all have their distinct ideas of how to use DPI:

• government agencies and content providers, who are interested in the monitoring and filtering of information flows (political control)
• network operating staff, who have to deal with more malware and bandwidth-hungry applications than ever before and who often have limitations for expanding bandwidth on the last mile (technological efficiency),
• vertically integrated ISPs that want to create additional revenues or protect them, e.g. through preventing the internet from cannibalizing their telephone- or video-on-demand revenues (economic interests).

DPI thus has the potential to change the nature of the internet, by making it a less open network, by introducing means for political control, and by stifling economic openness. But a potential does not necessarily, and rarely fully, translate into reality. DPI usage does not have to implement all the above functions of the highly awake postal worker. Some use-cases of DPI already seem to be disappearing. They do so for different reasons:

• Market Reactions: NebuAd has ended its behaviour-based marketing activities because of the public outcry, and UK ad injection provider Phorm may undergo the same fate. The ISPs are publicly fleeing from this model for extra revenue before their customers flee from them.
• Legislation: The European Parliament has voted against demands of the music and film industry, which was pushing for mandatory copyright filtering provisions. This happened mainly because of an intensive publicity campaign by internet users’ rights groups.
• Regulatory Action: ISPs Comcast in the US and Rogers in Canada have undergone scrutiny by regulatory and privacy authorities because they throttled some of their users’ traffic based on what seemed appropriate and what not.
• Technological Circumvention: A growing number of filesharing and other programs now allow for encrypting their traffic, which makes DPI-based copyright filtering impossible.

An important factor in all these cases is awareness and transparency. The market as well as technology vendors and public bodies reacted only after privacy advocates, bloggers and consumer protection groups had published how DPI works and what it does to the users’ privacy and the idea of an open Internet. As long as DPI vendors can successfully hide under ambiguous terms like “intelligent network” or “network management”, the dangerous potential of DPI will not be under enough public scrutiny.

It may well be that there is a sustainable and legitimate market for DPI technology, but with a much smaller set of use-cases. These will probably include corporate firewalls and malware filters, and potentially differentiated internet access pricing models and behavioural advertising – if this is done very openly and on an opt-in basis.

In the end, DPI teaches us again that while engineers invent powerful technologies, it is society and its norms, rules, and institutions that define if and how these technologies should and will be used. Any technology use-case that violates fundamental rights and user expectations is doomed to die. This does not happen automatically, of course. But the internet users’ rights groups have become a powerful force, and if they are supported by fundamental beliefs and basic rights of society, there is not much to do against them.

Appendix: Previous Technologies and DPI – Use Cases and Drivers

Political Control

Technological Efficiency

Economic Interests

Final Version of 11 March 2009
Commissioned as a contribution to a publication by the Canadian Privacy Commissioner
Roger Clarke

© Xamax Consultancy Pty Ltd, 2008-09

The job of an intermediary node on the Internet is to pass each packet on to another node closer to the addressee. Deep packet inspection involves an intermediary node also poking its nose inside the packet.

Inspection of the header, and even the contents of the message, may be consensual. Even if it is not consensual, it may be beneficial to all parties. However, the proliferation of uncontrolled, non-consensual access is currently threatening to underminethe open, public Internet as it has been known for its first 15 years of operation.

Worse, these intrusions bring with them the threat that communications over the Internet may become much less free than the communications channels that residents of relatively free nations used in the pre-Internet era.

The term ‘deep packet inspection’ refers to a technique that is being imposed on data communications networks in order to probe into the contents of passing traffic. This short paper commences with some background, provides an overview of the technique, and undertakes a brief analysis of its implications.

The term ‘packet’ is ambiguous, and there are advantages in avoiding it. This paper uses the more straightforward term ‘message’ to refer to that which passes from a sender to a recipient.

The Internet comprises a very large number of nodes, each of which is a computer capable of performing a wide range of functions. Messages are created in a ‘sending node’, and addressed to a ‘receiving node’. In order to get from sender to recipient, messages pass through many other nodes, which are usefully referred to as ‘intermediary nodes’. The number of intermediary nodes that messages pass through is typically about 20. A large message is broken into as many parts as necessary in order to comply with the maximum message-size that intermediary nodes along the way are prepared to handle.

The task of an intermediary node is to compute the next node to pass each message on to, in order to either deliver it to the intended recipient, or get it one step closer. The notion of ‘deep packet inspection’ involves an intermediary node doing more than that. In order to analyse the technique’s implications, it is necessary to understand a little about the layers of processing involved in data transmission.

Raw media (such as cable and radio waves) require considerable electronic engineering expertise, infrastructure, hardware and software to make them useful for the transmission of data. That expertise is embodied in ‘protocols’ (rules of engagement) that are implemented in software in the sending, intermediary and receiving nodes.

Interpreting the binary digits that are transmitted on those media requires a further and different kind of expertise. Another layer of software performs this functions. It implements a further set of protocols, and depends on sender and recipient addresses and other administrative data being stored in headers that are added to the underlying message content.

Shifting the groups of bits from one node tothe next requires a different kind of expertise again, a number of protocols, software packages in each node, and an extra header added onto the message. De-constructing large messages into small ones and re-constructing them back into the original message requires another layer of expertise, protocols, software and header. And conveying the semantics of the message requires yet another of each.

In short, transmitting a message from a sending node via intermediary nodes to a receiving node involves a stack of protocols, software and headers. The protocol stack is roughly modelled by the first of the following diagrams, and the headers by the second.

Exhibit 1: The Protocol Stack in Operation

Exhibit 2: The Message and the Accumulation of Headers

Intermediary nodes run a number of software packages to perform the various functions at each level of the protocol stack. The best-known term for such software is ‘router’. Used correctly, this refers to the software operating at the middle level of the stack, which handles the Internet Protocol (IP). Router software depends on lower-level software (switches and hubs).

The term ‘router’ is often used in misleading ways, however. It may refer to all of the layers of software combined, rather than just one layer. And often it refers to the device (the ‘intermediary node’) rather than just the software.

Software in an intermediary node, in performing its function as a way-station passing messages from a sender to a recipient, only needs to look at the header associated with the relevant protocol. It has no intrinsic need to look at the deep-nested headers associated with higher-level protocols, let alone at the data deep inside the message. So a well-behaved intermediary node does what it needs to do in order to pass messages on, and nothing more. In terms of Exhibit 1, that work is performed in the Network Layer, by the software called a router.

There are a number of circumstances in which an intermediary node can perform additional functions, as an agent of the sender or recipient. A general term for such software is a ‘proxy-server’.

A recipient may use software on their own machine to scan incoming email, evaluate the headers and content in order to assess the likelihood that it is spam, and flag (or, more riskily, delete) messages whose spam-score exceeds some threshhold. Similarly, a recipient may use software on their own machine to scan the content of web-pages they have requested, and possibly block display of the page if the scan detects content that is undesirable in some way. A third example is commonly referred to as a ‘firewall’. A firewall detects messages that are being directed at processes within the user’s machine that are not expecting to receive such messages.

Rather than having such functions performed on their own device, a recipient may request an intermediary to provide ‘spam-filtering’, ‘web-page filtering’ or ‘firewall’ services. Such services may be offered by companies that provide consumers with connections to the Internet (which are often referred to as Internet Service Providers – ISPs). Where the consumer actively requests it, or provides informed and free consent to it, such services are positive and worthwhile enhancements to basic Internet infrastructure.

The previous examples all involved a message recipient. Circumstances also arise in which the sender may take advantage of additional services from an intermediary node. In particular, a proxy-server may send a message on behalf of the real sender, or manage a session of multiple messages between the sender and a remote server.

One example is called by the obscure name ‘reverse-proxy’. For example, a person who is currently away from their normal place of work (e.g. on a client’s site, in a hotel or at home) can be made to appear to a remote server as though they were at work.This service is commonly offered by university libraries to academics, enabling them to access publications databases that the library subscribes to, and to do so from anywhere in the world.

Another purpose to which proxy-servers are put is to obscure the sender’s network location (their ‘IP-Address’). Such services are commonly referred to as anonymous remailers and tools for anonymous web-surfing. They may offer anonymity. Alternatively,where an investigator has the technical capability and the legal authority to access relevant look-up tables, they offer pseudonymity rather than unbreakable anonymity. Such services are valuable, and arguably essential, for ‘people with something to hide’, such as whistle-blowers, protected witnesses, victims of domestic violence, celebrities, notorieties, and people in security-sensitive occupations, including undercover operatives and spies.

In order to perform these services, the software running on the intermediary node has to read the message content, or at least the deepest-nested ‘application headers’; hence the term ‘deep packet inspection’.

There are further circumstances in which an intermediary node can perform additional functions which are generally beneficial to all participants.

An intermediary node performs a function as a ‘gateway’ if it operates a transition facility between the Internet and some other network. For example, one participant in a telephone call may be using VOIP (voice over IP) but the other may be on the conventional Public Switched Telephone Network (PSTN, sometimes referred to as a landline), or on a cellular network (i.e. using a mobile phone). A gateway performs for messages much the same function as an intermodal terminus does for cargo – lifting containers on and off trucks, trains and ships.

Another example is ‘network cache’. Many web-pages are requested by multiple web-browser users in a short period of time. An intermediary node can save everyone time and money by storing (‘caching’) the page for a while after the first request. This avoids having to unnecessarily fetch the same content a second time from a distant server.

To perform these services, however, gateway and network cache software have to read both the ‘application headers’, at the deepest level of the message, and the message content itself. This represents an intrusion inside the message envelope. Such behaviour may be justifiable on the grounds of efficiency, or perhaps implied consent. But care is needed, because the person whose message is being handled may not be aware of the activity, and may perceive problems that the operator of the intermediary node does not.

Some intermediary nodes contain software that reads deep-nested headers and even content, without the consent of the parties to the message, and for purposes that are not consistent with the interests of the parties. There are several categories, each ofwhich has potentially serious negative implications for the parties, and for society as a whole.

An intermediary node may access the content of the message and either use it for the purposes of the interceptor, or disclose it to some other party. One example of this is software that detects and accumulates email-addresses – for use by spammers. Similarly, software may ‘sniff out’ credit-card details sent in email messages and typed into web-forms
– for use in financial fraud.

Another example is message-monitoring by law enforcement agencies. In many jurisdictions, such monitoring is subject to judicial warrants and tight controls, but in others (including nominally free countries such as the UK, the USA and Australia) those independent authorisations and controls have been subverted, using terrorism as the excuse. As a result, a considerable amount of message-interception is being conducted in the absence of demonstrated and reasonable grounds for suspicion of criminal behaviour.

A further possibility is adaptation of the message and onforwarding of something that purports to have originated with the sender, but did not. This creates further possibilities for fraud, and for the ‘planting’ of evidence.

Another form of intrusion is masquerade by the intermediary node as though it were the recipient, and provision of a falsified response. This is understood to have been the mechanism whereby the People’s Republic of China (PRC) has returned (and continues to return?) false responses to searches submitted to remote search-engines, and fake ‘not found’ messages in response to requests for web-pages blocked by the regime.

Yet another example of intrusion is the blocking ofmessages by an intermediary node on the basis that some aspect of the header information or of the message itself is deemed to offend some rule imposed by the party that operates the node. This is commonly the case in un-free regimes such as Burma, the PRC and Iran. But it is also the mechanism proposed by nominally free nations that are adopting a ‘nanny state’ role and seeking to censor such content as on-line gambling, pornography (however defined) and dissident political speech (however defined). See Dedman & Sullivan (2008) and ONI (2008).

Singapore was an early mover among economically advanced nations. But currently, governments in the USA and Australia are trying to impose much the same repressive measures. Such interference represents concrete steps towards the authoritarian future presaged in Clarke (2001).

The term ‘deep packet inspection’ refers to access by software running in an intermediary node to header data, and even the message-content, that the node does not need to access in order to perform its inherent function of passing messages on, along their journey from sender to recipient.

Deep packet inspection may be performed at the request, or with the consent, of a party to the message. This is an enhancement to fundamental Internet infrastructure.

Deep packet inspection may be performed without the consent of the parties to the message, but in such a manner that all parties benefit. Primary examples are enhanced response-time and the avoidance of unnecessary transmission of large files, through ‘network caching’. This is more problematical than consensual access, because some party is making the judgement that the intrusion is beneficial to all parties.

Finally, and of far more serious concern, deep packet inspection may be performed not only without the authority of the sender and recipient, but also for purposes that are, or at least may be, against the interests of some of the parties. This requires strong justification, tight controls, and enforcement mechanisms. Unfortunately, these are seriously lacking, and both Internet Service Providers and government agencies in many countries (both nominally authoritarian and nominally free) are abusing and undermining Internet infrastructure in the process.

Imagine the postal service steaming open your letters so that they could scan the content, work out your interests, and then deliver a better class of junk mail. Most people would be horrified, yet some of the UK’s largest ISPs are planning to do something even more intrusive. They will capture the details of all the online searches you make, all of the web pages you visit – solely to serve up targeted online adverts. This isn’t happening for some altruistic aim of making adverts more relevant, but because the ISPs will get a cut from the advertising revenue, and Phorm, the technology vendor involved, will charge advertisers extra for delivering up an especially receptive audience.

You might think that “there ought to be a law against it” – and you’d be right. Analysis by the Foundation for Information Policy Research (FIPR) shows that the complicated way in which the Phorm system works means that the ISPs will commit criminal offences, and could also face civil litigation for the unauthorised processing of copyrighted material.

The Phorm system snoops on all web page requests, and in particular it picks out the search terms used on Google and other search engines. The system also monitors the contents of any web pages visited, looks for the commonest words, and tries to discern what the pages are about. This works up to a point – early search engines used similar schemes – but isn’t especially accurate. Accurate or not, a distillation of this information is matched against advertiser word lists, for example, if “flight” and “hotel” appear, then perhaps you’ll be a sucker for a travel advert. If so, then when you next visit a participating website, the adverts won’t be random but will have a travel theme to them – with the highest bidder getting to put their message in front of you, and the ISP getting a back-hander for participating.

However, UK criminal law calls snooping on web traffic “interception” and can send you to prison for it. There are statutory defences for the ISP (or indeed the postal service) looking at traffic for operational purposes (so your mailman can look at the address on the envelope), but this is irrelevant because it isn’t an ISP operational matter to deduce whether or not you’re a travel junkie.

The ISPs involved with Phorm will obtain the permission of their customers to be snooped upon (albeit this permission is rather an afterthought, and early trials didn’t bother with such niceties). Unfortunately for the ISPs, in the UK this is necessary but not sufficient, because interception is illegal unless BOTH ends of the communication give permission. This is a fundamental (and clearly intentional) change made by Parliament in 2000 from the previous one-sided regime. What’s more, the 2002 EU “Directive on Privacy and Electronic Communications” also makes it clear that both ends’ permission is needed.

As it happens, the two-sided requirement gave the legislators several headaches, and so there are special provisions to permit the police to listen in to a kidnapper’s ransom demand and secondary legislation sets out “Lawful Business Practice” to permit stockbrokers to record their instructions, and call centres to perform quality monitoring. None of what the ISPs intend will come under Lawful Business Practice.

Readers may be surprised to have got this far without any mention of the UK’s Data Protection Act 1998 (DPA). It is relevant, in that the Phorm system will regularly be processing “sensitive” personal data and must therefore arrange for an informed opt-in. However, not much more of the DPA will apply because Phorm has carefully designed its systems to evade the provisions of the Act – and providing pseudonyms for users in the form of unique identifiers gets them an awfully long way.

But the real reason the DPA is scarcely relevant is that people’s outrage at the system is expressed in the language of privacy, and there is a significant difference between “privacy” and “data protection”.

When the taxman looks at your financial affairs, they trample all over your privacy, but their systems are completely DPA compliant. Likewise, the Phorm system may learn that someone they know of by an opaque identifier is fascinated by the prospect of travelling to Israel, and they will stay with the letter of the DPA law. However, they’ve learnt something very private about that user’s opinions. If they were a Saudi Arabian student studying in the UK, subsequent serving of targeted adverts, and the information thereby revealed, could lead to embarrassment or much worse.

The bottom line for me, when I consider the Phorm system, is that having ISPs snoop into the personal lives of their customers for a trivial financial gain is inherently objectionable. It is simply not what ISPs should be doing. That the system turns out to infringe a number of laws should simplify blocking its deployment; it’s not the reason that it has to be stopped.

The idea that general purpose communications networks should be subject to special obligations, and that those obligations are understood to benefit the rest of society, has a long and distinguished history. For more than a hundred years, policymakers subjected these networks – think post, telegraph, and telephone – to obligations not to discriminate among communications and to keep their customers’ information private . (I call these networks “general purpose” to distinguish them from networks dedicated to one-to-many broadcasts, like television, cable, and satellite.)

The successor general purpose network is Internet access. It’s replacing the telephone and the post. Just as Western Union finally sent its last telegraph in February 2006, these older general purpose networks will become extinct someday. This won’t happen for a while; the pace of telecommunications modality extinction is glacial. But no one can deny that Internet access is now essential to modern communications.

Somehow we’ve forgotten the close traditional relationship between basic communications and the functions of the state itself. The key reason that basic communications (and basic transport) were subject to nondiscrimination and privacy obligations was that these pieces of social infrastructure were closely associated with sovereigns. True, states may initially have gotten involved with transport and communications networks (even if the state was not providing the network itself) to ensure that the state’s communications and vehicles could move smoothly and swiftly across its territory in the service of national security and law enforcement interests. After this self-protective priority was ensured, a second role of the state – ensuring equal access to essential physical utilities and services and making sure that users’ information was treated with respect – became operative.

Over the last five years or so, this basic set of social requirements for general purpose U.S. networks has been thrown overboard. Through definitional legerdemain and a certain amount of judicial gullibility, we’ve ended up treating Internet access as if it was a Broadway show: privately controlled, content-driven, and subject to no particular social demands. And we have very few of these shows running at this point; most people have few choices of providers, prices are high, speeds are slow, and Internet access is inseparably bundled with several other “services.”

One important element of social policy that has been jettisoned along the way concerns the treatment of user data. In the telephone world, Section 222 of the Communications Act prohibited carriers from using consumer information for marketing purposes. Period. Now those same carriers are providing Internet access, and with the FCC’s help they have freed themselves of the strictures of Section 222. They can plumb the depths of packets, use the resulting data to target advertising, copy all data and shunt it off to other companies, prioritize streams of traffic based on what users are doing, and pull whatever stunts they feel like in terms of DNS redirection.

Thus, the two central social obligations that we used to impose on general purpose network providers – nondiscrimination and confidential treatment of user data – have been completely undermined by the private, highly concentrated operators of Internet access.

Network operators are taking the view that disclosure of their practices will address and resolve any possible consumer protection issues. They’re saying that as long as a consumer has been told what is going on, all is well. They’re also saying that they are doing the same kinds of things that free Web applications (like Yahoo! and Google) have been doing for years.

Every essay of this sort can make only one point, and here is the point of this piece: Transport is not the same thing as the vehicle using that transport. The providers of Internet access should be treated like the basic, general purpose actors they are. In particular, they should not be permitted to use subscriber data for their own business purposes. Acting otherwise confounds consumer expectations and runs counter to more than a hundred years of basic communications understandings. Remember, this is fundamentally the role of the state we’re talking about. Add in the crucial role of general purpose networks for economic growth and innovation, and you have some powerful arguments against network level deep packet inspection.

The idea of separation between transport and “other” is taken quite seriously in other corners of the world. For example, Singapore, the city of Amsterdam, and the city of Stockholm have all required fiber networks to be architected along passive, open access lines. Any company can come and install electronics in those fibers, and competition is fierce. The European Commission’s Information Society and Media department, led by Commissioner Viviane Reding, has recently released a paper calling for this kind of open access approach to Internet connectivity. Separation and non-discrimination both militate against allowing deep packet inspection by network providers.

Network providers would like us all to muddle along in the weeds of disclosure details, but DPI presents a much more fundamental issue: What should the providers of general purpose network access be permitted to do as a social and economic policy matter? For me, the answer is clear. They should be required to stick to the business of transport.

In recent years a controversy has erupted in Canada, the United States and other parts of the industrialized world regarding the provision of Internet services. The controversy centers on the relationship between the entities that provide connectivity to the Internet (ISPs) and the traffic that flows their networks. A long-standing principle of the Internet’s architecture — known as “network neutrality” — says ISPs should not discriminate on the basis of the content that flows through their pipes. And yet today, ostensibly for reasons of efficiency and cost, that is precisely what many ISPs are doing. The practice, known as Deep Packet Inspection (DPI), involves network managers of ISPs developing procedures that track, inspect, and re-route or delay traffic based on the type of protocol being employed or the content of the communication being transmitted. Like many others, I believe that if DPI is adopted as the Internet’s norm, it will undermine the Internet’s foundational architecture and much of its novel and beneficial effects, threaten freedom of speech, access to information, and privacy online, and further carve up and degrade a valuable global commons.

While the controversy has bubbled up in North American and Europe, DPI is, in fact, widely practiced around the world, and an examination of some of the ways it is employed elsewhere may give us a glimpse of the future here. For the last six years, working with colleagues at Harvard, Cambridge, and Oxford Universities plus partners worldwide, I have helped marshal a talented pool of researchers, organized under the OpenNet Initiative (ONI) and Information Warfare Monitor (IWM) projects, to lift the lid on the Internet and document what goes on “beneath the surface.” For most people, the Internet’s infrastructure is largely invisible; the user’s experience begins and ends with the terminal that sits in front of them. However, it is deep within the subterranean realms of the Internet’s infrastructure – through the fibre optic cables, long haul lines, satellite uplinks, routers, and Internet exchanges — that power is increasingly exercised. Fortunately, as the Internet is an open public network, those with the knowledge and skills are able to interrogate it directly and uncover and expose these types of practices.

According to the latest findings of the ONI, more than two dozen countries now engage in some kind of Internet content filtering in which ISPs act as the frontline defense against content deemed politically, socially or strategically threatening. As evidence of mounting problems, we are presently testing for Internet censorship in 71 countries. Presumably dozens more engage in surveillance for the same reasons, although far less is known and documented about those practices. In countries where the rule of law is not regularly respected, and free speech and access to information is rare, widely cherished norms concerning “network neutrality” have little basis in reality. In China, Burma, Vietnam, Tunisia, Saudi Arabia, Yemen, Ethiopia, UAE, Syria, Pakistan, Iran, and Uzbekistan, to name a few of the worst offenders, governments routinely order ISPs to engage in DPI to block access to the websites of political opposition movements and human rights groups. In some of the most egregious cases, like Kyrgyzstan and Belarus, we have documented ISPs secretly disabling access to opposition websites leading up to and during election periods, and then restoring normal Internet connectivity afterwards — a phenomenon we have dubbed “just in time” filtering. Most of the ISP’s DPI practices take place without oversight or public accountability, and so errors, malicious redirects, and collateral blocking are legion. So is a phenomenon we call “mission creep”: once the practice of filtering has been enabled for whatever reason, the temptation to use it for a wide variety of other social and political problems is enormous. For example, Pakistan started out filtering access to satirical images and videos of the Prophet Muhammed; it now also blocks access to any websites related to the troublesome domestic Baluchistan insurgency.

To be sure, Canada is not Belarus, China, or Pakistan. And, of course, ISPs here claim they are engaged in DPI for narrow reasons of bandwidth control, and not for political reasons. Can we trust them? Recent research from the IWM should raise concerns. As detailed in our report, called Breaching Trust, our researcher Nart Villeneuve discovered that the Chinese version of Skype was not only filtering keywords on the instant messaging client, it was systematically uploading the messages containing the keywords to insecure servers in China. We were able to access, view, and download millions of messages containing sensitive political and economic information ostensibly collected at the behest of Chinese public security organizations. Many people suspected there was a “backdoor” in Skype and that the Chinese version was a Trojan horse for Chinese intelligence; the company publicly denied these worries in 2006. Our research proved they were wrong.

Even more instructive is our August 2005 ONI bulletin, which found that the Canadian ISP, Telus, was blocking subscribers’ access to a website set up by an employee labor union. Our research at the time showed that not only was Telus blocking access to the pro-union website, but it was collaterally filtering 766 additional, unrelated websites. Although our report and other observers questioned whether Telus violated CRTC regulations in blocking access to the pro-union website, Telus responded by saying that under contractual agreements with its customers, it has the right to block access to certain sites, such as those containing child pornography. No mention was made of the collateral filtering we discovered and as far as we know, Telus was not disciplined in any manner by the CRTC.

Once the norm against network neutrality is breached for whatever reasons, the relationship between Internet intermediaries and the communications they facilitate fundamentally changes, and with it the character of the Internet itself. The research of the ONI and IWM suggests strongly that pressures around mission creep mount, collateral blocking explodes, and the enforcement of public security is delegated to often unaccountable and mendacious private entities. Is that the Internet we want?

Phorm is an innovative digital technology company focused on creating a more relevant Internet experience for users and more value for advertisers, publishers, Internet Service Providers (ISPs) and others in the online ecosystem. Produced after years of customer research and technological development, Phorm’s proprietary, patent-pending technology offers a paradigm shift in both audience segmenting techniques and online user data privacy.

In February 2008, in conjunction with its UK ISP partners BT, Virgin Media and TalkTalk, Phorm launched the Open Internet Exchange (OIX), an online behavioral advertising platform designed to protect user privacy and anonymity.  Unlike other online advertising models popular today, Phorm’s OIX is revolutionary in that it provides direct benefit not just for publishers and advertisers but also for consumers and ISPs without using Personally Identifiable Information (PII) or storing specific browsing information.  While other advertising technologies routinely store data such as search terms, IP addresses, login or account details or other information which could be used to derive identity, Phorm stores none of these.  In fact, Phorm’s OIX technology records only an anonymous cookie containing a randomly generated user ID and a time stamp in conjunction with a pre-existing interest category.

While the cookie logged to this category or “channel” is completely anonymous, Phorm’s privacy protection does not stop there.  The OIX system is built in such a way that there must be multiple triggers for user inclusion in any channel.  The result is that end user inclusion in a channel cannot be used to determine which specific event or “trigger” caused membership.  OIX cannot determine this because the specific trigger is not recorded – only the membership in the larger channel.  This new approach makes it impossible to look at any stored data and to know where a user has browsed on the Internet or what a user searched.
Phorm’s privacy controls work by assigning an anonymous cookie to each consenting customer within a participating ISP.  This cookie is in no way related or linked to ISPs’ authentication systems or technology within the ISP that would allow the ID to be made identifiable.  Indeed, this cookie is not accessible outside of the OIX system and therefore cannot be linked to external data sources, a problem common to other technologies.

Through partnerships with ISPs the OIX technology is able to determine when a specific cookie has triggered channel membership and assign the anonymous cookie to the appropriate channel.  The system is designed so as not to follow an anonymous browser as it traverses certain “sensitive” areas.  To avoid encountering potentially identifiable or sensitive information, OIX specifically excludes secure sites and pages (https), non-web traffic (such as email, FTP or VoIP), popular web-based email systems and form submissions.   To further avoid potential privacy concerns the OIX technology does not allow targeting or the delivery of ads based upon certain sensitive categories such as adult content, sensitive medical information or alcohol/drug interest.  The OIX does not look at numeric content over 3 digits in length which could contain personal information, and is designed to exclude proper names.

In addition, Phorm also has instituted procedural controls and human oversight to prevent the creation of any channel which could inadvertently target or collect information specific to identified individuals.  All this is done to prevent even the inadvertent ability for Phorm, our ISP partners or any third party to ever be able to connect even the limited data Phorm stores (anonymous cookie, channel and timestamp) to an identified person.  Phorm believes the first tenet of data security is data minimization – data not stored is data not at risk of being misused or misappropriated.

Phorm’s revolutionary approach to online advertising provides numerous benefits to all participants in the Internet ecosystem.

For Consumers
With transparency and choice, OIX allows consumers to receive more relevant advertising, and unlike other systems with lower standards of transparency and choice, OIX is not reliant on knowing who the consumer is to provide relevant advertising.

For Advertisers
Advertisers are able to reach the audience appropriate to their offering.  Tailored advertising allows niche advertisers who previously were not able or willing to advertise online to participate.

Online Publishers
OIX allows participating publishers to achieve a premium value for their advertising space.  This increased value for publishers offers them the ability to reduce the number of less valuable ads in favor of fewer, more valuable and tailored ads.  It also allows smaller (“long tail”) publishers to effectively enter the competitive online market and serve a wider array of advertisers.  Also, as time has shown, increased publisher success has led to a richer array of free offerings and a move away from subscription-based content.

ISPs
Phorm’s OIX technology provides a new and much-needed revenue stream to broadband providers facing dramatically increasing bandwidth consumption by subscribers using online video, music, VoIP and gaming.  This allows ISPs to invest in their networks without increasing rates for consumers.

In summary, Phorm has built a system from the ground up to respect user anonymity, transparency and consumer choice.  This system has been audited by Ernst and Young, and leading privacy consultancy 80/20 Thinking has completed a full Privacy Impact Assessment (PIA) on Phorm’s technology (copies available at privacy.phorm.com).  Phorm’s technology has been cleared by the relevant UK regulatory authorities as capable of compliance with the European Data Directive, and as of October 2008, Phorm is in a technical trial phase with British Telecom.  As other global ISPs continue to examine the many benefits of Phorm for the Internet ecosystem, Phorm expects many more ISPs to conclude that they too can play a role in creating a new model for a more relevant, yet more private, Internet experience for consumers.

Each day, billions of messages roam the Internet, divided into small packets that each seek their way across the Internet. Only the sender and recipient reassemble the packets to get the message. Unfortunately, many messages today are malicious, with worms, viruses, or spyware, which organised criminals exploit to commit cybercrimes on a large scale. Here, deep packet inspection (DPI) might come to the rescue, since it allows monitoring and filtering of packets wherever they happen to pass. DPI can also meet other objectives in security, service provision, or compliance assurance. But do we really want to have a technology that enables instant, ubiquitous monitoring of everything that travels the Internet?

DPI is the next surveillance application that enters society unnoticed and suddenly is there, begging to be used. It follows closed-circuit television (CCTV), aerial photography, miniature cameras, directional microphones, biometrics, olfactory sensors, automated face and number-plate recognition, data mining, and profiling as yet another way of watching (over) us. In recent years, we have seen an enormous increase in data generation, processing, and storage: we are not only a networked but also a database society. DPI enlarges the surveillance toolkit primarily by allowing many more actors to collect data and to use them for their own purposes.

Leaving aside issues of private-actor surveillance, I want to call attention here to the use governments are likely to make of DPI. Once Internet Service Providers (ISPs), or other companies for that matter, embrace DPI, they can monitor and select passing traffic much more sophisticatedly than by merely scanning header information. This capacity can prove of great benefit to law enforcement agencies and intelligence services, who can use existing investigation powers to enlist assistance of ISPs. Particularly relevant is that DPI allows for real-time monitoring, and hence facilitates a preventative approach as opposed to the retroactive approach that law enforcement traditionally used.

DPI therefore adds to the trend that broader groups of unsuspected citizens are under surveillance: rather than investigating relatively few individuals on the basis of reasonable indications that they have committed a crime, more people, including groups, are nowadays being watched for slight indications of being involved in (potential) crimes. Thus, the ‘footprint’ of criminal law and intelligence is slowly widening to cover more circles of society. This preventative tendency in law enforcement fits the movement towards a risk society (Beck) and a culture of control (Garland). The factual explosion of data generation, inspection, and storage enable the government to collect and use significantly more data about citizens than before, and this increase is not only quantitative but also qualitative. The personal lives of citizens are reflected in their Internet behaviour, and if that can be monitored ubiquitously and perpetually, they are becoming increasingly transparent to the government.

An increased government power of knowledge over citizens is not necessarily wrong, since changes in society may warrant such a shift. However, it should be carefully argued that increased surveillance is indeed necessary, and empirical data are required to substantiate this. Surveillance developments are, however, often rather matter-of-fact; the whole process is piecemeal with small individual steps, which together constitute a giant leap. The policy and societal debates often focus on the individual steps rather than on the entire leap, and it is questionable whether the cumulative move towards surveillance and preventative risk control is evidence-based and well-considered. A key recommendation for legislatures is to pay more attention to empirical underpinning of surveillance measures and their cumulative effect, to commission evaluation studies, and to use sunset clauses in legislation in case a measure does not show effect.

Also, more checks and balances are required. The increased government power needs to be balanced by additional checks, notably with more transparency requirements (citizens must know which data are being collected and processed for which purposes) and with enhanced audit and supervision. Independent authorities should regularly check whether the government uses its powers correctly and legitimately. The criminal court is no longer the primary instrument to check the execution of investigation powers, since many cases are not brought before the court, and alternative supervision mechanisms should be considered.

In surveillance debates, data protection is a key element. To my view, the legal framework for data protection has become outdated. The assumption of preventing data processing as much as possible is no longer valid in the current networked database society. Large-scale data collection and correlation is inevitable nowadays, and the emergence of DPI serves to underline this. Therefore, instead of focusing data protection on prevention in the data collection stage, it should rather be focused on decent treatment in the data usage stage. In other words, data protection is valuable not so much to enhance privacy, but to ensure transparency of government and non-discrimination.

While data protection can serve to regulate the use of data, it remains to be discussed whether DPI should be allowed for government use in the first place. Here, other elements of privacy come to the fore: protection of the home, family relations, and correspondence. These elements are likely to be infringed by DPI. Since privacy is a core constitutional value to safeguard citizens’ liberty and autonomy in a democratic constitutional state, DPI should be critically assessed. DPI could be accepted as a new tool for law enforcement, if it turns out a necessary addition to the current investigation toolkit. But the cumulative power of this toolkit to make unsuspected citizens completely transparent to the government surely requires a fundamental rethinking of legal protection. Society needs substantial new checks and balances to counter-balance the increase in government power over its citizens.