Shane Harris
Under secret orders from Former President George W Bush, the NSA was already building its own version of Total Information Awareness.
By Shane Harris
A decade ago, a Pentagon research project called “Total Information Awareness” sparked panic because of its seemingly Orwellian interest in categorising and mining every aspect of our digital lives. It was “the supersnoop’s dream,” declared William Safire of the New York Times, a “computerised dossier on your private life from commercial sources, (combined with) every piece of information that government has about you. . . .”
If this sounds reminiscent of the current uproar over NSA surveillance, you’re paying attention. That’s because the NSA monitoring tools are similar to — and, in many cases directly based on — the technology that Total Information Awareness (TIA) tried to use.
The idea behind TIA was to give US intelligence analysts access to the vast universe of electronic information stored in private databases that might be useful for detecting the next plot. Data such as phone call records, emails, and Internet searches. Retired Admiral John Poindexter, who ran the TIA programme, wanted to build what he called a “system of systems” that would access all this raw information, sort and analyse it, and hopefully find indications of terrorist plotting.
The NSA was the biggest collector of electronic data in the government, and Poindexter, who met NSA deputy director Bill Black in February 2002, thought the NSA would be a natural partner in his endeavour. But what he didn’t know was that under secret orders from President George W Bush, the NSA was already building its own version of Total Information Awareness.
Fewer than 100 people at the NSA knew that for the past few months, the agency had been monitoring the phone calls and other electronic communications of Americans, and that it was obtaining copies of domestic phone call records and looking at them for potential clues about terrorist attacks.
The NSA went on to build its own total information awareness system. What was once an idea in Poindexter’s head is now a fully realised global surveillance apparatus capable of gathering unprecedented amounts of digital information for near real-time analysis, or to be stored for future investigations, perhaps years from now.
There are several key respects in which the NSA’s system today mirrors that which Poindexter proposed more than a decade ago.
Access to many categories of private information
TIA envisioned, as its name suggested, access to the total universe of electronic information that might be useful for investigating terrorists. It placed particular emphasis on phone records, e-mails, Internet searches, travel records, and financial transactions — because in order to plan attacks, terrorists need to communicate, conduct research, move around, and make purchases.
Using the PRISM system, it can read emails and see Internet searches, as well as forms of electronic messages that didn’t even exist when TIA was proposed, such as Facebook messages. The agency has also obtained credit card receipt transactions and records from Internet service providers.
Use of “virtual” databases
Rather than trying to make copies of private databases and hold them in a government facility, TIA proposed a kind of federated or “virtual” database. The system would effectively reach out and touch the private databases themselves, or systems that were set up attached to them, working with the information at, or close to, the source and siphoning off what it needed for analysis.
This is what’s happening with the NSA’s Internet mining tool known as PRISM. The NSA doesn’t have “direct access” to company servers, but obtains information on an as-needed basis using a technology that some have described as a drop box.
The company deposits the information NSA wants in the box, and NSA takes it.
(Lack of) privacy protection
NSA abandoned this privacy research when it took over Poindexter’s programmes in 2003, and a privacy appliance as sophisticated as what was hoped for in TIA still doesn’t exist. However, the NSA’s database of phone call records, known as Mainway, now has some privacy controls, according to intelligence employees who have used the system. The database does not contain any names, nor is the NSA collecting geolocation data that could pinpoint a user on a map, according to administration officials. When an analyst comes upon a phone number associated with a US citizen or legal resident, a black ‘X’ mark appears over the number, says one former defence intelligence employee. Administration officials have said publicly that the databases only can be queried as part of a terrorism investigation, and that it has been accessed about 300 times last year.
Much less is known about how PRISM protects the privacy and identities of US persons, whose communications the NSA cannot target without a warrant.
Use of broad searches
Poindexter believed that in order to find the proverbial “needle in the haystack,” analysts needed to be able to look at a lot of haystacks. TIA would cast a wide net searching among mostly innocent and innocuous communications for those that merited further scrutiny.
NSA attempts to do just that with PRISM. It is meant to filter out potentially meaningful signals from an ocean of noise. Gen Keith Alexander, the NSA director, has said that in the vast majority of terrorist attacks that the United States was able to stop, this kind of analysis was essential.
Reliance on court orders
The NSA has come to rely on the Foreign Intelligence Surveillance Court, which is now issuing broad orders for information that, prior to the 9/11 attacks, would have been unimaginable. The court has sanctioned the copying of all phone records in the United States. It also reviews the government’s Internet surveillance methods in an attempt to ensure that they don’t unreasonably scoop up Americans’ data too. This is far from perfect science. On at least one occasion, the court has found that these procedures were unconstitutional. We still don’t know how they were changed to make them legal.
Amended privacy laws
In 2007 and 2008, after some of the NSA’s secret programmes were exposed, Congress debated changes to the Foreign Intelligence Surveillance Act.
The key question still unanswered is whether any of this surveillance actually prevents terrorist attacks.
Perhaps then we can finally decide whether building Total Information Awareness was a good idea.
WP-BLOOMBERG
Under secret orders from Former President George W Bush, the NSA was already building its own version of Total Information Awareness.
By Shane Harris
A decade ago, a Pentagon research project called “Total Information Awareness” sparked panic because of its seemingly Orwellian interest in categorising and mining every aspect of our digital lives. It was “the supersnoop’s dream,” declared William Safire of the New York Times, a “computerised dossier on your private life from commercial sources, (combined with) every piece of information that government has about you. . . .”
If this sounds reminiscent of the current uproar over NSA surveillance, you’re paying attention. That’s because the NSA monitoring tools are similar to — and, in many cases directly based on — the technology that Total Information Awareness (TIA) tried to use.
The idea behind TIA was to give US intelligence analysts access to the vast universe of electronic information stored in private databases that might be useful for detecting the next plot. Data such as phone call records, emails, and Internet searches. Retired Admiral John Poindexter, who ran the TIA programme, wanted to build what he called a “system of systems” that would access all this raw information, sort and analyse it, and hopefully find indications of terrorist plotting.
The NSA was the biggest collector of electronic data in the government, and Poindexter, who met NSA deputy director Bill Black in February 2002, thought the NSA would be a natural partner in his endeavour. But what he didn’t know was that under secret orders from President George W Bush, the NSA was already building its own version of Total Information Awareness.
Fewer than 100 people at the NSA knew that for the past few months, the agency had been monitoring the phone calls and other electronic communications of Americans, and that it was obtaining copies of domestic phone call records and looking at them for potential clues about terrorist attacks.
The NSA went on to build its own total information awareness system. What was once an idea in Poindexter’s head is now a fully realised global surveillance apparatus capable of gathering unprecedented amounts of digital information for near real-time analysis, or to be stored for future investigations, perhaps years from now.
There are several key respects in which the NSA’s system today mirrors that which Poindexter proposed more than a decade ago.
Access to many categories of private information
TIA envisioned, as its name suggested, access to the total universe of electronic information that might be useful for investigating terrorists. It placed particular emphasis on phone records, e-mails, Internet searches, travel records, and financial transactions — because in order to plan attacks, terrorists need to communicate, conduct research, move around, and make purchases.
Using the PRISM system, it can read emails and see Internet searches, as well as forms of electronic messages that didn’t even exist when TIA was proposed, such as Facebook messages. The agency has also obtained credit card receipt transactions and records from Internet service providers.
Use of “virtual” databases
Rather than trying to make copies of private databases and hold them in a government facility, TIA proposed a kind of federated or “virtual” database. The system would effectively reach out and touch the private databases themselves, or systems that were set up attached to them, working with the information at, or close to, the source and siphoning off what it needed for analysis.
This is what’s happening with the NSA’s Internet mining tool known as PRISM. The NSA doesn’t have “direct access” to company servers, but obtains information on an as-needed basis using a technology that some have described as a drop box.
The company deposits the information NSA wants in the box, and NSA takes it.
(Lack of) privacy protection
NSA abandoned this privacy research when it took over Poindexter’s programmes in 2003, and a privacy appliance as sophisticated as what was hoped for in TIA still doesn’t exist. However, the NSA’s database of phone call records, known as Mainway, now has some privacy controls, according to intelligence employees who have used the system. The database does not contain any names, nor is the NSA collecting geolocation data that could pinpoint a user on a map, according to administration officials. When an analyst comes upon a phone number associated with a US citizen or legal resident, a black ‘X’ mark appears over the number, says one former defence intelligence employee. Administration officials have said publicly that the databases only can be queried as part of a terrorism investigation, and that it has been accessed about 300 times last year.
Much less is known about how PRISM protects the privacy and identities of US persons, whose communications the NSA cannot target without a warrant.
Use of broad searches
Poindexter believed that in order to find the proverbial “needle in the haystack,” analysts needed to be able to look at a lot of haystacks. TIA would cast a wide net searching among mostly innocent and innocuous communications for those that merited further scrutiny.
NSA attempts to do just that with PRISM. It is meant to filter out potentially meaningful signals from an ocean of noise. Gen Keith Alexander, the NSA director, has said that in the vast majority of terrorist attacks that the United States was able to stop, this kind of analysis was essential.
Reliance on court orders
The NSA has come to rely on the Foreign Intelligence Surveillance Court, which is now issuing broad orders for information that, prior to the 9/11 attacks, would have been unimaginable. The court has sanctioned the copying of all phone records in the United States. It also reviews the government’s Internet surveillance methods in an attempt to ensure that they don’t unreasonably scoop up Americans’ data too. This is far from perfect science. On at least one occasion, the court has found that these procedures were unconstitutional. We still don’t know how they were changed to make them legal.
Amended privacy laws
In 2007 and 2008, after some of the NSA’s secret programmes were exposed, Congress debated changes to the Foreign Intelligence Surveillance Act.
The key question still unanswered is whether any of this surveillance actually prevents terrorist attacks.
Perhaps then we can finally decide whether building Total Information Awareness was a good idea.
WP-BLOOMBERG
