NSA shopping for snoopwareFebruary 25, 2006
Last September, the National Security Agency was granted a patent for a technique that could be used to determine the physical location of an Internet address. The technique exploits the tiny time delays in the transmission of Internet data.
As the NSA visit to the Silicon Valley venture capitalists indicates, the actual development of such technologies often comes from private companies.
The National Security Agency officials slipped into Silicon Valley on one of the agency's periodic technology shopping expeditions this month. On the wish list, computerized systems that reveal connections between seemingly innocuous and unrelated pieces of information.
If the tools they were looking for are new, their application is well established: the practice of data mining was pioneered two decades ago both by intelligence agencies and by supercomputer companies looking for commercial markets.
Technology industry executives and government officials said that the intelligence agency systems take such techniques further, applying software analysis tools now routinely used by law enforcement agencies to identify criminal activities and political terrorist organizations that would otherwise be missed by human eavesdroppers.
But by fundamentally changing the nature of surveillance, high-tech data mining raises privacy concerns that are only beginning to be debated widely. That is because to find illicit activities it is necessary to turn loose software sentinels to examine all digital behavior whether it is innocent or not.
"The theory is that the automated tool that is conducting the search is not violating the law," said Mark D. Rasch, the former head of computer-crime investigations for the Justice Department.
The NSA has invested billions in computerized tools for monitoring millions of phone conversations around the world — not only logging them, but also determining content — and more recently in trying to sweep up information from the Internet.
In the wake of 9/11, the potential for mining immense databases of digital information gave rise to a program called Total Information Awareness, developed by Admiral John M. Poindexter, the former national security adviser, while he was a program manager at the Defense Advanced Research Projects Agency.
Although Congress abruptly canceled the program in October 2003, the legislation provided a specific exemption for "processing, analysis and collaboration tools for counterterrorism foreign intelligence."
John Arquilla, a professor of defense analysis at the Naval Postgraduate School in Monterey, Calif., who was a consultant on Admiral Poindexter's Total Information Awareness project, said that the $40 billion spent each year by intelligence agencies had failed to exploit the power of data mining in correlating information readily available from public sources, like monitoring Internet chat rooms. Instead, he said, the government has been investing huge sums in surveillance of phone calls of American citizens.
He was alluding to databases maintained at an AT&T data center in Kansas, which now contain electronic records of 1.92 trillion telephone calls, going back decades. Every telephone call generated a record: number called, time of call, duration of call, billing category and other details. New calls are entered into the database immediately after they end.
The Electronic Frontier Foundation has asserted in a lawsuit that the AT&T Daytona system, a giant warehouse of calling records and Internet message routing information, was the foundation of the NSA's effort to mine telephone records without a warrant.
The mining of the databases in other law enforcement investigations is well established, with documented results. One application of the database technology, called Security Call Analysis and Monitoring Platform, or Scamp, offers access to about nine weeks of calling information. It currently handles about 70,000 queries a month from fraud and law enforcement investigators, according to AT&T documents.
Digital technology makes gov't surveillance easier
AT&T sued over NSA eavesdropping