There are a ton of problems with data mining for potential enemies of the state. Privacy is one, of course. But another is its questionable utility. It doesn’t make you a jihadist, because you’ve e-mailed Chris Allbritton, who interviews guerillas sometimes. Or because you’ve said “bomb” and “trainwreck” in the same overseas call. Just look at all the hijinks with our “no-fly” lists, to see what an imprecise science we’re talking about here.
So I guess I’m not surprised to learn from tomorrow’s New York Times that the NSA’s domestic eavesdropping project — which some seem to think is awfully similar to a rather infamous data mining program — produced a “flood” of tips, and “virtually all of [which] led to dead ends or innocent Americans.”
More than a dozen current and former law enforcement and counterterrorism officials, including some in the small circle who knew of the secret eavesdropping program and how it played out at the F.B.I., said the torrent of tips led them to few potential terrorists inside the country they did not know of from other sources and diverted agents from counterterrorism work they viewed as more productive.
“We’d chase a number, find it’s a schoolteacher with no indication they’ve ever been involved in international terrorism – case closed,” said one former F.B.I. official, who was aware of the program and the data it generated for the bureau. “After you get a thousand numbers and not one is turning up anything, you get some frustration…”
Officials who were briefed on the N.S.A. program said the agency collected much of the data passed on to the F.B.I. as tips by tracing phone numbers in the United States called by suspects overseas, and then by following the domestic numbers to other numbers called. In other cases, lists of phone numbers appeared to result from the agency’s computerized scanning of communications coming in and out of the country for names and keywords that might be of interest. The deliberate blurring of the source of the tips caused some frustration among those who had to follow up.
F.B.I. field agents, who were not told of the domestic surveillance programs, complained they often were given no information about why names or numbers had come under suspicion. A former senior prosecutor, who was familiar with the eavesdropping programs, said intelligence officials turning over the tips “would always say that we had information whose source we can’t share, but it indicates that this person has been communicating with a suspected Al Qaeda operative.” He said, “I would always wonder, what does ’suspected’ mean?”…
Aside from the director, F.B.I. officials did not question the legal status of the tips, assuming that N.S.A. lawyers had approved. They were more concerned about the quality and quantity of the material, which produced “mountains of paperwork” that was often more like raw data than conventional investigative leads.
“It affected the F.B.I. in the sense that they had to devote so many resources to tracking every single one of these leads, and, in my experience, they were all dry leads,” the former senior prosecutor said.
Of course, any wide-spread investigation is going to mean a ton of dead ends. But, under normal circumstances, if there’s a problem with the information you get, you can go back to your sources, ask more questions, hit them up again. If all you’re getting is a list of names and numbers, however, there’s no follow-up possible. No chance to prioritize the information. No way of telling whether this run of the algorithm is actually going to work, this time.
UPDATE 01/07/06 12:03AM: Does it strike anybody else as odd that the NSA’s “unofficial ambassador,” author James Bamford, is now suing to stop the domestic spying program? Do you think he’d be doing that without the tacit approval of at least some of his contacts within the agency?
UPDATE 01/01/06 12:29 PM: Al Gore was one of my least-favorite presidential candidates of all time. But he’s got this NSA thing nailed.
President Lincoln, of course, suspended habeas corpus during the Civil War, and some of the worst abuses prior to those of the current administration were committed by President Wilson during and after World War I, with the notorious red scare and “Palmer Raids.”
…But in each of these cases throughout American history, when the conflict and turmoil subsided, our nation recovered its equilibrium and absorbed the lessons learned in a recurring cycle of excess and regret.
But there are reasons for concern this time around that conditions may be changing so that this cycle may not repeat itself. For one thing, we have for decades been witnessing the slow and steady accumulation of presidential power….
A second reason to believe that we may be experiencing something new, outside that historical cycle, is that we are, after all, told by this administration that the war footing upon which he has tried to place the country is going to last, in their phrase, “for the rest of our lives.”
And so we are told that the conditions of national threat that have been used by other presidents to justify arrogations of power will in this case persist in near perpetuity.
Third, we need to be keenly aware of the startling advances in the sophistication of eavesdropping and surveillance technologies with their capacity to easily sweep up and analyze enormous quantities of information and then mine it for intelligence. And this adds significant vulnerability to the privacy and freedom of enormous numbers of innocent people at the same time as the potential power of those technologies grows.
Those technologies do have the potential for shifting the balance of power between the apparatus of the state and the freedom of the individual in ways that are both subtle and profound.
Don’t misunderstand me. The threat of additional terror strikes is real and the concerted efforts by terrorists to acquire weapons of mass destruction does indeed create a real imperative to exercise the powers of the executive branch with swiftness and agility.
Moreover, there is an in fact an inherent power conferred by the Constitution to any president to take unilateral action when necessary to protect the nation from a sudden and immediate threat. And it is simply not possible to precisely define in legalistic terms exactly when that power is appropriate and when it is not.
But the existence of that inherent power cannot be used to justify a gross and excessive power grab lasting for many years and producing a serious imbalance in the relationship between the executive and the other two branches of government.
So you’d think that Jonas would be all into the idea of using these data-mining systems to predict who the next terrorist attacker might be. 
Now, Darpa is showing its 
The Transportation Security Administration’s airline screening system “tends to 
