Snowden used simple software to mine data

 10 Feb 2014 - 3:13


LOS ANGELES: The National Security Agency whistle-blower Edward Snowden used inexpensive and widely available software to plunder the agency’s networks, it has been reported, raising further questions about why he was not detected.
Intelligence officials investigating the former contractor, who leaked thousands of documents to media outlets last year, determined that he used web crawler software designed to search, index and back up websites to “scrape” highly classified files, the New York Times reported yesterday.
The unusual activity triggered a brief challenge from agency officials but Snowden persuaded them it was legitimate and continued mining data.
“We do not believe this was an individual sitting at a machine and downloading this much material in sequence,” an unnamed official told the Times. The process, the official said, was “quite automated”.
Web crawlers move from website to website, following links embedded in each document, and can copy everything they encounter.