Sat. Apr 20th, 2024

XKeyscore: NSA tool collects ‘nearly everything a user does on the internet’

 XKeyscore gives ‘widest-reaching’ collection of online data  NSA analysts require no prior authorization for searches  Sweeps up emails, social media activity and browsing history
One presentation claims the XKeyscore program covers 'nearly everything a typical user does on the internet'
One presentation claims the XKeyscore program covers ‘nearly everything a typical user does on the internet’

A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals, according to documents provided by whistleblower Edward Snowden.

The NSA boasts in training materials that the program, called XKeyscore, is its “widest-reaching” system for developing intelligence from the internet.

The latest revelations will add to the intense public and congressional debate around the extent of NSA surveillance programs. They come as senior intelligence officials testify to the Senate judiciary committee on Wednesday, releasing classified documents in response to the Guardian’s earlier stories on bulk collection of phone records and Fisa surveillance court oversight.

The files shed light on one of Snowden’s most controversial statements, made in his first video interview published by the Guardian on June 10.

“I, sitting at my desk,” said Snowden, could “wiretap anyone, from you or your accountant, to a federal judge or even the president, if I had a personal email”.

US officials vehemently denied this specific claim. Mike Rogers, the Republican chairman of the House intelligence committee, said of Snowden’s assertion: “He’s lying. It’s impossible for him to do what he was saying he could do.”

But training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed.

 

XKeyscore, the documents boast, is the NSA’s “widest reaching” system developing intelligence from computer networks – what the agency calls Digital Network Intelligence (DNI). One presentation claims the program covers “nearly everything a typical user does on the internet”, including the content of emails, websites visited and searches, as well as their metadata.

Analysts can also use XKeyscore and other NSA systems to obtain ongoing “real-time” interception of an individual’s internet activity.

Under US law, the NSA is required to obtain an individualized Fisa warrant only if the target of their surveillance is a ‘US person’, though no such warrant is required for intercepting the communications of Americans with foreign targets. But XKeyscore provides the technological capability, if not the legal authority, to target even US persons for extensive electronic surveillance without a warrant provided that some identifying information, such as their email or IP address, is known to the analyst.

One training slide illustrates the digital activity constantly being collected by XKeyscore and the analyst’s ability to query the databases at any time.

Photograph: Guardian
Photograph: Guardian

The purpose of XKeyscore is to allow analysts to search the metadata as well as the content of emails and other internet activity, such as browser history, even when there is no known email account (a “selector” in NSA parlance) associated with the individual being targeted.

Analysts can also search by name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used.

One document notes that this is because “strong selection [search by email address] itself gives us only a very limited capability” because “a large amount of time spent on the web is performing actions that are anonymous.”

The NSA documents assert that by 2008, 300 terrorists had been captured using intelligence from XKeyscore.

Analysts are warned that searching the full database for content will yield too many results to sift through. Instead they are advised to use the metadata also stored in the databases to narrow down what to review.

A slide entitled “plug-ins” in a December 2012 document describes the various fields of information that can be searched. It includes “every email address seen in a session by both username and domain”, “every phone number seen in a session (eg address book entries or signature block)” and user activity – “the webmail and chat activity to include username, buddylist, machine specific cookies etc”.

Email monitoring

In a second Guardian interview in June, Snowden elaborated on his statement about being able to read any individual’s email if he had their email address. He said the claim was based in part on the email search capabilities of XKeyscore, which Snowden says he was authorized to use while working as a Booz Allen contractor for the NSA.

One top-secret document describes how the program “searches within bodies of emails, webpages and documents”, including the “To, From, CC, BCC lines” and the ‘Contact Us’ pages on websites”.

To search for emails, an analyst using XKS enters the individual’s email address into a simple online search form, along with the “justification” for the search and the time period for which the emails are sought.

Photograph: Guardian
Photograph: Guardian
Photograph: Guardian
Photograph: Guardian

The analyst then selects which of those returned emails they want to read by opening them in NSA reading software.

The system is similar to the way in which NSA analysts generally can intercept the communications of anyone they select, including, as one NSA document put it, “communications that transit the United States and communications that terminate in the United States”.

One document, a top secret 2010 guide describing the training received by NSA analysts for general surveillance under the Fisa Amendments Act of 2008, explains that analysts can begin surveillance on anyone by clicking a few simple pull-down menus designed to provide both legal and targeting justifications. Once options on the pull-down menus are selected, their target is marked for electronic surveillance and the analyst is able to review the content of their communications:

Photograph: Guardian
Photograph: Guardian

Chats, browsing history and other internet activity

Beyond emails, the XKeyscore system allows analysts to monitor a virtually unlimited array of other internet activities, including those within social media.

An NSA tool called DNI Presenter, used to read the content of stored emails, also enables an analyst using XKeyscore to read the content of Facebook chats or private messages.

Photograph: Guardian
Photograph: Guardian

An analyst can monitor such Facebook chats by entering the Facebook user name and a date range into a simple search screen.

Photograph: Guardian
Photograph: Guardian

Analysts can search for internet browsing activities using a wide range of information, including search terms entered by the user or the websites viewed.

Photograph: Guardian
Photograph: Guardian

As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls “nearly everything a typical user does on the internet”.

Photograph: Guardian
Photograph: Guardian

The XKeyscore program also allows an analyst to learn the IP addresses of every person who visits any website the analyst specifies.

Photograph: Guardian
Photograph: Guardian

The quantity of communications accessible through programs such as XKeyscore is staggeringly large. One NSA report from 2007 estimated that there were 850bn “call events” collected and stored in the NSA databases, and close to 150bn internet records. Each day, the document says, 1-2bn records were added.

William Binney, a former NSA mathematician, said last year that the agency had “assembled on the order of 20tn transactions about US citizens with other US citizens”, an estimate, he said, that “only was involving phone calls and emails”. A 2010 Washington Post article reported that “every day, collection systems at the [NSA] intercept and store 1.7bn emails, phone calls and other type of communications.”

The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: “At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.”

To solve this problem, the NSA has created a multi-tiered system that allows analysts to store “interesting” content in other databases, such as one named Pinwale which can store material for up to five years.

It is the databases of XKeyscore, one document shows, that now contain the greatest amount of communications data collected by the NSA.

Photograph: Guardian
Photograph: Guardian

In 2012, there were at least 41 billion total records collected and stored in XKeyscore for a single 30-day period.

Photograph: Guardian
Photograph: Guardian

Legal v technical restrictions

While the Fisa Amendments Act of 2008 requires an individualized warrant for the targeting of US persons, NSA analysts are permitted to intercept the communications of such individuals without a warrant if they are in contact with one of the NSA’s foreign targets.

The ACLU’s deputy legal director, Jameel Jaffer, told the Guardian last month that national security officials expressly said that a primary purpose of the new law was to enable them to collect large amounts of Americans’ communications without individualized warrants.

“The government doesn’t need to ‘target’ Americans in order to collect huge volumes of their communications,” said Jaffer. “The government inevitably sweeps up the communications of many Americans” when targeting foreign nationals for surveillance.

An example is provided by one XKeyscore document showing an NSA target in Tehran communicating with people in Frankfurt, Amsterdam and New York.

Photograph: Guardian
Photograph: Guardian

In recent years, the NSA has attempted to segregate exclusively domestic US communications in separate databases. But even NSA documents acknowledge that such efforts are imperfect, as even purely domestic communications can travel on foreign systems, and NSA tools are sometimes unable to identify the national origins of communications.

Moreover, all communications between Americans and someone on foreign soil are included in the same databases as foreign-to-foreign communications, making them readily searchable without warrants.

Some searches conducted by NSA analysts are periodically reviewed by their supervisors within the NSA. “It’s very rare to be questioned on our searches,” Snowden told the Guardian in June, “and even when we are, it’s usually along the lines of: ‘let’s bulk up the justification’.”

In a letter this week to senator Ron Wyden, director of national intelligence James Clapper acknowledged that NSA analysts have exceeded even legal limits as interpreted by the NSA in domestic surveillance.

Acknowledging what he called “a number of compliance problems”, Clapper attributed them to “human error” or “highly sophisticated technology issues” rather than “bad faith”.

However, Wyden said on the Senate floor on Tuesday: “These violations are more serious than those stated by the intelligence community, and are troubling.”

In a statement to the Guardian, the NSA said: “NSA’s activities are focused and specifically deployed against – and only against – legitimate foreign intelligence targets in response to requirements that our leaders need for information necessary to protect our nation and its interests.

“XKeyscore is used as a part of NSA’s lawful foreign signals intelligence collection system.

“Allegations of widespread, unchecked analyst access to NSA collection data are simply not true. Access to XKeyscore, as well as all of NSA’s analytic tools, is limited to only those personnel who require access for their assigned tasks … In addition, there are multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring.”

“Every search by an NSA analyst is fully auditable, to ensure that they are proper and within the law.

“These types of programs allow us to collect the information that enables us to perform our missions successfully – to defend the nation and to protect US and allied troops abroad.”


X-Keyscore:
Allows the NSA and Allies to Monitor Emails,
Web Browsing, Internet Searches and Social Media

NSA’s Google for the World’s Private Communications

XKeyscore 

XKeyscore (XKEYSCORE or XKS) is a secret computer system used by the United States National Security Agency (NSA) for searching and analyzing global Internet data, which it collects in real time. The NSA has shared XKeyscore with other intelligence agencies, including the Australian Signals Directorate, Canada’s Communications Security Establishment, New Zealand’s Government Communications Security Bureau, Britain’s Government Communications Headquarters, Japan’s Defense Intelligence Headquarters, and Germany’s Bundesnachrichtendienst.[1]

In July 2013, Edward Snowden publicly revealed the program’s purpose and use by the NSA in The Sydney Morning Herald and O Globo newspapers. The code name was already public knowledge because it was mentioned in earlier articles, and, like many other code names, it appears in job postings and online résumés of employees.[2][3]

On July 3, 2014, German public broadcaster Norddeutscher Rundfunk, a member of ARD, published excerpts of XKeyscore’s source code.[4] A team of experts analyzed the source code.[5] cited


X-Keyscore spy program tracks ‘nearly all’ web use

Top-secret US intelligence documents leaked by whistleblower Edward Snowden have revealed details of a key signals intelligence program used by the Australian intelligence community to harvest internet and telecommunications traffic across the Asia-Pacific region.
The system known as X-Keyscore allows the US National Security Agency and international partners including Australia to monitor ”nearly everything a typical user does on the internet”, according to the leaked documents published by Britain’s Guardian newspaper.
X-Keyscore: Allows the NSA and allies to monitor emails, web browsing, internet searches and social media.CREDIT:MAYU KANAMORI/JESSICA HROMAS
X-Keyscore: Allows the NSA and allies to monitor emails, web browsing, internet searches and social media.CREDIT:MAYU KANAMORI/JESSICA HROMAS
According to classified intelligence training materials, X-Keyscore allows the NSA and its allied partners to comprehensively monitor the emails, web browsing, internet searches and social media use of targets.
This includes ”real-time target activity [tipping]” and a ”rolling buffer of three days of all unfiltered data” with the ”full take” stored at collection facilities – enabling analysts to retrospectively access the communications of newly identified targets.
Significantly, all the secret documents are classified as available to personnel from ”Five-eyes” intelligence partners: the US, UK, Canada, Australia and New Zealand.
Australian intelligence sources recently confirmed to Fairfax Media that Australia’s electronic espionage agency, the Defence Signals Directorate, was a ”full partner” in the program.
It is claimed that by 2008 more than 300 terrorists had been captured thanks to intelligence from X-Keyscore.
However, Australian intelligence sources emphasise the reach of the system for diplomatic, political and economic intelligence collection on ”targets of interest across the whole Asia-Pacific” – including China, Indonesia, Malaysia, India, Sri Lanka and Pakistan.
Advertisement
Documents previously disclosed by Mr Snowden identified Australian signals intelligence facilities at Geraldton in Western Australia, Shoal Bay near Darwin, HMAS Harman near Canberra and the US-Australian Joint Defence Facility at Pine Gap near Alice Springs as contributors to the global collection of internet and telecoms traffic under the X-Keyscore program.
Fairfax recently reported the construction of a new data storage facility at HMAS Harman to support the surge in data collection by Australian intelligence agencies.
Last week the US House of Representatives only narrowly voted down a proposal to defund the NSA’s ability to collect electronic information, including phone call records.
But President Barack Obama has been forced to declassify aspects of the surveillance programs.
Democrat senator Dianne Feinstein – the chairwoman of the US Senate intelligence committee and a staunch supporter of the wide-ranging electronic surveillance – says she is looking into reforms including greater public reporting of phone interception and meta-data collection statistics
Australian diplomat cables show close consultation between US and Australian officials about Mr Snowden’s disclosures. However, most details have been redacted on the grounds they would reveal confidential discussions or ”comment and analysis of any implications of Snowden’s breach for Australian communications systems”.
NSA director General Keith Alexander, who is also Commander of the United States Cyber Command, defended the surveillance programs this week at the annual Black Hat computer security conference in Las Vegas.


XKEYSCORE

NSA’s Google for the World’s Private Communications

Morgan Marquis-Boire cited theintercept.com

ONE OF THE National Security Agency’s most powerful tools of mass surveillance makes tracking someone’s Internet usage as easy as entering an email address, and provides no built-in technology to prevent abuse. Today, The Intercept is publishing 48 top-secret and other classified documents about XKEYSCORE dated up to 2013, which shed new light on the breadth, depth and functionality of this critical spy system — one of the largest releases yet of documents provided by NSA whistleblower Edward Snowden.

The NSA’s XKEYSCORE program, first revealed by The Guardian, sweeps up countless people’s Internet searches, emails, documents, usernames and passwords, and other private communications. XKEYSCORE is fed a constant flow of Internet traffic from fiber optic cables that make up the backbone of the world’s communication network, among other sources, for processing. As of 2008, the surveillance system boasted approximately 150 field sites in the United States, Mexico, Brazil, United Kingdom, Spain, Russia, Nigeria, Somalia, Pakistan, Japan, Australia, as well as many other countries, consisting of over 700 servers.

These servers store “full-take data” at the collection sites — meaning that they captured all of the traffic collected — and, as of 2009, stored content for 3 to 5 days and metadata for 30 to 45 days. NSA documents indicate that tens of billions of records are stored in its database. “It is a fully distributed processing and query system that runs on machines around the world,” an NSA briefing on XKEYSCORE says. “At field sites, XKEYSCORE can run on multiple computers that gives it the ability to scale in both processing power and storage.”

Illustration: Blue Delliquanti and David Axe for The Intercept
Illustration: Blue Delliquanti and David Axe for The Intercept

XKEYSCORE also collects and processes Internet traffic from Americans, though NSA analysts are taught to avoid querying the system in ways that might result in spying on U.S. data. Experts and privacy activists, however, have long doubted that such exclusions are effective in preventing large amounts of American data from being swept up. One document The Intercept is publishing today suggests that FISA warrants have authorized “full-take” collection of traffic from at least some U.S. web forums.

The system is not limited to collecting web traffic. The 2013 document, “VoIP Configuration and Forwarding Read Me,” details how to forward VoIP data from XKEYSCORE into NUCLEON, NSA’s repository for voice intercepts, facsimile, video and “pre-released transcription.” At the time, it supported more than 8,000 users globally and was made up of 75 servers absorbing 700,000 voice, fax, video and tag files per day.

The reach and potency of XKEYSCORE as a surveillance instrument is astonishing. The Guardian report noted that NSA itself refers to the program as its “widest reaching” system. In February of this year, The Intercept reported that NSA and GCHQ hacked into the internal network of Gemalto, the world’s largest provider of cell phone SIM cards, in order to steal millions of encryption keys used to protect the privacy of cell phone communication. XKEYSCORE played a vital role in the spies’ hacking by providing government hackers access to the email accounts of Gemalto employees.

Numerous key NSA partners, including Canada, New Zealand and the U.K., have access to the mass surveillance databases of XKEYSCORE. In March, the New Zealand Herald, in partnership with The Interceptrevealed that the New Zealand government used XKEYSCORE to spy on candidates for the position of World Trade Organization director general and also members of the Solomon Islands government.

These newly published documents demonstrate that collected communications not only include emails, chats and web-browsing traffic, but also pictures, documents, voice calls, webcam photos, web searches, advertising analytics traffic, social media traffic, botnet traffic, logged keystrokes, computer network exploitation (CNE) targeting, intercepted username and password pairs, file uploads to online services, Skype sessions and more.

Bulk collection and population surveillance

XKEYSCORE allows for incredibly broad surveillance of people based on perceived patterns of suspicious behavior. It is possible, for instance, to query the system to show the activities of people based on their location, nationality and websites visited. For instance, one slide displays the search “germansinpakistn,” showing an analyst querying XKEYSCORE for all individuals in Pakistan visiting specific German language message boards.

As sites like Twitter and Facebook become increasingly significant in the world’s day-to-day communications (a Pew study shows that 71 percent of online adults in the U.S. use Facebook), they become a critical source of surveillance data. Traffic from popular social media sites is described as “a great starting point” for tracking individuals, according to an XKEYSCORE presentation titled “Tracking Targets on Online Social Networks.”

When intelligence agencies collect massive amounts of Internet traffic all over the world, they face the challenge of making sense of that data. The vast quantities collected make it difficult to connect the stored traffic to specific individuals.

Internet companies have also encountered this problem and have solved it by tracking their users with identifiers that are unique to each individual, often in the form of browser cookies. Cookies are small pieces of data that websites store in visitors’ browsers. They are used for a variety of purposes, including authenticating users (cookies make it possible to log in to websites), storing preferences, and uniquely tracking individuals even if they’re using the same IP address as many other people. Websites also embed code used by third-party services to collect analytics or host ads, which also use cookies to track users. According to one slide, “Almost all websites have cookies enabled.”

The NSA’s ability to piggyback off of private companies’ tracking of their own users is a vital instrument that allows the agency to trace the data it collects to individual users. It makes no difference if visitors switch to public Wi-Fi networks or connect to VPNs to change their IP addresses: the tracking cookie will follow them around as long as they are using the same web browser and fail to clear their cookies.

Illustration: Blue Delliquanti and David Axe for The Intercept
Illustration: Blue Delliquanti and David Axe for The Intercept

Apps that run on tablets and smartphones also use analytics services that uniquely track users. Almost every time a user sees an advertisement (in an app or in a web browser), the ad network is tracking users in the same way. A secret GCHQ and CSE program called BADASS, which is similar to XKEYSCORE but with a much narrower scope, mines as much valuable information from leaky smartphone apps as possible, including unique tracking identifiers that app developers use to track their own users. In May of this year, CBC, in partnership with The Interceptrevealed that XKEYSCORE was used to track smartphone connections to the app marketplaces run by Samsung and Google. Surveillance agency analysts also use other types of traffic data that gets scooped into XKEYSCORE to track people, such as Windows crash reports.

In a statement to The Intercept, the NSA reiterated its position that such sweeping surveillance capabilities are needed to fight the War on Terror:

“The U.S. Government calls on its intelligence agencies to protect the United States, its citizens, and its allies from a wide array of serious threats. These threats include terrorist plots from al-Qaeda, ISIL, and others; the proliferation of weapons of mass destruction; foreign aggression against the United States and our allies; and international criminal organizations.”

Indeed, one of the specific examples of XKEYSCORE applications given in the documents is spying on Shaykh Atiyatallah, an al Qaeda senior leader and Osama bin Laden confidant. A few years before his death, Atiyatallah did what many people have often done: He googled himself. He searched his various aliases, an associate and the name of his book. As he did so, all of that information was captured by XKEYSCORE.

XKEYSCORE has, however, also been used to spy on non-terrorist targets. The April 18, 2013 issue of the internal NSA publication Special Source Operations Weekly boasts that analysts were successful in using XKEYSCORE to obtain U.N. Secretary General Ban Ki-moon’s talking points prior to a meeting with President Obama.

Illustration: Blue Delliquanti and David Axe for The Intercept
Illustration: Blue Delliquanti and David Axe for The Intercept

XKEYSCORE for hacking: Easily collecting user names, passwords and much more

XKEYSCORE plays a central role in how the U.S. government and its surveillance allies hack computer networks around the world. One top-secret 2009 NSA document describes how the system is used by the NSA to gather information for the Office of Tailored Access Operations, an NSA division responsible for Computer Network Exploitation (CNE) — i.e., targeted hacking.

Particularly in 2009, the hacking tactics enabled by XKEYSCORE would have yielded significant returns as use of encryption was less widespread than today. Jonathan Brossard, a security researcher and the CEO of Toucan Systems, told The Intercept: “Anyone could be trained to do this in less than one day: they simply enter the name of the server they want to hack into XKEYSCORE, type enter, and are presented login and password pairs to connect to this machine. Done. Finito.” Previous reporting by The Intercept revealed that systems administrators are a popular target of the NSA. “Who better to target than the person that already has the ‘keys to the kingdom?’” read a 2012 post on an internal NSA discussion board.

This system enables analysts to access web mail servers with remarkable ease.

The same methods are used to steal the credentials — user names and passwords — of individual users of message boards.

Hacker forums are also monitored for people selling or using exploits and other hacking tools. While the NSA is clearly monitoring to understand the capabilities developed by its adversaries, it is also monitoring locations where such capabilities can be purchased.

Other information gained via XKEYSCORE facilitates the remote exploitation of target computers. By extracting browser fingerprint and operating system versions from Internet traffic, the system allows analysts to quickly assess the exploitability of a target. Brossard, the security researcher, said that “NSA has built an impressively complete set of automated hacking tools for their analysts to use.”

Given the breadth of information collected by XKEYSCORE, accessing and exploiting a target’s online activity is a matter of a few mouse clicks. Brossard explains: “The amount of work an analyst has to perform to actually break into remote computers over the Internet seems ridiculously reduced — we are talking minutes, if not seconds. Simple. As easy as typing a few words in Google.”

These facts bolster one of Snowden’s most controversial statements, made in his first video interview published by The Guardian on June 9, 2013. “I, sitting at my desk,” said Snowden, could “wiretap anyone, from you or your accountant, to a federal judge to even the president, if I had a personal email.”

Indeed, training documents for XKEYSCORE repeatedly highlight how user-friendly the program is: with just a few clicks, any analyst with access to it can conduct sweeping searches simply by entering a person’s email address, telephone number, name or other identifying data. There is no indication in the documents reviewed that prior approval is needed for specific searches.

In addition to login credentials and other target intelligence, XKEYSCORE collects router configuration information, which it shares with Tailored Access Operations. The office is able to exploit routers and then feed the traffic traveling through those routers into their collection infrastructure. This allows the NSA to spy on traffic from otherwise out-of-reach networks. XKEYSCORE documents reference router configurations, and a document previously published by Der Spiegel shows that “active implants” can be used to “cop[y] traffic and direc[t]” it past a passive collector.

XKEYSCORE for counterintelligence

Beyond enabling the collection, categorization, and querying of metadata and content, XKEYSCORE has also been used to monitor the surveillance and hacking actions of foreign nation states and to gather the fruits of their hacking. The Intercept previously reported that NSA and its allies spy on hackers in order to collect what they collect.

Once the hacking tools and techniques of a foreign entity (for instance, South Korea) are identified, analysts can then extract the country’s espionage targets from XKEYSCORE, and gather information that the foreign power has managed to steal.

Monitoring of foreign state hackers could allow the NSA to gather techniques and tools used by foreign actors, including knowledge of zero-day exploits—software bugs that allow attackers to hack into systems, and that not even the software vendor knows about—and implants. Additionally, by monitoring vulnerability reports sent to vendors such as Kaspersky, the agency could learn when exploits they were actively using need to be retired because they’ve been discovered by a third party.

Seizure v. searching: Oversight, audit trail and the Fourth Amendment

By the nature of how it sweeps up information, XKEYSCORE gathers communications of Americans, despite the Fourth Amendment protection against “unreasonable search and seizure” — including searching data without a warrant. The NSA says it does not target U.S. citizens’ communications without a warrant, but acknowledges that it “incidentally” collects and reads some of it without one, minimizing the information that is retained or shared.

But that interpretation of the law is dubious at best.

XKEYSCORE training documents say that the “burden is on user/auditor to comply with USSID-18 or other rules,” apparently including the British Human Rights Act (HRA), which protects the rights of U.K. citizens. U.S. Signals Intelligence Directive 18 (USSID 18) is the American directive that governs “U.S. person minimization.”

Kurt Opsahl, the Electronic Frontier Foundation’s general counsel, describes USSID 18 as “an attempt by the intelligence community to comply with the Fourth Amendment. But it doesn’t come from a court, it comes from the executive.”

If, for instance, an analyst searched XKEYSCORE for all iPhone users, this query would violate USSID 18 due to the inevitable American iPhone users that would be grabbed without a warrant, as the NSA’s own training materials make clear.

Opsahl believes that analysts are not prevented by technical means from making queries that violate USSID 18. “The document discusses whether auditors will be happy or unhappy. This indicates that compliance will be achieved by after-the-fact auditing, not by preventing the search.”

Screenshots of the XKEYSCORE web-based user interface included in slides show that analysts see a prominent warning message: “This system is audited for USSID 18 and Human Rights Act compliance.” When analysts log in to the system, they see a more detailed message warning that “an audit trail has been established and will be searched” in response to HRA complaints, and as part of the USSID 18 and USSID 9 audit process.

Because the XKEYSCORE system does not appear to prevent analysts from making queries that would be in violation of these rules, Opsahl concludes that “there’s a tremendous amount of power being placed in the hands of analysts.” And while those analysts may be subject to audits, “at least in the short term they can still obtain information that they shouldn’t have.”

During a symposium in January 2015 hosted at Harvard University, Edward Snowden, who spoke via video call, said that NSA analysts are “completely free from any meaningful oversight.” Speaking about the people who audit NSA systems like XKEYSCORE for USSID 18 compliance, he said, “The majority of the people who are doing the auditing are the friends of the analysts. They work in the same office. They’re not full-time auditors, they’re guys who have other duties assigned. There are a few traveling auditors who go around and look at the things that are out there, but really it’s not robust.”

In a statement to The Intercept, the NSA said:

“The National Security Agency’s foreign intelligence operations are 1) authorized by law; 2) subject to multiple layers of stringent internal and external oversight; and 3) conducted in a manner that is designed to protect privacy and civil liberties. As provided for by Presidential Policy Directive 28 (PPD-28), all persons, regardless of their nationality, have legitimate privacy interests in the handling of their personal information. NSA goes to great lengths to narrowly tailor and focus its signals intelligence operations on the collection of communications that are most likely to contain foreign intelligence or counterintelligence information.”


XKeyscore presentation from 2008 – read in full

Training materials for the XKeyscore program detail how analysts can use it and other systems to mine enormous agency databases and develop intelligence from the web

XKeyscore: NSA tool collects ‘nearly everything a user does on the internet’

written by  cited
  • XKeyscore gives ‘widest-reaching’ collection of online data
  • NSA analysts require no prior authorization for searches
  • Sweeps up emails, social media activity and browsing history
One presentation claims the XKeyscore program covers 'nearly everything a typical user does on the internet'
One presentation claims the XKeyscore program covers ‘nearly everything a typical user does on the internet’
A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals, according to documents provided by whistleblower Edward Snowden.
The NSA boasts in training materials that the program, called XKeyscore, is its “widest-reaching” system for developing intelligence from the internet.
The latest revelations will add to the intense public and congressional debate around the extent of NSA surveillance programs. They come as senior intelligence officials testify to the Senate judiciary committee on Wednesday, releasing classified documents in response to the Guardian’s earlier stories on bulk collection of phone records and Fisa surveillance court oversight.
The files shed light on one of Snowden’s most controversial statements, made in his first video interview published by the Guardian on June 10.
“I, sitting at my desk,” said Snowden, could “wiretap anyone, from you or your accountant, to a federal judge or even the president, if I had a personal email”.
US officials vehemently denied this specific claim. Mike Rogers, the Republican chairman of the House intelligence committee, said of Snowden’s assertion: “He’s lying. It’s impossible for him to do what he was saying he could do.”
But training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed.
XKeyscore, the documents boast, is the NSA’s “widest reaching” system developing intelligence from computer networks – what the agency calls Digital Network Intelligence (DNI). One presentation claims the program covers “nearly everything a typical user does on the internet”, including the content of emails, websites visited and searches, as well as their metadata.
Analysts can also use XKeyscore and other NSA systems to obtain ongoing “real-time” interception of an individual’s internet activity.
Under US law, the NSA is required to obtain an individualized Fisa warrant only if the target of their surveillance is a ‘US person’, though no such warrant is required for intercepting the communications of Americans with foreign targets. But XKeyscore provides the technological capability, if not the legal authority, to target even US persons for extensive electronic surveillance without a warrant provided that some identifying information, such as their email or IP address, is known to the analyst.
One training slide illustrates the digital activity constantly being collected by XKeyscore and the analyst’s ability to query the databases at any time.
Photograph: Guardian
Photograph: Guardian
The purpose of XKeyscore is to allow analysts to search the metadata as well as the content of emails and other internet activity, such as browser history, even when there is no known email account (a “selector” in NSA parlance) associated with the individual being targeted.
Analysts can also search by name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used.
One document notes that this is because “strong selection [search by email address] itself gives us only a very limited capability” because “a large amount of time spent on the web is performing actions that are anonymous.”
The NSA documents assert that by 2008, 300 terrorists had been captured using intelligence from XKeyscore.
Analysts are warned that searching the full database for content will yield too many results to sift through. Instead they are advised to use the metadata also stored in the databases to narrow down what to review.
A slide entitled “plug-ins” in a December 2012 document describes the various fields of information that can be searched. It includes “every email address seen in a session by both username and domain”, “every phone number seen in a session (eg address book entries or signature block)” and user activity – “the webmail and chat activity to include username, buddylist, machine specific cookies etc”.
Email monitoring
In a second Guardian interview in June, Snowden elaborated on his statement about being able to read any individual’s email if he had their email address. He said the claim was based in part on the email search capabilities of XKeyscore, which Snowden says he was authorized to use while working as a Booz Allen contractor for the NSA.
One top-secret document describes how the program “searches within bodies of emails, webpages and documents”, including the “To, From, CC, BCC lines” and the ‘Contact Us’ pages on websites”.
To search for emails, an analyst using XKS enters the individual’s email address into a simple online search form, along with the “justification” for the search and the time period for which the emails are sought.
Photograph: Guardian
Photograph: Guardian
Photograph: Guardian
Photograph: Guardian
The analyst then selects which of those returned emails they want to read by opening them in NSA reading software.
The system is similar to the way in which NSA analysts generally can intercept the communications of anyone they select, including, as one NSA document put it, “communications that transit the United States and communications that terminate in the United States”.
One document, a top secret 2010 guide describing the training received by NSA analysts for general surveillance under the Fisa Amendments Act of 2008, explains that analysts can begin surveillance on anyone by clicking a few simple pull-down menus designed to provide both legal and targeting justifications. Once options on the pull-down menus are selected, their target is marked for electronic surveillance and the analyst is able to review the content of their communications:
Photograph: Guardian
Photograph: Guardian
Chats, browsing history and other internet activity
Beyond emails, the XKeyscore system allows analysts to monitor a virtually unlimited array of other internet activities, including those within social media.
An NSA tool called DNI Presenter, used to read the content of stored emails, also enables an analyst using XKeyscore to read the content of Facebook chats or private messages.
Photograph: Guardian
Photograph: Guardian
An analyst can monitor such Facebook chats by entering the Facebook user name and a date range into a simple search screen.
Photograph: Guardian
Photograph: Guardian
Analysts can search for internet browsing activities using a wide range of information, including search terms entered by the user or the websites viewed.
Photograph: Guardian
Photograph: Guardian
As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls “nearly everything a typical user does on the internet”.
Photograph: Guardian
Photograph: Guardian
The XKeyscore program also allows an analyst to learn the IP addresses of every person who visits any website the analyst specifies.
Photograph: Guardian
Photograph: Guardian
The quantity of communications accessible through programs such as XKeyscore is staggeringly large. One NSA report from 2007 estimated that there were 850bn “call events” collected and stored in the NSA databases, and close to 150bn internet records. Each day, the document says, 1-2bn records were added.
William Binney, a former NSA mathematician, said last year that the agency had “assembled on the order of 20tn transactions about US citizens with other US citizens”, an estimate, he said, that “only was involving phone calls and emails”. A 2010 Washington Post article reported that “every day, collection systems at the [NSA] intercept and store 1.7bn emails, phone calls and other type of communications.”
The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: “At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.”
To solve this problem, the NSA has created a multi-tiered system that allows analysts to store “interesting” content in other databases, such as one named Pinwale which can store material for up to five years.
It is the databases of XKeyscore, one document shows, that now contain the greatest amount of communications data collected by the NSA.
Photograph: Guardian
Photograph: Guardian
In 2012, there were at least 41 billion total records collected and stored in XKeyscore for a single 30-day period.
Photograph: Guardian
Photograph: Guardian
Legal v technical restrictions
While the Fisa Amendments Act of 2008 requires an individualized warrant for the targeting of US persons, NSA analysts are permitted to intercept the communications of such individuals without a warrant if they are in contact with one of the NSA’s foreign targets.
The ACLU’s deputy legal director, Jameel Jaffer, told the Guardian last month that national security officials expressly said that a primary purpose of the new law was to enable them to collect large amounts of Americans’ communications without individualized warrants.
“The government doesn’t need to ‘target’ Americans in order to collect huge volumes of their communications,” said Jaffer. “The government inevitably sweeps up the communications of many Americans” when targeting foreign nationals for surveillance.
An example is provided by one XKeyscore document showing an NSA target in Tehran communicating with people in Frankfurt, Amsterdam and New York.
Photograph: Guardian
Photograph: Guardian
In recent years, the NSA has attempted to segregate exclusively domestic US communications in separate databases. But even NSA documents acknowledge that such efforts are imperfect, as even purely domestic communications can travel on foreign systems, and NSA tools are sometimes unable to identify the national origins of communications.
Moreover, all communications between Americans and someone on foreign soil are included in the same databases as foreign-to-foreign communications, making them readily searchable without warrants.
Some searches conducted by NSA analysts are periodically reviewed by their supervisors within the NSA. “It’s very rare to be questioned on our searches,” Snowden told the Guardian in June, “and even when we are, it’s usually along the lines of: ‘let’s bulk up the justification’.”
In a letter this week to senator Ron Wyden, director of national intelligence James Clapper acknowledged that NSA analysts have exceeded even legal limits as interpreted by the NSA in domestic surveillance.
Acknowledging what he called “a number of compliance problems”, Clapper attributed them to “human error” or “highly sophisticated technology issues” rather than “bad faith”.
However, Wyden said on the Senate floor on Tuesday: “These violations are more serious than those stated by the intelligence community, and are troubling.”
In a statement to the Guardian, the NSA said: “NSA’s activities are focused and specifically deployed against – and only against – legitimate foreign intelligence targets in response to requirements that our leaders need for information necessary to protect our nation and its interests.
“XKeyscore is used as a part of NSA’s lawful foreign signals intelligence collection system.
“Allegations of widespread, unchecked analyst access to NSA collection data are simply not true. Access to XKeyscore, as well as all of NSA’s analytic tools, is limited to only those personnel who require access for their assigned tasks … In addition, there are multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring.”
“Every search by an NSA analyst is fully auditable, to ensure that they are proper and within the law.

XKEYSCORE Spy Program Revealed by Snowden Still a Problem

By Radhamely De Leon cited

There are still significant privacy issues with an NSA spying program years after Snowden revealed its existence.

A government watchdog committee is facing criticism for failing to provide sufficient oversight over XKEYSCORE, an NSA surveillance program revealed by Edward Snowden in 2013.

The U.S. Privacy and Civil Liberties Oversight Board (PCLOB) conducted a classified investigation into XKEYSCORE, a highly classified program that the NSA uses to analyze enormous global troves of internet data and communications. That report was delivered late last year to the NSA, Congress, and other executive branch agencies along with recommendations from individual board members. One member of the board who conducted this investigation publicly blasted the nature of the board’s investigation this week, revealing the existence of the report and also blasting it.

“I had hoped that the former majority of the Board would have conducted a more thorough investigation of this highly-classified surveillance program that is unlikely to be scrutinized by another independent oversight authority in the near future,” Travis LeBlanc, a member of the PCLOB wrote, adding that he had “serious reservations” with the classified report.

According to a 2008 presentation acquired by The GuardianXKEYSCORE can collect data from all of its global servers, which at the time spanned 150 locations, using a single query. It can collect metadata from users and also use data from their internet usage to locate them.

LeBlanc writes that, among many things, the report failed to address any algorithmic biases that XKEYSCAPE may have or whether it had the correct compliance procedures in place.

According to LeBlanc, the board “refused” to follow up with any compliance reports that were deemed Questionable Intelligence Activities (QIA), which the Department of Defense defines as an action that resulted in the illegal surveillance or improper review of an individual’s communications. The amount of total QIA’s reported was redacted from the statement.

He also noted that he found it “concerning” that the NSA appeared to not have a written legal analysis until the board requested one in 2015 since these analyses are used to create compliance policies and procedures. The legal analysis that was provided also used decades-old cases to assert that XKEYSCAPE was being used in compliance with the Fourth Amendment.

According to LeBlanc, the board did not do much to investigate the system’s compliance program, which already does not require that analysts receive privacy and civil liberties and compliance training.

The board also failed to investigate the efficacy or cost of the program, which LeBlanc notes is one of the most “basic” parts of an oversight investigation. The 2008 presentation shows that the XKEYSCAPE already had 700 servers across the world but could be scaled even further just by adding more servers. If the NSA has continued to use XKEYSCAPE since then, there’s no telling how much the system has grown or how many people have been affected by its data collection.

“On these points and others, the former Board’s report unfortunately reads more like a book report summary of the XKEYSCORE program than an independent oversight analysis grappling with key concerns in this evolving technological legal landscape,” LeBlanc said in his statement.

LeBlanc urged them to declassify the statement for the sake of transparency as “the public is rightfully worried about secret surveillance programs,” he said.

According to LeBlanc, the board has made no effort to declassify the report.


XKeyScore –

the NSA’s secret tool that collects and reveals

‘nearly everything a user does on the internet’

by Tibi Puiu cited

 


Check out our article on the NSA SPYING SOFTWARE for CELL PHONES called Pegasus – Pegasus spyware: FBI vows not to use after grilling from Capitol Hill