Random Access Markets: The Free Market Of Information


Bitcoin News Feed writes:
Random Access Markets: The Free Market Of Information

This article is featured in Bitcoin Magazine’s “The Inscription Issue”. Click here to get your Annual Bitcoin Magazine Subscription.

Click here to download a PDF of this article.

The Value Of Bits

Data is the most liquid commodity market in the world. In the smartphone era, unless extreme precautions are taken, everywhere you go, everything you say, and everything you consume is quantifiable among the infinite spectrum of the information goods markets. Information goods, being inherently nonphysical bits of data, can be conceptualized, crafted, produced, or manufactured, disseminated, and consumed exclusively as digital entities. The internet, along with other digital technologies for computation and communication, serves as a comprehensive e-commerce infrastructure, facilitating the entire life cycle of designing, producing, distributing, and consuming a wide array of information goods. The seamless transition of existing information goods from traditional formats to digital formats is easily achievable, not to mention the collection of media formats completely infeasible in the analog world.

A preliminary examination of products within the information goods industry reveals that, while they all exist as pure information products and are uniformly impacted by technological advancements, their respective markets undergo distinct economic transformation processes. These variations in market evolution are inherently tied to differences in product characteristics, production methods, distribution channels, and consumption patterns. Notably, the separation of value creation and revenue processes introduces opportunistic scenarios, potentially leaving established market players with unprofitable customer bases and costly yet diminishing value-creation processes.

Simultaneously, novel organizational architectures may emerge in response to evolving technological conditions, effectively creating and destroying traditional information good markets overnight. The value chains, originally conceived under the assumptions of the traditional information goods economy, undergo radical redesigns as new strategies and tooling materialize in response to the transformative influence of digital production, distribution, and consumption on conventional value propositions for data. For example, mass surveillance was never practical when creating even a single photo meant hours of labor within a specialized photo development room with specific chemical and lightning conditions. Now that there is a camera on every corner, a microphone in every pocket, a ledger entry for every financial transaction, and the means to transmit said data essentially for free across the planet, the market conditions for mass surveillance have unsurprisingly given rise to mass surveillance as a service.

An entirely new industry of “location firms” has grown, with The Markup having demarcated nearly 50 companies selling location data as a service in a 2021 article titled “There’s a Multibillion-Dollar Market for Your Phone’s Location Data” by Keegan and Ng. One such firm, Near, is self-described as curating “one of the world’s largest sources of intelligence on People and Places”, having gathered data representing nearly two billion people across 44 countries. According to a Grand View Research report titled “Location Intelligence Market Size And Share Report, 2030”, the global location intelligence data market cap was worth an estimated “$16.09 billion in 2022 and is projected to grow at a compound annual growth rate (CAGR) of 15.6% from 2023 to 2030”. The market cap of this new information goods industry is mainly “driven by the growing penetration of smart devices and increasing investments in IoT [internet of things] and network services as it facilitates smarter applications and better network connectivity”, giving credence to the idea that technological advancement front-runs network growth which front-runs entirely new forms of e-commerce markets. This, of course, was accelerated by the COVID-19 pandemic, in which government policies resulted in “the increased adoption of location intelligence solutions to manage the changing business scenario as it helps businesses to analyze, map, and share data in terms of the location of their customers”, under the guise of user and societal health.

Within any information goods market, there are only two possible outcomes for market participants: distributing the acquired data or keeping it for yourself.

Click the image above to subscribe!

The Modern Information Goods Market

In the fall of 2021, China launched the Shanghai Data Exchange (SDE) in an attempt to create a state-owned monopoly on a novel speculative commodities market for data scraped from one of the most digitally surveilled populations on the planet. The SDE offered 20 data products at launch, including customer flight information from China Eastern Airlines, as well as data from telecommunications network operators such as China Unicom, China Telecom, and China Mobile. Notably, one of the first known trades made at the SDE was the Commercial Bank of China purchasing data from the state-owned Shanghai Municipal Electric Power Company under the guise of improving their financial services and product offerings.

Shortly before the founding of this data exchange, Huang Qifan, the former mayor of Chongqing, was quoted saying that “the state should monopolize the rights to regulate data and run data exchanges”, while also suggesting that the CCP should be highly selective in setting up data exchanges. “Like stock exchanges, Beijing, Shanghai and Shenzhen can have one, but a general provincial capital city or a municipal city should not have it.”

While the current information goods market has led to such innovations such as speculation on the purchasing of troves of user data, the modern data market was started in earnest at the end of the 1970s, exemplified in the formation of Oracle Corporation in 1977, named after the CIA’s “Project Oracle”, which featured eventual Oracle Corporation co-founders Larry Ellison, Robert Miner, and Ed Oates. The CIA was their first customer, and in 2002, nearly $2.5 billion worth of contracts came from selling software to federal, state, and local governments, accounting for nearly a quarter of their total revenue. Only a few months after September 11, 2001, Ellison penned an op-ed for The New York Times titled “A Single National Security Database” in which the opening paragraph reads “The single greatest step we Americans could take to make life tougher for terrorists would be to ensure that all the information in myriad government databases was copied into a single, comprehensive national security database”. Ellison was quoted in Jeffrey Rosen’s book The Naked Crowd as saying “The Oracle database is used to keep track of basically everything. The information about your banks, your checking balance, your savings balance, is stored in an Oracle database. Your airline reservation is stored in an Oracle database. What books you bought on Amazon is stored in an Oracle database. Your profile on Yahoo! is stored in an Oracle database”. Rosen made note of a discussion with David Carney, a former top-three employee at the CIA, who, after 32 years of service at the agency, left to join Oracle just two months after 9/11 to lead its Information Assurance Center:

“How do you say this without sounding callous?” [Carney] asked. “In some ways, 9/11 made business a bit easier. Previous to 9/11 you pretty much had to hype the threat and the problem.” Carney said that the summer before the attacks, leaders in the public and private sectors wouldn’t sit still for a briefing. Then his face brightened. “Now they clamor for it!”

This relationship has continued for 20 years, and in November 2022, the CIA awarded its Commercial Cloud Enterprise contract to five American companies — Amazon Web Services, Microsoft, Google, IBM, and Oracle. While the CIA did not disclose the exact value of the contract, documents released in 2019 suggested it could be “tens of billions” of dollars over the next 15 years. Unfortunately, this is far from the only data market integration of the private sector, government agencies, and the intelligence community, perhaps best exemplified by data broker LexisNexis.

LexisNexis was founded in 1970, and is, as of 2006, the world’s largest electronic database for legal and public-records-related information. According to their own website, LexisNexis describes themselves as delivering “a comprehensive suite of solutions to arm government agencies with superior data, technology and analytics to support mission success”. LexisNexis consists of nine board members: CEO Haywood Talcove; Dr. Richard Tubb, the longest serving White House physician in U.S. history; Stacia Hylton, former Deputy Director of the U.S. Marshal Service; Brian Stafford, former Director of the U.S. Secret Service; Lee Rivas, CEO for the public sector and health care business units of LexisNexis Risk Solutions; Howard Safir, former NYPD Commissioner and Associate Director of Operations for the U.S. Marshals Service; Floyd Clarke, former Director of the FBI; Henry Udow, Chief Legal Officer and Company Secretary for the RELX Group; and lastly Alan Wade, retired Chief Information Officer for the CIA.

While Wade was still employed by the CIA, he founded Chiliad with Christine Maxwell, sister of Ghislaine Maxwell, and daughter of Robert Maxwell. Christine Maxwell is considered “an early internet pioneer”, having founded Magellan in 1993, one of the premier search engines on the internet. After selling Magellan to Excite, she reinvested her substantial windfall into another big data search technology company: the aforementioned Chiliad. According to a 2020 report by OYE.NEWS, Chiliad made use of “on-demand, massively scalable, intelligent mining of structured and unstructured data through the use of natural language search technologies”, with the firm’s proprietary software being “behind the data search technology used by the FBI’s counterterrorism data warehouse”.

As recently as November 2023, the Wade-connected LexisNexis was given a $16-million, five-year contract with the U.S. Customs and Border Protection “for access to a powerful suite of surveillance tools”, according to available public records, providing access to “social media monitoring, web data such as email addresses and IP address locations, real-time jail booking data, facial recognition services, and cell phone geolocation data analysis tools”. Unfortunately, this is far from the only government agency to utilize LexisNexis’ data brokerage with the aims of circumnavigating constitutional law and civil liberties in regards to surveillance.

In the fall of 2020, LexisNexis was forced to settle for over $5 million after a class action lawsuit alleged the broker sold Department of Motor Vehicle data to U.S. law firms, who were then free to use it for their own business purposes. “Defendants websites allow the purchase of crash reports by report date, location, or driver name and payment by credit card, prepaid bulk accounts or monthly accounts”, the complaint reads. “Purchasers are not required to establish any permissible use provided in the DPPA to obtain access to Plaintiffs’ and Class Members’ MVRs”. In the summer of 2022, a Freedom of Information Act request revealed a $22 million contract between Immigration and Customs Enforcement and LexisNexis. Sejal Zota, a director at Just Futures Law and a practicing attorney working on the lawsuit, made note that LexisNexis makes it possible for ICE to “instantly access sensitive personal data — all without warrants, subpoenas, any privacy safeguards or any show of reasonableness”.

In the aforementioned complaint from 2022, the use of LexisNexis’ Accurint product allows “law enforcement officers [to] surveil and track people based on information these officers would not, in many cases, otherwise be able to obtain without a subpoena, court order, or other legal process…enabling a massive surveillance state with files on almost every adult U.S. consumer”.

A Series Of Tubes

In 2013, it came to the public’s attention that the National Security Agency had covertly breached the primary communication links connecting Yahoo and Google data centers worldwide. This information was based on documents published by WikiLeaks, originally obtained from former NSA contractor Edward Snowden, and corroborated by interviews of government officials.

As per a classified report dated January 9, 2013, the NSA transmits millions of records daily from internal Yahoo and Google networks to data repositories at the agency’s Fort Meade, Maryland headquarters. In the preceding month, field collectors processed and returned 181,280,466 new records, encompassing “metadata” revealing details about the senders and recipients of emails, along with time stamps, as well as the actual content, including text, audio, and video data.

The primary tool employed by the NSA to exploit these data links is a project named MUSCULAR, carried out in collaboration with the British Government Communications Headquarters (GCHQ). Operating from undisclosed interception points, the NSA and GCHQ copy entire data streams through fiber-optic cables connecting the data centers of major Silicon Valley corporations.

This becomes particularly perplexing when considering that, as revealed by a classified document acquired by The Washington Post in 2013, both the NSA and the FBI were already actively tapping into the central servers of nine prominent U.S. internet companies. This covert operation involved extracting audio and video chats, photographs, emails, documents, and connection logs, providing analysts with the means to monitor foreign targets. The method of extraction, as outlined in the document, involves direct collection from the servers of major U.S. service providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, and Apple.

During the same period, the newspaper The Guardian reported that GCHQ — the British counterpart to the NSA — was clandestinely gathering intelligence from these internet companies through a collaborative effort with the NSA. According to documents obtained by The Guardian, the PRISM program seemingly allows GCHQ to bypass the formal legal procedures required in Britain to request personal materials such as emails, photos, and videos, from internet companies based outside the country.

PRISM emerged in 2007 as a successor to President George W. Bush’s secret program of warrantless domestic surveillance, following revelations from the news media, lawsuits, and interventions by the Foreign Intelligence Surveillance Court. Congress responded with the Protect America Act in 2007 and the FISA Amendments Act of 2008, providing legal immunity to private companies cooperating voluntarily with U.S. intelligence collection. Microsoft became PRISM’s inaugural partner, marking the beginning of years of extensive data collection beneath the surface of a heated national discourse on surveillance and privacy.

In a June 2013 statement, then-Director of National Intelligence James R. Clapper said “information collected under this program is among the most important and valuable foreign intelligence information we collect, and is used to protect our nation from a wide variety of threats. The unauthorized disclosure of information about this important and entirely legal program is reprehensible and risks important protections for the security of Americans”.

So why the need for collection directly from fiber optic cables if these private companies themselves are already providing data to the national intelligence community? Upon further inquiry into the aforementioned data brokers to the NSA and CIA, it would appear that a vast majority of the new submarine fiber optic cables — essential infrastructure to the actualization of the internet as a global data market — are being built out by these same private companies. These inconspicuous cables weave across the global ocean floor, transporting 95-99% of international data through bundles of fiber-optic strands scarcely thicker than a standard garden hose. In total, the active network comprises over 1,100,000 kilometers of submarine cables.

Traditionally, these cables have been owned by a consortium of private companies, primarily telecom providers. However, a notable shift has emerged. In 2016, a significant surge in submarine cable development began, and notably, this time, the purchasers are content providers — particularly the data brokers Meta/Facebook, Google, Microsoft, and Amazon. Of note is Google, having acquired over 100,000 kilometers of submarine cables. With the completion of the Curie Cable in 2019, Google’s ownership of submarine cables globally stands at 1.4%, as measured by length. When factoring in cables with shared ownership, Google’s overall share increases to approximately 8.5%. Facebook is shortly behind with 92,000 kilometers, with Amazon at 30,000, and Microsoft with around 6,500 kilometers from the partially owned MAREA cable.

There is a notable revival in the undersea cable sector, primarily fueled by investments from Facebook and Google, accounting for around 80% of 2018-2020 investments in transatlantic connections — a significant increase from the less than 20% they accounted for in the preceding three years through 2017, as reported by TeleGeography. This wave of digital giants has fundamentally transformed the dynamics of the industry. Unlike traditional practices where phone companies established dedicated ventures for cable construction, often connecting England to the U.S. for voice calls and limited data traffic, these internet companies now wield considerable influence. They can dictate the cable landing locations, strategically placing them near their data centers, and have the flexibility to modify the line structures — typically costing around $200 million for a transatlantic link — without waiting for partner approvals. These technology behemoths aim to capitalize on the increasing demand for rapid data transfers essential for various applications, including streaming movies, social messaging, and even telemedicine.

The last time we saw such an explosion of activity in building out essential internet infrastructure was during the dot-com boom of the 1990s, in which phone companies spent over $20 billion to install fiber-optic lines beneath the oceans, immediately before the massive proliferation of personal computers, home internet modems, and peer-to-peer data networks.

Data Laundering

The birthing of new compression technologies in the form of digital media formats itself would not have given rise to the panopticon we currently operate under without the ability to obfuscate mass uploading and downloading of this newly created data via the ISP rails of both public and private sector infrastructure companies. There is likely no accident that the creation of these tools, networks, and algorithms were created under the influence of national intelligence agencies right before the turn of the millennium, the rise of broadband internet, and the sweeping unconstitutional spying on citizens made legal via the Patriot Act in the aftermath of the events on September 11, 2001.

Only 15 years old, Sean Parker, the eventual founder of Napster and first president of Facebook — a former DARPA project titled LifeLog — caught the gaze of the FBI for his hacking exploits, ending in state-appointed community ser­vice. One year later, Parker was recruited by the CIA after winning a Virginia state computer science fair by developing an early internet crawling application. Instead of continuing his studies, he interned for a D.C. startup, FreeLoader, and eventually UUNet, an internet service provider. “I wasn’t going to school,” Parker told Forbes. “I was technically in a co-op program but in truth was just going to work.” Parker made nearly six figures his senior year of high school, eventually starting the peer-to-peer music-sharing site that became Napster in 1999. While working on Napster, Parker met investor Ron Conway, who has backed every Parker product since, having also previously backed PayPal, Google, and Twitter, among others. Napster has been credited as one of the fastest-growing businesses of all time, and its influence on information goods and data markets in the internet age cannot be overstated.

In a study conducted between April 2000 and November 2001 by Sandvine titled “Peer-to-peer File Sharing: The Impact of File Sharing on Service Provider Networks”, network measurements revealed a notable shift in bandwidth consumption patterns due to the launch of new peer-to-peer tooling, as well as new compression algorithms suc