GDELT, a digital news monitoring service backed by Google Jigsaw, has released a massive, open set of linking data, containing 1.78 billion links in CSV, with four fields for each link: "FromSite,ToSite,NumDays,NumLinks."
The dataset has been purged of boilerplate links from headers and footers and is intended to help researchers analyze trends in linking behavior, in service of GDELT's mission to "support new theories and descriptive understandings of the behaviors and driving forces of global-scale social systems from the micro-level of the individual through the macro-level of the entire planet."
It's 396MB compressed, or 986MB uncompressefd.
One of the most useful ways to use this dataset is to sort by the "NumDays" field to rank the top outlets linking to a given site or the top outlets that linked to another outlet. Using the NumDays field allows you to rank connections based on their longevity and filter out momentary bursts (such as a major story leading an outlet to run dozens and dozens of articles linking to an outside website for several days and then never linking to that website again).
The entire dataset was created with a single line of SQL in Google BigQuery, taking just 64.9 seconds and processing 199GB.
Who Links To Whom? The 30M Edge GKG Outlink Domain Graph April 2016 To Jan 2019 – The GDELT Project [GDELT Project]
(via Naked Capitalism)
Princeton's Center for Information Technology Policy is a marvellous interdisciplinary research center, and it is advertising for "visitors" for one-year stints: postdocs, policy fellows and visiting IT professors.
Today, we are told that the bigness of Big Tech giants was inevitable: the result of "network effects." For example, once everyone you want to talk to is on Facebook, you can't be convinced to use another, superior service, because all the people you'd use that service to talk to are still on Facebook. And […]
Freedom House, a US-government-funded nonprofit think-tank whose mission is to spread democracy, issues an annual "internet freedom" report; this year, for the ninth consecutive year, the report says that internet freedom has declined, and it's all thanks to social media.
A typical security camera can give you peace of mind. But that might be offset by the nagging feeling that it’s watching you, too. The best kind of security system is one that protects both your home and your privacy, and the blurams Dome Pro 1080p Security Camera is one rare model that’s set up […]
Sous vide cooking: It sounds fancy, but it’s actually one of the easiest and most reliable ways to cook. It’s the reason why many restaurants are able to put out delicious dishes with a consistent flavor. All you need is the right equipment, and that hasn’t always been available to those outside the resto crowd. […]
The more you use your computer, the more it becomes possible for others to use it too. Where there are anti-virus systems, there are hackers looking for a way to get around them. That’s why it’s important to get software that doesn’t just passively scout for viruses in the background. The folks behind GlassWire have […]