Metadata and the Wider World

You’ve probably heard metadata discussed in the news, usually in the context of “retained metadata on phone calls” along with denials that any phone call has ever been tapped.

This is completely true. No phone calls are being recorded or listened to (at least not in these data retention programs). What is being recorded is the metadata about the phone call:

  • when the call was made
  • how long the call was, and
  • who the call was made to

This seems pretty innocuous, except it can be linked with other metadata:

  • who else has this caller called
  • who else has called this caller (all callers connected to the first number)
  • how often does each caller call that number
  • who else calls the numbers involved, and do they call each other.

By examining the connections between these people, much can be inferred. (Inferred Metadata can be very valuable.)

With a lot less metadata than the mesh of connected phone calls that modern government organizations collect, Keiran Healey constructs the plot to overthrow the British Government in the young Colony using only publicly known data on membership of certain organizations.

From that starting point, in Using Metadata to find Paul Revere he uses nothing more than this public information to reach this conclusion:

So, there you have it. From a table of membership in different groups we have gotten a picture of a kind of social network between individuals, a sense of the degree of connection between organizations, and some strong hints of who the key players are in this world. And all this—all of it!—from the merest sliver of metadata about a single modality of relationship between people. I do not wish to overstep the remit of my memorandum but I must ask you to imagine what might be possible if we were but able to collect information on very many more people, and also synthesize information from different kinds of ties between people!

It’s a very good read to understand the power of metadata, and more than a little scary.