So Anonymous (quintuple slogan: “We are Anonymous, We are legion, We never forgive, We never forget, Expect us.”) is at it again, defacing the DOJ sentencing website (including the above video) and threatening release of Justice Department secrets. Gotta say the production quality on the video is surprisingly good. The voice sounds like a synthesized version of Liam Neeson, which is funny—maybe they’re going for the “moral authority” thing that he’s often been associated with.
I never quite know what to make of Anonymous, though. On the one hand, they’re law-breaking hackers; on the other hand, some of the grievances listed in the video are actually reasonable critiques of the justice system. On the one hand, they’re practicing extortion on a grand scale, threatening destructive information release (they liken their files to atomic bombs) if their policy objectives aren’t met; on the other hand, I think it’s healthy for society to have influential non-governmental actors.
What if the individuals behind Anonymous (that assumes it isn’t some massive artificial intelligence floating around in the cloud somewhere) were to use their many skills and obvious passion to reform society through persuasion rather than extortion? Somebody in the organization has something of a rhetorical gift, if that video is any indication. Surely people so clever could find better ways of using their time. But those better ways wouldn’t be anything like as glorious and high-profile as the cyberterrorism thing.
Late 19th century Europe was the cradle of modern anarcho-terrorist philosophy, justifying violence as the only cry that would be heard by an oppressive state. For the most part, though, all those guys did was blow people up. It’s hard to say that the course of history was really swayed by them. Even Al Qaeda’s 9/11 achieved none of the change its planners had hoped. So who does change society? How is corruption and oppression really brought down?
Gorbachev could reform the Soviet Union because he was an influential insider. Hitler was brought down by a massive war machine, by strategic blunders, by “Aryan” arrogance. Yet influential insiders often perpetuate oppression (Kim Jong Il, anybody?) and massive war machines enforce it. The oppressor’s folly is the freeman’s hope; but foolish freemen cast their freedom away of their own choice.
My suggestion is for Anonymous to find some way to reform society and retain its flair for the dramatic that doesn’t hypocritically mock the rule of law they claim to seek. But I guess they didn’t ask me, did they?
Hello, friends! Hello, blog! Been a while, eh? In the long history of this blog (existing in various forms since January 2004!) there have often been dry spells. Droughts, if you will. But then something drives me back to the blog to at least make an appearance.
I sort of feel like it’s time for a blogging renaissance. It seems like somewhat of a stodgy old medium when compared to Facebook, Twitter, Instagram, and the like. But maybe I’m a little bit stodgy at heart. I like thinking, I like writing, I like sharing—just not always in bite-sized snippets. So here I am blogging.
Time for a little life update: yes, I’m technically still in grad school. No, I’m not getting a PhD. Yes, I’ve been in school for aaaaaaaaaaaaaages. Yes, my thesis advisor is supposed to be reading my thesis this week. No, I haven’t heard from him yet. Yes, that could be a good thing. And yes, that could also be a bad thing.
Wow, those sentences felt oddly like a bit vector.
In the time while I’ve been waiting for feedback on my thesis, I’ve remembered once again just how much school stresses me out. Without the sense of having to do work on my thesis every day hanging over my head, I have found myself feeling happier, more connected to people, etc.
I’m working for Adobe as a “research scientist”, which is sort of big talk for someone who’s still finishing his master’s thesis. I get to work on some cool problems, play with huge amounts of data, learn about all kinds of new stuff like Hadoop and cloud computing, and solidify my understanding of Bayesian stats / machine learning.
The trouble is that I have to work in a highly distracting and noisy office environment that leaves me irritated and unable to focus. “Attention restoration theory” has something to say about this. The environment I work in during the week is vastly more hostile to sustained focus than the environment I’m in right now. At work, there are noisy air conditioning fans, noisy people, low walls and no privacy (it’s supposed to encourage collaboration), people walking by, people conversing, etc. Here at home, I hear a gentle rumble of distant cars outside, and there are no visual distractions or other noise inside. At work I’m expected to sit in an office chair and work at a desk. Here at home I’m sitting on my bed. At work if I get sleepy I just have to “power through it”, doing crappy work due to fatigue, because there is no provision made for naps. Here at home I can just take a nap when I need one.
In this post, I’m going to introduce you to a cool-looking graph, tell you what it means, and give the technical details of its generation—all because I think America might care. Here we go.
Introduction
Politics in America are hopelessly partisan, and all of the bickering serves only to cripple our nation at a moment of crisis when decisive action is called for. You know it. I know it. Barack Obama and John Boehner know it. Your grandma knows it.
Or do we know it? The belief that American politics has become more polarized in recent decades is widespread. But is there any evidence for it? While I make no attempt to provide a complete explanation for this disturbing trend in our nation’s governance, in this post I present some work that I believe provides an answer—a resounding confirmation that, according to at least one view of the situation, the politics of the United States are now more deeply divided than ever.
Though this work was done in collaboration with Michael Dimond as part of an advanced data mining course (CS676) at BYU, I believe I am the sole author of the portions of our report excerpted below.
The Cool-Looking Graph
Here’s the pretty picture:
United States Senate Legislator Similarity Network 1789-2011.
Bask in its glory—and be grateful, because that thing took a lot of work! Make sure to click on the image to see the full-sized version. (It will open in a new window/tab.)
What It Means
The above graph is a visual representation of the United States Senate across 222 years of legislative history. It is, in essence, a social network of senators across time—who voted like whom, what cliques and factions formed, etc. In other words, retroactive Facebook for America’s past politicians? No, that’s going too far….
Anyway, here’s how to interpret the graph. Each node (circle) represents a senator. An arc is drawn between two nodes if the two senators at the endpoints voted on the same bill at least once and voted the same way on bills more than 75% of the time. Size and color of nodes indicate their centrality (a measure of importance) in the network. Scanning from left (1789) to right (2011), a few trends emerge:
The height of the graph increases. Much of this can be attributed to the increase in the number of states, from 13 to 50, meaning the number of senators serving simultaneously increased by 74.
The graph alternates between unity and polarization. Visually, unity looks like a single “stream” of nodes, whereas polarization is the graph splitting into two components that move in slightly different directions.
In recent decades, the height of the graph has continued to increase in spite of the number of senators being fixed at 100 since 1959. I assert that this corresponds to the phenomenon of increased polarization between the two parties.
I am interested in whether the flow of the graph can be correlated with developments in the American two-party system. Feel free to let me know your thoughts on that. For those wishing to play with the graph data, it’s available here.
Technical Details
This stuff gets pretty computer sciencey, so only read on if you really want to nerd out.
Data
The graph is generated using an aggregated and sanitized version of the THOMAS congressional data from govtrack.us. This yields 2.1 GiB of primarily XML-encoded congressional data from the 1st to the 112th congress. The data includes a record of votes by all legislators on all roll calls since the 1st congress, as well as party affiliation.
Social Graph Inference
Let $latex L$ be the set of all legislators and $latex S$ be the set of all sessions of congress. We define a legislator-to-legislator similarity function $latex \sigma : L \times L \rightarrow [0,1]$ that returns a similarity score for all pairs of legislators that ever voted on the same roll call:
[latex size=3]
\sigma(l_{1},l_{2})=\frac{SameVotes(l_{1},l_{2})}{PossibleVotes(l_{1},l_{2})} \\
\\
\phantom{\sigma(l_{1},l_{2})}=\frac{\sum_{s \in S : l_{1} \in s \wedge l_{2} \in s} \sum_{r\in Rolls(s)} \beta\left [vote(l_{1},r)=vote(l_{2},r) \right ]}{\sum_{s \in S : l_{1} \in s \wedge l_{2} \in s} |Rolls(s)|}
[/latex]
where
$latex Rolls(s)$ returns the set of all roll calls (votes) occurring in session $latex s$;
$latex \beta[x]$ is an indicator function returning 1 when $latex x$ is true, 0 otherwise;
$latex vote(l,r)$ returns the vote cast by legislator $latex l$ on roll $latex r$; and
$latex l \in s$ is true iff legislator $latex l$ served in congressional session $latex s$.
We use this similarity measure to construct a legislator affinity graph as follows:
Let $latex G=(V,E)$ be an undirected graph with a set of vertices $latex V$ and a set of weighted edges $latex E$, such that
$latex V=\{Vertex(l) : l \in L\}$ and
$latex E=\{Edge(l_{1}, l_{2}, \sigma(l_{1},l_{2})) : (l_{1},l_{2}) \in L \times L \wedge \sigma(l_{1},l_{2}) > \theta\}$
where
$latex Vertex(l)$ yields the vertex associated with a given legislator $latex l$;
$latex Edge(l_{1},l_{2},w)$ yields an undirected edge with weight $latex w$ and endpoints $latex Vertex(l_{1})$ and $latex Vertex(l_{2})$,
and $latex \theta \in [0,1]$ is a minimum similarity threshold.
Rendering
In practice, the above $latex \theta$ must be set high (I used 0.75) to prevent the number of edges from being excessively large. Once the graph was constructed, it was loaded into Gephi, a graph visualization tool. Betweenness centralities were computed, nodes were sized and colored, and a force-directed layout algorithm was applied. I then manually rotated the graph so that earlier senators are located on the left and more recent senators on the right, to give the effect of a rough historical timeline. I exported this as an SVG file, then loaded it in the Inkscape vector graphics program. With the benefit of 16GB of RAM, I coaxed Inkscape into rendering a 20,000 pixel width PNG image of the graph. This was finally scaled to 10,000 pixels wide for web distribution using GIMP.
Acknowledgements
Thanks to Christophe Giraud-Carrier for teaching the class for which this graph was generated, and to Michael Dimond who, though not directly working on this portion of our project, was nevertheless an excellent collaborator. And to my friend who convinced me to finally finish this post.
