joshhansen.net

Category: the world

“[T]hings both in heaven and in the earth, and under the earth; things which have been, things which are, things which must shortly come to pass; things which are at home, things which are abroad; the wars and the perplexities of the nations, and the judgments which are on the land; and a knowledge also of countries and of kingdoms—” Doctrine and Covenants 88:79

  • A Picture’s Worth A Thousand Senators: Staring Into The Gaping Ideological Chasm That Divides Congress

    In this post, I’m going to introduce you to a cool-looking graph, tell you what it means, and give the technical details of its generation—all because I think America might care. Here we go.

    Introduction

    Politics in America are hopelessly partisan, and all of the bickering serves only to cripple our nation at a moment of crisis when decisive action is called for. You know it. I know it. Barack Obama and John Boehner know it. Your grandma knows it.

    Or do we know it? The belief that American politics has become more polarized in recent decades is widespread. But is there any evidence for it? While I make no attempt to provide a complete explanation for this disturbing trend in our nation’s governance, in this post I present some work that I believe provides an answer—a resounding confirmation that, according to at least one view of the situation, the politics of the United States are now more deeply divided than ever.

    Though this work was done in collaboration with Michael Dimond as part of an advanced data mining course (CS676) at BYU, I believe I am the sole author of the portions of our report excerpted below.

    The Cool-Looking Graph

    Here’s the pretty picture:

    United States Senate Legislator Similarity Network 1789-2011.

    Bask in its glory—and be grateful, because that thing took a lot of work! Make sure to click on the image to see the full-sized version. (It will open in a new window/tab.)

    What It Means

    The above graph is a visual representation of the United States Senate across 222 years of legislative history. It is, in essence, a social network of senators across time—who voted like whom, what cliques and factions formed, etc. In other words, retroactive Facebook for America’s past politicians? No, that’s going too far….

    Anyway, here’s how to interpret the graph. Each node (circle) represents a senator. An arc is drawn between two nodes if the two senators at the endpoints voted on the same bill at least once and voted the same way on bills more than 75% of the time. Size and color of nodes indicate their centrality (a measure of importance) in the network. Scanning from left (1789) to right (2011), a few trends emerge:

    1. The height of the graph increases. Much of this can be attributed to the increase in the number of states, from 13 to 50, meaning the number of senators serving simultaneously increased by 74.
    2. The graph alternates between unity and polarization. Visually, unity looks like a single “stream” of nodes, whereas polarization is the graph splitting into two components that move in slightly different directions.
    3. In recent decades, the height of the graph has continued to increase in spite of the number of senators being fixed at 100 since 1959. I assert that this corresponds to the phenomenon of increased polarization between the two parties.

    I am interested in whether the flow of the graph can be correlated with developments in the American two-party system. Feel free to let me know your thoughts on that. For those wishing to play with the graph data, it’s available here.

    Technical Details

    This stuff gets pretty computer sciencey, so only read on if you really want to nerd out.

    Data

    The graph is generated using an aggregated and sanitized version of the THOMAS congressional data from govtrack.us. This yields 2.1 GiB of primarily XML-encoded congressional data from the 1st to the 112th congress. The data includes a record of votes by all legislators on all roll calls since the 1st congress, as well as party affiliation.

    Social Graph Inference

    Let $latex L$ be the set of all legislators and $latex S$ be the set of all sessions of congress. We define a legislator-to-legislator similarity function $latex \sigma : L \times L \rightarrow [0,1]$ that returns a similarity score for all pairs of legislators that ever voted on the same roll call:

    [latex size=3]
    \sigma(l_{1},l_{2})=\frac{SameVotes(l_{1},l_{2})}{PossibleVotes(l_{1},l_{2})} \\
    \\
    \phantom{\sigma(l_{1},l_{2})}=\frac{\sum_{s \in S : l_{1} \in s \wedge l_{2} \in s} \sum_{r\in Rolls(s)} \beta\left [vote(l_{1},r)=vote(l_{2},r) \right ]}{\sum_{s \in S : l_{1} \in s \wedge l_{2} \in s} |Rolls(s)|}
    [/latex]

    where

    • $latex Rolls(s)$ returns the set of all roll calls (votes) occurring in session $latex s$;
    • $latex \beta[x]$ is an indicator function returning 1 when $latex x$ is true, 0 otherwise;
    • $latex vote(l,r)$ returns the vote cast by legislator $latex l$ on roll $latex r$; and
    • $latex l \in s$ is true iff legislator $latex l$ served in congressional session $latex s$.

    We use this similarity measure to construct a legislator affinity graph as follows:

    Let $latex G=(V,E)$ be an undirected graph with a set of vertices $latex V$ and a set of weighted edges $latex E$, such that

    • $latex V=\{Vertex(l) : l \in L\}$ and
    • $latex E=\{Edge(l_{1}, l_{2}, \sigma(l_{1},l_{2})) : (l_{1},l_{2}) \in L \times L \wedge \sigma(l_{1},l_{2}) > \theta\}$

    where

    • $latex Vertex(l)$ yields the vertex associated with a given legislator $latex l$;
    • $latex Edge(l_{1},l_{2},w)$ yields an undirected edge with weight $latex w$ and endpoints $latex Vertex(l_{1})$ and $latex Vertex(l_{2})$,
    • and $latex \theta \in [0,1]$ is a minimum similarity threshold.

    Rendering

    In practice, the above $latex \theta$ must be set high (I used 0.75) to prevent the number of edges from being excessively large. Once the graph was constructed, it was loaded into Gephi, a graph visualization tool. Betweenness centralities were computed, nodes were sized and colored, and a force-directed layout algorithm was applied. I then manually rotated the graph so that earlier senators are located on the left and more recent senators on the right, to give the effect of a rough historical timeline. I exported this as an SVG file, then loaded it in the Inkscape vector graphics program. With the benefit of 16GB of RAM, I coaxed Inkscape into rendering a 20,000 pixel width PNG image of the graph. This was finally scaled to 10,000 pixels wide for web distribution using GIMP.

    Acknowledgements

    Thanks to Christophe Giraud-Carrier for teaching the class for which this graph was generated, and to Michael Dimond who, though not directly working on this portion of our project, was nevertheless an excellent collaborator. And to my friend who convinced me to finally finish this post.

  • Wireless Mesh Networking As An Agent of Political Subversion

    Just putting this out there as a ‘ping’ to see if anybody else has thought about this. A recent post at One Free Korea inspired an interesting thought. The general idea is that wireless mesh networking technology being developed and deployed these days seems like an ideal foundation for a dissident network disseminating information in oppressed societies such as North Korea, Myanmar, Cuba, and others. Because discovery of a node by government authorities would almost definitely have severe consequences for the node operator, it’s important that node operators be hard to track down.

    Network desiderata:

    • Secure. Broadcasts must somehow be kept from prying government eyes. If activists plan an anti-government protest on compromised channels, the government will know exactly where and when to place riot police to shut it down. So security is essential. This means encryption, but it also means security at the human level. Before admitting a node to the mesh, appropriate measures should be taken to ensure it isn’t a government-sponsored infiltration attempt.
    • Opportunistic. Nodes know when they’re close to other nodes and take advantage of the opportunity to send and receive messages, within constraints.
    • Minimalistic. It’s important to maximize distribution of messages while minimizing the number of nodes each individual node is aware of. This achieves a sort of compartmentalization of information, which at least makes it more difficult for authorities to track down the rest of the network should one of the nodes be compromised.
    • Defensive. Should a node be compromised, measures should be taken to contain the damage. If the owner of the node is able, he can send a special broadcast alerting its neighbors to take precautions. Such precautions might include destroying or hiding of hardware, fleeing so as to avoid interrogation, etc. The alert could be forwarded to as much of the network as possible, prompting radio silence and reorganization of the network to isolate the security breech (moving the neighbors of the compromised node from the center of the mesh out to its periphery where they would do less damage if they, too, were compromised.)
    • Smart. If node A has already sent message 1 to node B, it should never send that message again.
    • Concise. The less time spent actually sending out data, the less the chances of being discovered by means of triangulation. Messages should use a minimal amount of broadcast to send a maximal amount of information. This means compression, and sending of mainly text as opposed to images, audio, or video (though these could have their place).
    • Intermittent. Messages need to be queued rather than being sent ahead immediately. If a node is silent most of the time and only broadcasts occasionally, it’s less likely to be discovered.
    • Location-Unpredictable. A node operator who always broadcasts from the same location is more likely to be discovered, so nodes should be able to broadcast from different locations, reducing the likelihood of discovery.

    Hardware characteristics:

    • Cheap. At least, cheap enough to be fairly widely deployed.
    • Mobile. Totalitarian regimes tend to deny their citizens freedom of movement, so when people do move, the network should take advantage of the opportunity provided to transfer information from one connected component to another. For example, suppose a Pyongyang resident were to visit Hamhung. The network nodes in Hamhung, being disconnected from those in Pyongyang, would not have received communications sent by Pyongyang nodes. But the visitor node, acting as an intermediary, could transmit the Pyongyang broadcasts to the Hamhung nodes.
    • Variable range. Never broadcast at higher power than necessary, yet be able to send very-long-range communications if needed.
    • (Depending on availability of electricity) Low power.
    • Ample storage. If two different connected components remain totally separated for years, a visitor node needs to be able to bring as many broadcasts with it as possible when it reconnects them.

    Software characteristics:

    • Localized. People should be able to easily enter messages in their own language and (hopefully) writing system.

    One of the key outstanding issues would be how such a mesh would be formed to begin with, given that the target population wouldn’t have many other channels by which to coordinate the setup.

    The main objectives of a network like this would be to enable political mobilization by otherwise-oppressed peoples; to provide information about the nature of the oppressive regime through grass-roots news reports, commentaries, etc.; and to provide a connection to the world beyond the nation’s borders. It would be extremely dangerous to deploy such a thing. People would be caught and executed for participating. But the long-run potential payoff of providing free and uncensored information flows could be immense: the liberation of an entire people—intellectually, spiritually, politically, physically, economically, culturally.

  • If You’re A Member of Congress, Vote ‘No’ on Health Care Reform!

    To quote one of my favorite economists, Gary Becker:

    Despite the long debate, many provisions of both the House and Senate bills remain highly controversial. These include, among many others, the way the uninsured would get coverage, the de-emphasis on health savings accounts, the postponement until 2018 of the elimination of the tax advantages from expensive employer-based health plans, no increase in the ability of persons and companies in one state to contract with insurance companies located in other states, and especially the minor efforts to raise out of pocket expenses by consumers of health care in order to reduce their overuse of doctors, drugs, and even hospitals. Such a badly designed health care bill would on the whole worsen rather than improve the American health care system…. [link]

    And, from his fellow-blogger, Richard Posner:

    Because [the health care reform bill currently in congress] is unpopular among the general public, its enactment by a simple majority in both Houses would raise a valid question about the representative character of Congress…. [T]he health care program has been kicking around in Congress for a year, and the inability of its supporters to convince the public of the program’s wisdom, coupled with the program’s enormous cost and its potentially disruptive consequences for the health care industry…and indeed the entire economy, may make people question the democratic legitimacy of enacting the program with just a simple majority in the House and Senate. [link]

    I oppose the health care reform bill because our government’s fiscal health is already in dire shape, and this bill will turn a dangerous fiscal disease into a terminal one. The Democrats’ push to pass this bill is a face-saving sellout of the welfare of the American people who, as I understand it, largely oppose the legislation. If you are a member of congress or perhaps an intern or electronic aggregator doing the bidding of one, be aware that this man wants you to vote ‘no’! Think of the future—not just this fall’s elections.