Yes, Heman is in the Bible!

For a recent project in my text mining class we were required to explore ways of visualizing textual data. This is a tricky problem due to the nature of text, mainly because it is both categorical and sparse. Visualizing numerical data is, relatively speaking, a piece of cake, because real numbers can all be related to each other rather easily. But categorical data doesn’t generally have it that easy. Even if you can order the data, such as putting words in alphabetical order, it doesn’t always give you much insight into the text that the words comprise. After all, we tend to care more about what the text says than about what letters the words it uses start with.

My work was inspired by the cool stuff at Visual Complexity, and heavily influenced by Chris Harrison’s “Visualizing the Bible” page. For one of his visualizations Chris Harrison’s approach was to extract a “social network” of sorts from the biblical text. The more often a name occurs, the bigger it and its dot are on the chart. If two names occur in the same verse they are considered to be somehow connected and have a line drawn between them. The result is pretty cool, so I decided to apply the same general approach to the leaked emails from the University of East Anglia’s Climatic Research Unit.

I grabbed the emails from somewhere-or-other on the Internet (a torrent, maybe? The filename is cru-foia.tbz2) and set to parsing them. Unfortunately, whereas the names of the Bible have been conveniently compiled, nobody has made a list of names in the CRU emails, as far as I know. So I had to resort to automatic means. To that end, I downloaded Lev Ratinov’s LBJ NER tagger from UIUC CS and used it to identify the names in each email. If two names showed up in the same email, a connection was made between them. The more times they co-occurred, the stronger the link. Once all of the names and their connections were counted, they were dumped to a file in the GEXF graph interchange format.

To actually produce a visualization, I used Gephi, an up-and-coming graph visualization tool that allows real-time, interactive visualization of complex graphs. I also worked with a couple of other datasets: a “scriptures” dataset, including the Bible, Apocrypha, Book of Mormon, and Pearl of Great Price; and a dataset covering LDS General Conference addresses from the 1880′s to about 1950. The General Conferences data has been processed using any top non-stopwords (words that aren’t things like ‘the’, ‘to’, ‘with’, etc.) rather than names, which gives less-interesting results.

The results with Gephi can be quite striking, as with the “Heman” image at the beginning of this post. For those with the right browser (Firefox 3.6 or Chrome), here’s a video of using Gephi to work with the CRU data:

Downloads

• Slides from my presentation
• CRU GEXF file
• Scriptures GEXF file
• Book of Mormon-only GEXF file
• General Conference GEXF file

Last night I went to the “Evening for New Graduate Students” at BYU (which was actually secretly or not-so-secretly open to all graduate students — note for next year ). President Samuelson spoke first. During that short talk and during his devotional address on Tuesday, I had the feeling that I have really undervalued President Samuelson’s ideas in the past. Maybe that’s because he doesn’t have the sort of voice you might hear on TV or the radio. The last part of the program was a speech by Dr. Wynn Sterling, Dean of Graduate Studies. He presented a very exciting view of grad school and the potential to become involved to a greater degree in mankind’s quest for knowledge. He encouraged us to engage in that pursuit, even to the point of disagreeing with our advisors and their colleagues. (This seemed promising for me, since I can never seem to keep my mouth shut at lab meetings, colloquiums, and thesis defenses….)

Dr. Sterling’s view of graduate school was idealistic. It contrasts with another common vision of the graduate experience: the realistic. This is the viewpoint of the likes of PHD comics and the satirical essay How to Publish a Scientific Comment in 1 2 3 Easy Steps (which I discovered via Greg Mankiw’s blog). It also seems to be confirmed by the extreme frustration felt by some of my friends in their master’s programs.

I do not accuse Dr. Sterling of any sort of blindness or naiveté when I say that his vision is idealistic. In fact, I like to think that he presented an idealist vision as a sort of counterpoint to the difficulties and even cynicism that often afflict grad students.

2. Opinion Leaders?

When the media announce a new trend in public opinion, I often respond skeptically, asking whether their report is cause or effect. Can data-based analysis determine whether this is just paranoia or if there are some instances of the media leading rather than merely reporting public opinion (not including editorial and opinion page articles)? Most recently articles like this on rising skepticism about the mission in Afghanistan have reminded me of this question.

3. Bathwater

Two retrospectives on the economists’ role in the financial crisis:

• one titled The Last Temptation of Risk by Barry Eichengreen of Berkeley
• and a more recent one by Paul Krugman of Princeton titled How Did Economists Get It So Wrong?

The two articles paint eerily similar and yet vitally different pictures. Largely, Eichengreen blames the crisis on selective reading and self-serving interpretation of free market economics. Krugman blames an idealistic romance with the neo-neoclassical economics that arose after Keynesianism faded. Eichengreen suggests that the future holds a prominent place for empirical economics research. Krugman highlights behavioral economics and hopes for a Keynesian renaissance.

Krugman’s paper is well-crafted, but I think Eichengreen’s is a better portrayal of reality. Maybe that’s my free-marketeer self speaking. But I just can’t help thinking there’s a baby sitting in the economic bathwater that people are dumping out their windows these days. The ideas I learned in my economics classes were not empty — they were just idealized. To abandon them wholesale now reminds me of the ideologically-motivated cataclysms that Chomsky led linguistics through every decade or so. To put it another way, while relativistic physics explained major gaps in the Newtonian model, it didn’t keep Newtonian physics from being a good-enough description of the world for most purposes. Newton wasn’t wrong so much as he was incomplete.

But it’s Eichengreen’s focus on empiricism that really wins me over. We live in an age of data: vast — almost incomprehensibly huge — stores of data waiting to be utilized. Actually making use of it is at once one of the greatest challenges and one of the greatest opportunities of our time. (I believed that even before my two weeks in a class about data mining.) These huge amounts of data give us an opportunity to reason inductively more than ever before, whereas past models of reality relied on a small number of unproven fundamental tenets (“axioms”, “theorems”, “laws”) from which a theoretical structure was assembled by means of deductive reasoning. While these deductive systems are very powerful in addition to having much the same elegance as mathematics (an aesthetic appeal not to be underestimated), they build a very large superstructure atop a relatively small foundation. Any cracks in the foundation can threaten the whole system.

In a way the tension between fact and theory mirrors the idealism/realism contrast mentioned earlier. Humans seem to have a cognitive bias in favor of uniform explanations of phenomena, giving fuel to idealistic theories. Linguists face a similar crisis of empiricism versus theory; sadly(?) there won’t likely be a linguistic analog to global economic catastrophe to shake their academic confidence and encourage a reassessment (Tower of Babel 2: Confoundations ?)

4. Why Are Academic Disciplines Polypolistic?

Or rather, when will disciplines rely less upon a small number of arbiters of what is or isn’t “credible scholarship”? Instead of a few important journals, couldn’t much of the discussion occur right here in the blogosphere? Are scholars really so ill-mannered that they can’t carry out their debates in real time before a world audience just like the open source hackers and the Wikipedians? Even the U.S. Congress seems transparent when compared to some of the academic oligarchies.

Had economics been democratized, in a sense, would it have been less susceptible to the sort of groupthink that seemingly got it into trouble? Or would it just have been a different type of groupthink? How do you kill the echo chamber without simply gagging everybody?

Speaking of open scholarly discourse, I now wish to present a(n) hypothesis [indefinite article parenthesized for correctness in certain British Commonwealth nations {hint: it's not Fiji.}]:

5. A(n) Hypothesis

I hypothesize that music modeling will encounter much less of a data sparseness problem than word-level language modeling. This issue came up in a PhD thesis proposal I attended today, and it made me think: though I agree that music and human language are similar in many ways, music seems more closely analogous to the character-level or phonological properties of language, rather than to its word-level, syntactic properties. In other words, a phoneme trigram model’s entropy will be much closer to a note trigram model’s entropy than to a word trigram model’s entropy. Does that even make sense? And, is it correct?

6. Terminus

And so it ends. 10 bonus points if you read this.

BYU Natural Language Processing Laboratory

Feature Engineering Console

Spoken Language Identification Project

Statistical NLP Library

Pacific Northwest National Laboratory

APGEMS 3

CRAFT Database GUI

• “Gutenberg’s Heir: The Internet As An Agent of Linguistic Change.” SCHWA: Linguistic Perspectives on the English Language. 1:1 (2006), 41-52. [pdf]
• “Example of Human Health Impacts Assessment Using Imported Soil Concentration Spreadsheet Data (Example No. 2).” ARAMS Tutorials. May 2004. [pdf | local copy]

Conference Presentations

• “Features for the People: An Expert-Enabling User Interface for Feature Engineering in Spoken Language Identification.” Brigham Young University College of Physical and Mathematical Sciences Spring Research Conference 2008. 15 March 2008. [pdfs:abstract | slides | slides+notes]

Class Papers

• “Is There A Human-Specific Language Faculty?” Linguistics Senior Paper. 13 December 2006. [pdf]

Rejected Papers

• “Features for the People: An Expert-Enabling User Interface for Feature Engineering.” Submitted to LREC 2008 but not accepted. [pdf]

Craziness of project for school, craziness of trying (with varying degrees of success) to keep the dinner group organized, craziness with temple committee stuff, craziness starting a poetry club…. Craziness! Did I say craziness?

All of that contrasted with The Most Relaxing Weekend EVAR (to vernaculate). I attribute the relaxatiousness of the weekend to a combination of watching General Conference, and a drive in the canyon to see the colorful trees with some friends.

Things I wish I could be working on right now but just haven’t gotten to:

• statistical analysis of collocations in the rongorongo corpus, plus development of GUI to facilitate it.
• finishing the story I’ve been working on for the last few months.
• reading “Poetry: An Introduction”.
• really learning about graduate programs and getting my applications started.
• solidifying/actually learning information theory and probability theory.

As the Spanish-speakers say: ¡Uf!

And, lastly, I’m introducing a new feature to at least this post. Here it is:

This Post Contributes to the Endless Self-renewal of Language By:

Promoting Uncommon Verbifications

• to vernaculate: to use the vernacular.

Coining New Words / Nonces

• relaxatiousness: the quality of inducing one to experience a sensation of relaxation.

Also, my article “Gutenberg’s Heir: The Internet as an Agent of Linguistic Change” was published in BYU’s English Linguistics student journal, Schwa, Issue 1. While I’m not quite sure how you can get a hold of a copy, I’m very proud to see my work in print.

So, two big milestones for me in the past two weeks. Hooray for a great end of the semester! Almost as if in celebration, my friends and I all went camping and hiking last yesterday near Hobble Creek, close to Springville.

Cool Flames

The Source

The Real Source (closeup)