New project: bigQuestions.com

http://www.room535.org/bigQuestions.com

bigQuestions.com(2004) is a data-mining and database project, which I hope will be of interest to people interested in databases, code, and language.

There are two parts to the project:

1. It searches web sites and library catalogues for words which signify "big" questions, i.e of philosophy, religion, science. It stores them with their contexts in a searchable database, which at present consists of approximately 65,000 library and 45,000 web entries. Visitors to the site can search the database for terms other than the big terms to see how these terms are used and contextualized in our culture, as often amusing and trivial as they are serious. The web sites where the terms occur are accessible by links as are the library databases, which can then be further searched.

2. There's a "real time" window where the results of the data-mining can be watched, both the raw accumulation of found data and the software code itself as it moves through its paces. The latter is a kind of mini-debugger for the perl script which does the real time searches for raw output. This is raw, in that the data will later be processed for inclusion in the database and then indexed.

I see in this a.m.'s RARE that Lewis LaCook is reporting on his survey about code. I have to confess that one of my reasons for doing bigQuestions.com is love of code. It's been a massive coding porject: the coding of the database and its structures, data-mining software for both web sites and library catalogues, the real time coding which is perl on the server and javascript in the browser, the DHTML interface etc. I started almost 2 years ago when I discovered the perl Z3950 module for accessing over 1000 online library catalogues. But I had to develop a module, now part of the perl Z3950 project, which could query any library, not just ones you were familiar with, without potentially freezing up so that it could be used over the web and for querying up to 1000 libraries also without hanging.

But this obsession with code combines with an interest in language and its cultural contexts. And with an attempt to mirror in a web project the experience of the Internet, or at least my experience. On the one hand, the Internet reaches out into a vast cultural and techological space that can only be imagined. On the other hand, there are the depths, the inner workings, the code. Both kinds of space are hidden from view; yet they intersect in the experience of the person sitting at the monitor. And this intersection has a parallel in the intersection of public and private space that also goes into the making of the Internet experience. The closest analogy to this kind of space is the architecture of large structures, which is also a technological and cultural space that can't be taken in all at once but at best imagined, which also has its hidden technological depths and is the nexus of public and private experience.
I had an exchange about this earlier in the year with Curt Cloninger: http://rhizome.org/thread.rhiz?thread100&text%113