runhello.com

I’ve had this blog for awhile, but barely posted in it. I’m tired of this, and it’s strange because the reason I haven’t posted isn’t a lack of things to say, the problem is just plain not getting around to writing anything. So, an experiment: From now on, I’m going to start making a post here every monday. Even if it’s not particularly coherent or long or I don’t have a specific topic and have to just rant out whatever is in my head, that’s fine. If the resulting posts turn out to be terrible, I’ll just stop doing them or bury them in a box deep beneath the earth or something. In the meantime, I’m just going to make an effort to post something here every week from here on out. So if you know me outside of this site, and a monday comes and you realize I didn’t post anything here, find me and yell at me.

Sound good? All right, here goes.

So I’ve been reading all this stuff about physics lately. I haven’t been doing this for any particular reason. At least, not any good reason; the reason I originally started was that I’d been reading a bunch of stuff about evolutionary biology so that I could argue with creationists, but then arguing with creationists started getting kind of boring, so I started looking around for something else to read.

Once I got started, though, it turned out that this is actually a really interesting time to be following physics.

Fundamental physics has been kind of in a holding pattern for about the last thirty years or so. Somewhere in the mid-70s, every single force, particle and effect in physics except gravity was carefully fit together into a single unified theory called the Standard Model, and past that, physicists kind of got stuck. This isn’t to say nothing’s been done in the last thirty years, mind you– experimentally it has really been a productive time– but all the work that has been done has just served to validate the Standard Model, theoretical physics’ last great achievement, as correct. Nobody’s been able to move beyond or supercede the Standard Model. And physicists really, really want to supercede the Standard Model. To even the physicists that developed it, the standard model has always seemed like kind of a Rube Goldberg contraption; it has all these unexplained fiddly bits and extra pieces that don’t seem to do anything, and it’s not clear why any of the parts fit together the way they do. Scientists have a very specific and clear idea of how this big Rube Goldberg machine works; but they don’t know why it works, or rather, they don’t know why the machine is fit together the way it is.

Theoretical physicists are convinced there’s some kind of underlying order to this that we haven’t just figured out yet, and that all the funny complexities of the Standard Model are just emergent side-effects of something much more fundamental and simple going on below the surface. Theoretical physicists are also annoyed that they haven’t been able to figure out how gravity fits in to all to this– in fact, they can’t figure out how to make gravity work together with any theory of quantum physics. So for the last thirty years theoretical physicists have been going to enormous lengths experimenting with grand unified theories and supersymmetric models and such, trying to come up with a better theory that explains the same things the Standard Model explains (and hopefully also gravity), but in a way that makes more sense. None of these attempts have so far worked. They’ve all turned out to not quite be able to predict the actual universe, or else have predicted the universe but with some weird quirk or other that doesn’t exist in reality, things like proton decay.

The longest-standing of these attempts at unification is something called String Theory, which has been worked on steadily for about twenty or thirty years now. String Theory isn’t really a theory– it’s more like a set of hypothetical theories that share certain characteristics. String Theory is really promising in theory and has gotten a lot of attention because it has the ability to describe a universe which has lots of particles and forces (which maybe include the kinds of particles and forces that the Standard Model contains) and which also has gravity, all based on nothing but simple vibrating stringlike objects following simple rules. But there are some problems, foremost among which is that in practice, nobody’s yet figured out how to use this to describe our universe. Every time people try to put together a string theory that might describe the things we see in our universe, it also describes lots of other things, things like lots of extra dimensions that have never been detected and enormous quantities of new kinds of particles that have never been seen. These isn’t exactly a problem because string theories are very flexible and so every time somebody finds a reason String Theory might not work, they can just modify the theory– adding ways of hiding the extra dimensions and particles, and then hiding the weird complexities they used to hide the extra dimensions, such that even though String Theory still predicts most of these things it predicts that they’d be hiding where we couldn’t find them. This has happened enough times by now that String Theory, which started with just some simple rules, has now gotten very complicated. It’s not clear whether this is a good or a bad thing. This might just mean String Theory is maturing, and so naturally picking up complexity as it moves toward completing an actual descriptive and usable theory. Or it might mean we’re on the wrong track entirely with String Theory, and this burgeoning complexity means we’re taking an originally good idea and torturing it into increasingly unrecognizable shapes, adding more and more epicycles every time something doesn’t quite work right, in an attempt to twist the idea into fitting a universe it fundamentally does not describe. Either way, we’re still no more sure whether String Theory can ever actually work than we were two decades ago, String Theory is still more a toolbox of promising mathematical ideas than an actual working theory, and although String Theory gets most of the attention these days it’s still the case that nobody knows how to move past the Standard Model.

Anyway.

This has all been the state of things up until very recently, but right now at this instant early 2007 some kind of interesting new developments are starting to crop up, and it’s starting to look like this holding pattern may be breaking sometime soon. These new developments are making physics very interesting to follow right now, since they mean we might see some new and very dramatic and unexpected developments in physics maybe even in the next year or two.

The first of these developments, and probably the most directly exciting because it involves things blowing up, is the Large Hadron Collider, a 17-mile concrete tunnel buried underneath Switzerland where, starting at the end of this year or beginning of the next, physicists will be smashing hydrogen atoms together to break them apart so they can look at the pieces. Experimental physics has kind of been in a pattern for the last fifty years where most of the progress is done by building a big machine that fires two particle beams at each other, then recording the little tiny explosions that happen when the beams collide and checking to see whether you know enough about physics to explain why those explosions looked the way they did. Once you you’re confident, yes, you can explain the little explosions from your big machine, you build a bigger machine that does the same thing and repeat. The reason this repetitive mode of experimentation has come about is that the most fundamental physical phenomena are only really detectable at very high energies; in a simplified sense, the more “electron-volts” you have, the more interesting things happen. Every time a new particle accelerator is built that can hit a higher electron-volt threshold, new kinds of particles become visible. This is again a bit of a simplification, but particle physics has been in a lot of ways doing this kind of game of particle leapfrog for the last half-century, where the experimentalists will build a bigger particle accelerator and discover some new particles nobody’s seen, and then the theoreticians will have to come up with a theory to explain what those particles are, but then the new theory will predict some new particles nobody’s ever seen and the experimentalists will have to build a bigger accelerator to look for them. There’s a bit of a problem with this system, though, which is that this pattern is strictly dependent on the regular production of these ever-more-expensive accelerators, each of which takes decades to plan and build and hundreds or thousands of people working together to design and operate. This is kind of a lot of eggs to put in one basket, so when the last big particle accelerator that was supposed to have been built– the Superconducting Supercollider, which was supposed to have been built in the 90s– got cancelled amidst massive mismanagement and governmental budget crunches, it kind of screwed the pattern up. (The cancellation of the supercollider has been kind of a factor in why theoretical physics has been in holding pattern mode lately, and also represented a real identity crisis for particle physicists, who for a long time during the Cold War basically had the ability to get as much money as they wanted from the government, while some other branches of science just begged for scraps, because during the Cold War it was considered a national priority that our giant concrete particle beam shafts be bigger and longer than the ones the Russians had.) In the meantime we’ve been stuck with the Tevatron, finished in the mid-80s, which scientists have just continued to use for new experiments. This means we’ve gotten some good chances to explore everything in the energy range up to the Tevatron’s limit of one terra-electron-volt, but nobody’s been able to look past that.

But now the Large Hadron Collider, which can go up to about 15 terra-electron-volts, is going online, with the first operational tests scheduled (link goes to a big pretty graph!) for October of this year. In the short term, this mostly just means probably a bunch of science stories in the news when this all starts up toward the end of the year. But then once a year or few has passed and the real science starts, physics is going to actually start changing. There’s a lot of possibilities of things that could happen– finding supersymmetric partners or tiny black holes or the other exotic things that String Theory and other unification theories predict. Even aside from getting lucky and finding something big like that, though, the big expected purpose of the LHC is going to be to find a particle called the Higgs Boson, which is the only thing in the universe which is predicted by the Standard Model but which has never been actually seen. Finding the Higgs Boson would be a big deal, among other things because part of what the Higgs does is cause things to have mass, so once we’ve found one and nailed down its properties this might be a push in the right direction for the people trying to figure out how to make the standard model play nice with gravity. The other, unbelievably frustrating but actually probably even more promising possibility is that the LHC won’t find the Higgs Boson. This would be a big deal because it’s starting to look like if the Higgs Boson can’t be found at 15 terra-electron-volts, then it very probably doesn’t even exist, meaning theoretical physicists would have to go back to the drawing board in a potentially big way.

Aside from all this, a second interesting ongoing development in physics is that, while all these problems with accelerators have been going on, a whole bunch of rich and occasionally just plain crazy experimental results (and pretty pictures) have been showing up in astronomy thanks to advances in space-based telescopes. Astronomers have been undergoing a quiet renaissance in the last ten years after the Hubble Space Telescope finally started working, turning up all kinds of stuff that– although physicists haven’t really managed to find any clues forward in it yet– provide some big juicy questions that the next breakthrough in physics might be able to make a go at tackling. The biggest of these is the discovery of “dark energy”, the idea behind which can be summarized as “the universe’s expansion is speeding up over time, and we have no idea why“. It’s not clear whether this recent period of rapid advancement in astronomy will be able to continue much longer, since between the upcoming scuttling of Hubble and the gradual shifting of science research out of NASA’s budget it’s now questionable whether some of even the really promising experiments planned to follow up on the Hubble discoveries will be able to actually get carried out. But even still, we should expect at least a couple upcoming surprises from those experiments that do manage to get launched.

A third, less concrete and more sociological development in physics lately has been the new backlash against String Theory, typified by (or maybe just consisting of) two new and “controversial” books by Lee Smolin and Peter Woit. String Theory has always had detractors, but lately– partially in response to some documentaries in the late 90s where String Theory physicists took their ideas to the public, partially as a result of the increasingly long period of time String Theory has gone now without predicting anything concrete, and partially in reaction to the recent embrace by many string theorists of a really, really bad idea called the “landscape”– a handful of these detractors have started making their case against String Theory in a very public and organized way, and a lot of average members of the public (like, well, me) are starting to take notice. So far this String Theory Counterrevolution doesn’t really seem to have had any real effect on the state of science itself; its chief byproduct seems to have been a series of blog wars between physicists. (Though, if the reason I’m mentioning all of this is to list some of the things that have made physics interesting to follow lately, it’s definitely the case that watching PH.Ds with decades of experience flame each other on their blogs is entertaining, in a pro-wrestling sort of way.) Still, the way this backlash against string theory has played out does seem to hint we may indirectly see some interesting advancements in the near future; many or all of the people speaking out against string theory lately have been doing so under the auspices of arguing actually not that String Theory is outright wrong or should be dropped, but simply with the intent of trying to argue that additional approaches to fundamental physics should be explored as well– many of the string theory backlashers are themselves incidentally proponents of still-embryonic alternate approaches, usually Loop Quantum Gravity in specific. Considering that these arguments are being made in the full hearing of the public and other people who are responsible for the actual funding of physics, there may soon be a bit more oxygen for these alternate approaches for to get the attention and funding needed for serious progress, so we may see some interesting developments coming from them before long.

The last kinda-interesting development in physics of late is the one that actually inspired this blog post and long tangent, which is the emergence of quantum computing as an active research field which may sometime soon actually produce useful machines. Basically, while particle physicists have been busting their asses trying to convince the government to buy bigger magnets and theoretical physicists have been gothing it out about their problems with strings, a lot of applied physicists, people in areas like quantum optics, have just been quietly doing their jobs and, although maybe not working on as large a scale as some of those other groups of physicists, actually getting shit done. One of these busy areas has been quantum computers, which over the last few years has undergone some really dramatic advances, from the first working programs running on a quantum computer in only 1998, to two years later it being a big deal when quantum computers with five or seven qubits were able to factor numbers up to 15, to by this year people getting all the way up to realistic levels like 16 qubits and sudden advances happening in things like memory transfer or gates for quantum computers. This is all a really intense rate of progress– it was not long ago at all that quantum computers only existed as buzzwordy black boxes used by science fiction authors, who assigned them improbable capabilities– and there’s not really any telling what exactly happens next.

Quantum computers work by storing information in the quantum states of various physical things. This is useful because of the weird ways quantum states act. Information in a traditional computer is stored using physical things themselves, like holes in a punchcard, or tiny pits on a DVD surface, or electricity in a capacitor in a RAM chip. Each of these different ways of physically encoding data, when you store a “bit” of information, it’s either a 0, or it’s a 1. Either there’s a pit in the DVD surface, or there’s not. Quantum states, on the other hand, are not necessarily one single thing at a time; when you store data as a quantum state, what you instead store is a probability that that quantum bit (qubit) is 0 or 1. This means that when you store a number in a qubit register, you don’t have to store just one number in there; you can store lots of different numbers, as superpositions of probabilities. This leads to lots of crazy possibilities, since it means a program running on a quantum computer can, in a sense, do more than one thing at the same time. In another sense, this is just fakery; you’re not actually doing more than one thing at once, you’re just probabilistically choosing one of a certain number of possible paths the program could have taken. This leads to another weird property, that quantum computers don’t always return the right answer– they at best just have a certain known probability of returning the right answer. But probabilistic algorithms actually are a real thing in computer science, and can be fundamentally faster than traditional ones, so this is actually okay: you can run a quantum algorithm over and over and average the result, and you’ll still get the right answer way faster than you would have with a traditional computer.

The thing is though that working with quantum states as quantum computers do is actually really, really hard. Quantum superposition is usually described to people using a thought experiment with a cat in a box and lots of big macroscopic objects like geiger counters and vials of poison, but this is mostly just a metaphor and it’s never anywhere near that easy to set up. There’s this “observation” in the Schrödinger’s cat thought experiment that collapses the waveform and forces the cat to be either alive or dead; this is usually described as “opening the box”, but actually any interaction whatsoever between the systems inside and outside the box, including stray heat propagation or vibrations in the air (say, from the cat yelling to be let out of the box) while the box is still closed, is enough to entangle the systems and collapse the waveform. Theoretical physicists doing thought experiments may get to gloss over these little details, but when doing quantum computing, these little details become incredibly practical and immediately relevant– because they mean that if the qubits in a quantum computer interact with their environment, ever, in any way, then the quantum state is destroyed and your computation is erased. This means that whatever things hold the qubits in your computer have to be effectively totally isolated from quantum interference of any kind, and continue to be so while doing gobs of things that normally no one would want, need, or try to do with a quantum state, like transport or copy it. Figuring out how to do this, and all the other things that quantum computing entails, means really stretching the edges of what we think we know about quantum physics, and means that quantum computing has actually become something of a hotspot for theoretical physics researchers at the same time that “normal” theoretical physics has been struggling to progress. I’m only able to perceive all of this from a far distance, of course, but still it’s been interesting to watch.

It’s been particularly interesting to me, though, because at the same time theoretical physics is getting a workout in the applied problem of building quantum computers, something else is getting a workout in the related applied problem of what to do with the quantum computers after they’re built: theoretical computer science. Theoretical computer science and computability theory is something I actually personally know something about (unlike, it is worth noting, quantum physics), and it’s incidentally a topic which gets very little attention. Computer science professors often do their best to convince the students passing through that things like turing machines and lambda calculus actually matter, but people don’t get computer science degrees to do computer science theory, they get computer science degrees to get a job programming– and in a day to day practical sense, aside from an occasional vague awareness of the existence of “big O” notation, the vast bulk of people doing programming have little use for or awareness of computation theory. There are specific domains where this stuff is still directly useful, and there were times in the past where this stuff was still directly practical and relevant, but on average, outside of pure academia, the underlying theoretical basis of computer science these days practically seems to mostly only exist for the construction of intellectual toys for geeks like me who apparently don’t have anything else to do. Quantum computing, however, means that all this foundational computer science stuff not only has become suddenly very relevant and useful, but also suddenly has a bunch of new and relatively accessible open questions: As far as I’m aware no quantum algorithms even existed before 1993, and there’s still a lot of important questions to be answered on the subject of, once quantum computers “work”, what the exact extent of their capabilities might be and what we should do with them.

So to me personally, quantum computing is kind of interesting on two levels. One, if I’m going to treat the science of physics as a spectator sport, then from a spectator’s perspective quantum computers are one of the more interesting places things are going on right now; two, because of the connection to pure computer science, I or other people familiar with computers or programming actually have a chance of directly understanding what’s happening in quantum computers, whereas with, say, the Higgs Boson (which really? I’ve seen no fewer than five distinct attempted explanations of the Higgs Boson now, and I still have no clue what the darned thing really is), we’re left mostly only able to read and repeat the vague summaries and metaphors of the people who’ve worked with the underlying math. And then there’s a third thing I can’t help but wondering about in the interaction between these two things: I’m wondering if someone like me who already understands computers but doesn’t understand physics might be able to use quantum computing as a way of learning something about quantum physics. Real quantum physics takes years of really hard work and really hard math to learn, and most colleges don’t even let you at this math until you’ve already dropped a significant amount of time investment into the low-level, more accessible physics courses; on the other hand, if you’re outside of a college and trying to learn this stuff on your own, it’s very hard to find sources that really tell you directly what’s going on instead of coddling you in layer after layer of increasingly elaborate metaphors. Metaphors of course are and should be a vital tool in learning physics, but when you come right down to it physics is math, and glancing at most materials on quantum physics, I tend to get the feeling I’m learning not quantum physics, but just learning an elaborate system of metaphors, which interact with all the other metaphors as archetypical symbols but are connected to nothing else. On the other hand, stuff on quantum computing has to actually work with the real-world nuts and bolts and math of quantum physics (unlike popular books on the subject), but because so much of it has to be worked with by computer scientists, non-physicists, at least some of it is surely going to be necessarily aimed at a lower level of background knowledge (unlike learning this stuff at a college would be). One would have to be careful not to fool oneself into thinking this could actually be a substitute for a full physics education, of course, but from the perspective of an interested spectator I can’t help but wonder if this might be a good way of sneaking into the world of higher-level physics through the back door.

This is all a roundabout way of leading up to saying that I’ve been looking into some stuff on quantum computing, and I’m going to try to use what little I’ve worked out so far to try to make a little toy model computer simulation thing, which I will then post about it here, on the internet, along with some hopefully psychedelic animated GIFs the simulation generated. If I’ve kind of got things right, then putting this post together in a form other people can hopefully understand will help me get my thoughts in order and test what I’ve got so far (I’m allowing myself the presumption of assuming anybody’s still reading at this point); if I’m just totally confused and wrong, meanwhile, then hopefully somebody who understands this stuff will eventually find this and point my mistakes out. A first little attempt at this is below.

Specifically, I’m curious whether one could define a quantum version of Conway’s Game of Life.

Conway’s Game of Life

“Life” is this thing this guy named John Conway came up with in the 70s. It’s referred to as a “game” because originally that’s more or less what it was; Conway originally published it as a brainteaser in Scientific American, intended to be played by hand on graph paper, with the original challenge being a $50 reward to the first person who could devise a Life pattern that continued growing endlessly no matter how long the game went on. Conway at first expected this was impossible; he was horribly, incredibly wrong. (An example of such a pattern is at right.)

Despite these fairly simple origins, Life turned out to be sort of the tip on the iceberg of an entire category of mathematical systems called “cellular automata”, which are now a recognized tool in computation theory and which have received by now a fair amount of attention; Stephen Wolfram, the guy who created Mathematica, spent ten years writing a very long book all about cellular automata (a book which, although I’ve seen a lot of doubt as to whether anything in the book is actually useful to any domain except computation theory, is universally agreed to contain lots of pretty pictures). Life itself has remained the most famous of these; today there are entire websites devoted to categorizing Life patterns of every kind you could think of, and most people with an education in programming have been forced to implement a Game of Life program at one point or other.

- - - - - 1 2 3 2 1 - O O O - 2 3 4 2 1 - O - - - -> 2 4 5 3 1 - - O - - 1 2 2 1 0 - - - - - 0 1 1 1 0

On the left is a grid of cells containing a glider; on the right is a grid containing the count of living neighbors that each cell on the left has. I’ve colored light red the dead spaces which in the next generation will be new births, dark red the live spaces which in the next generation will survive, and light blue the live spaces which in the next generation will die. Let this keep running a few more generations and you get the glider:

The term “cellular automata” just refers to systems that can be described by patterns of colored dots in “cells” (i.e., grid squares or some other shape on graph paper) and which change over “time” (i.e., redrawing the system over and over on pages of graph paper) according to some simple rule– in all variations of cellular automata I’ve ever seen, how a cell is colored in one frame depends on how its neighbors were colored in the previous frame. In the original Conway’s Life, the rule was that a cell is colored black in a new frame if

• in the previous frame it was colored black and was touching either three or four cells colored black, or
• the cell was colored white in the previous frame but was touching exactly three cells colored black.

Conway intended this as a sort of population model (of “life”) where black dots represented living things; black dots disappearing when they have too few or too many neighbors represent that lifeform dying of either loneliness or starvation, black dots appearing when they have a certain number of neighbors represent reproduction.

These rules favor non-life over life, so if you just fill a board up with randomly placed live and dead squares, within a couple of generations most of the life will have died off or settled into tiny static patterns, like little solid blocks or little spinners eternally looping through a two-frame animation. But you’ll also have little areas that wind up filled with sloshing waves of chaos, breaking against and devouring the static bits. If you watch this carefully, you’ll notice that chaotic as these waves look, the way Life patterns act when they collide with one another is actually very specific and structured. The systems in Life are chaotic enough that predicting what a random Life pattern you come across is going to do is just about always going to be impossible unless you sit down and actually run it. But if you sit down ahead of time and work out a catalog of patterns that you know what they do and how they interact with other patterns, and then stick to placing those patterns in very carefully controlled ways and places, you can actually use these smaller parts to design larger life patterns that have basically whatever behaviors you want.

You can build machines.

Life is an example of something we call a “model of computation”, which is a name we give to systems– machines, language systems, board games, whatever– that can be one way or another tricked into running some kind of “program” that’s dummied up for it, such that after a while its changes in state have effectively computed some sort of result. The kinds of computational models we normally care about are the ones that we say are “Turing complete”. This is a reference to the first ever proven universal model of computation, the Turing Machine devised by Alan Turing in 1936. The Turing Machine is a hypothetical computer, described as a sort of little stamping machine on wheels that slides back and forth on an infinite tape reading and rewriting symbols according to a little set of rules it’s been wired up with. The Turing Machine was not meant to ever actually be built; it was designed just to be easy to reason about and demonstrate interesting things in mathematical proofs. The most interesting thing about the Turing Machine is that it is in fact universal; there’s a generally accepted but technically unproven principle called the Church-Turing Thesis that any programming language, computer, whatever, that you can think of, can be emulated on a Turing Machine. This makes the Turing Machine suddenly really important as a sort of least common denominator of computers, since if you have a machine that can emulate a Turing Machine– in other words, a machine which is Turing complete– then that machine can inevitably emulate anything a Turing Machine can, including every other Turing complete machine, as well. Turing Complete models of computation are as common as dirt– they include every computer and programming language you can think of, as well as some really bizarre and deceptively useless-looking mathematical systems, a couple of which Conway invented— and they’re all approximately the same, because they can all do exactly those things the other Turing complete systems can.

The Game of Life is, as it happens, Turing complete, as was quite dramatically demonstrated a few years back when some guy actually built a Turing Machine that runs inside of a Life system. The Life Turing Machine is a machine in a hopelessly literal sense; viewed zoomed out, it looks like some kind of aerial photograph of some steam-powered contraption that a man from the 1800s with a handlebar moustache would demonstrate, after dramatically pulling off the canvas that covered it. The thing isn’t particularly efficient; if you for some reason took the time to make an emulator for an 386 Intel PC out of Turing machine rules, and then ran this emulator on the Life turing machine, I imagine the heat death of the universe would probably come before you managed to get Minesweeper loaded. But efficiency isn’t the important thing here: it works, and that’s what matters.

Something worth noting is that, although I’ve already claimed that turing machines are approximately the same as any computer you can think of, this isn’t quite true– quantum computers actually aren’t included in this generalization. Normal computers can emulate quantum computers, but only with unacceptable amounts of slowdown (insofar as certain kinds of proofs are concerned), and there are things quantum computers can do but normal Turing-complete machines outright can’t, specifically generate truly random numbers. This isn’t a problem for the Church-Turing thesis, exactly: We can just get around this by saying that the Church-Turing thesis is only supposed to describe deterministic machines, which quantum computers aren’t. Despite this, we still have to have some kind of least common denominator to reason about quantum computers with, so we have this thing called a Quantum Turing Machine that vaguely resembles a Turing Machine but is able to provide a universal model of computation for quantum computers.

So, all this in mind, here’s what I wonder: The Turing Machine is a universal model of conventional computation, and you can make minor modifications to the turing machine and get something that’s also a universal model of quantum computation. Life is also a universal model of conventional computation; you can make a turing machine in it. Is there some simple bunch of modifications that can made to Life that make it a universal model of quantum computation?

I originally started wondering about this in the context of Loop Quantum Gravity, something I mentioned earlier as an “alternative”, non-String theory of Physics. Loop Quantum Gravity is supposed to be a theory of Gravity that incidentally follows the rules of Quantum Physics. This is something that is understood to be either extremely difficult or impossible, for a couple reasons, one of which is that gravity has to play nice with the Theory of Relativity. The Theory of Relativity plays hell with quantum theories because it says the geometry of the universe is bending and warping all the time like silly putty, which makes the quantum theories scream “I CAN’T WORK UNDER THESE CONDITIONS!” and storm out of the building. Loop Quantum Gravity, at least as I understand it, gets around this in part by quantizing geometry itself– that is, it makes the very geometry the universe exists inside subject itself to a form of quantum behavior. One of the consequences of this as I understand it is that the geometry of the universe becomes discrete, which kind of means that it can be represented as if it were a big interlocking bunch of (albeit really weirdly shaped) cells on graph paper. In other words, you could if you really wanted to treat the geometry described by Loop Quantum Gravity as a board to run Life on. If true, this seems like an interesting idea to me because Loop Quantum Gravity is really hard to find any remotely accessible information on and all the information that is available is super high level math, so it seems like something like Life or some other cellular automata running on the nodes of a LQG spin network would be a fun little visual demonstration of how the thing is supposed to work. But before you did that, I assume, you’d have to have some kind of conception of what a cellular automata in a quantum universe would work like at all.

A Quantum Game of Life

Conveniently, creating cellular automata with quantum behavior isn’t at all a unique idea; looking around I’ve found that people have been working with QCA models almost as long as they’ve been working on Quantum computers, and several conceptions of cellular automata with quantum behavior are already around. I’m going to list them here, since I used some of them in formulating what comes after.

• First, there’s the universal quantum turing machine itself. Mark Chu-Carroll has an informal but very clear explanation here of how the quantum turing machine conceptually works; you can also find the original paper that first defined the QTM online under the title Quantum theory, the Church-Turing principle and the universal quantum computer. It’s a bit of a heavier read, but the introductory parts are plain readable everyday English and interesting for their own reasons.
• On top of this, there’s actually real ongoing research into what are called “quantum cellular automata” or “quantum dot cellular automata”. If you’ll look into these, you’ll find they seem to mostly consist of systems of little objects that look like dice arranged next to each other like dominoes, such that the painted dots in any individual die change color over time depending on how the dots in the closest adjacent die are colored. I’m not exactly sure how these things work, but I do notice two things. One, this version of cellular automata looks very serious and formal and appears to have real practical application in actually simulating quantum systems. Whereas what I’m hoping to make here is more a silly little toy program that makes pretty pictures. Two, doing anything in this version of QCAs seems highly dependent on how you arrange the tiling of the dominoes against each other, whereas I’m really hoping for something where the behavior is dictated by the way the “lifeforms” are positioned and not by the way the external geometry is preconfigured.
• Some guy named Wim van Dam in 1996 actually wrote an entire Master’s thesis on the subject of quantum cellular automata, under the simple title of “Quantum Cellular Automata“. Van Dam’s thesis there is actually well worth reading if you’re finding any of this so far interesting, and explains the motivations and workings of quantum computers, cellular automata, and hypothetical quantum cellular automata all way better than I do here– probably to be expected, since it’s an entire master’s thesis, instead of just a “this is how I wasted my Saturday” blog post. Potentially extremely conveniently for my purposes, Van Dam actually describes specifically how to formulate a one-dimensional cellular automata of the exact kind that the Game of Life uses. He does not, however, bother generalizing this to two dimensions as would allow one to implement the game of life in specific, since of course the one-dimensional version is sufficient for his goal of proving the QCAs work.
• Google incidentally turns up a short and concise 2002 paper titled “A semi-quantum version of the Game of Life“. Looking over this, however, it doesn’t appear to be exactly the thing I’m looking for (i.e. a reformulation of Life with quantum style rules). Instead, the paper suggests that such a thing is probably not a good idea, noting “A full quantum Life would be problematic given the known difficulties of quantum cellular automata” (uh oh), and instead takes the opposite tack, of trying to design a quantum system that can simulate life: specifically, they design a way of creating systems of quantum oscillators which interfere with one another in a way which “in the classical limit” exactly follows the rules of life, but which otherwise exhibits various interesting quantum mechanical properties as a result of the oscillators it’s made out of. Which is all interesting, but it isn’t quite what I was looking for (at least I don’t think so), and it also requires one to understand the quantum harmonic oscillator (which I don’t).

None of this exactly provides what I was looking for, since all of the attempts above, as it happens, are by people doing actual physics research. They’re thus doing their work under the constraints of a set of fairly serious goals, goals like applicability in proving things about quantum physics or even implementability as a real-world device. When I think about deriving a version of quantum Life, though, I’m just thinking of something that would be easy for me to implement and useful and fun for demonstrating things. This means that if I’m going to try to construct my own version of quantum Life, I’m going to be working off a somewhat more modest set of goals:

1. Just as in the original game of Life, whatever the basic rules are, they should be simple to understand and implement even to someone who doesn’t necessarily understand quantum physics and turing machines and such.
2. Also like the original game of life, running a simulation of quantum life should produce pretty pictures.
3. And also like the original game of life, the way patterns evolve should exhibit some sort of structure which could be potentially used by someone with lots of time on their hands to design starting patterns with arbitrary complex behaviors.
4. The game should incorporate or exhibit some kind of behavior of quantum mechanical systems. Hopefully the quantum behavior should incidentally impose some kind of interesting limitations or provide some kind of interesting capabilities to someone trying to design starting patterns that wouldn’t be the case in standard Life. In the best of all possible worlds it would be possible to demonstrate some interesting feature[s] of quantum mechanics by setting up quantum Life patterns and letting them run.
5. Ideally, hopefully, this simple quantum life should turn out to be a model of quantum computation– that is, it should be possible to simulate a quantum turing machine and thus emulate any quantum computer using this quantum life. Since (the Wim Van Dam paper proves this) it is possible to simulate a quantum turing machine on a universal quantum cellular automata and vice versa, it should be sufficient to show any of the “real” QCAs described above can be implemented or simulated as patterns in quantum life.

My first attempt at this follows; to test it, I did a quick and dirty implementation in Perl. Note that although I’m pretty confident of everything I’ve written in this blog post up to this point, after this point I’m in uncharted territory and for all I know everything after this paragraph is riddled with errors and based in gross misunderstandings of quantum physics. In fact, since as I type these words I haven’t actually started writing any of the Perl yet, I have no idea if this quantum Life thing is even going to work. (Though even if it doesn’t, I’m posting this damn thing anyway.)

(Update, Feb 2009: So, I wrote this post a couple years ago when I was first trying to learn about quantum physics and quantum computing, and I actually did make a couple mistakes. The one really huge mistake I made occurs a few paragraphs back, and it propagates forward into the part of the post that follows. The mistake is this: I claim in this post that a qubit in a quantum computer can be described by the probability that that bit is 0 or 1. This is close, but not right. A better way to put this is that a qubit can be described by a quantum probability that the bit is 0 or 1. What is the difference? Well, a normal probability is just a number between 0 and 1– like, 30%. However a quantum probability is a complex number– a number with an imaginary part– of absolute value less than one. This unusual notion of a “probability” turns out not only to make a big difference in the behavior of quantum systems, it’s the entire thing that makes quantum computers potentially more powerful than normal computers in the first place! Unfortunately I didn’t clearly realize all this when I wrote this blog post. So in the final part of this post, I try to define a “quantum” cellular automata– but since at the time I misunderstood what “quantum” meant, I wind up instead defining a cellular automata model which is in fact only probabilistic. Now, given, I think what follows is still kinda interesting, since the probabilistic cellular automata turns out to be kind of interesting unto itself. And it’s definitely a first step toward quantum cellular automata. But, just a warning, it’s not itself quantum. I did later took some first steps toward implementing an actual quantum cellular automata, but they’re not done yet. So hopefully someday I can revisit this and write another post in this series that gets it right this time.

If you want a clearer explanation of this whole “quantum probabilities” thing, I suggest you read this, which is about the clearest explanation of what the word “quantum” means I’ve ever found.)

The most logical way to construct quantum life, at least as far as I’m concerned, is to just take the trick used to create the QTM and apply it to the Life grid. In a Post machine [2-state Turing machine], each tape cell is equal to either state “0” or state “1”; in the QTM, the tape cell is essentially equal to a probability, the probability that the tape cell contains state “1”. This simple conceptual change, from states to probabilities of states, provides basically all of the QTM’s functionality. In Life, meanwhile, as in the Post machine, each grid cell is equal to either state “0” or state “1”; a naive quantum Life, then, could be created by just setting each grid cell essentially equal to a probability, the probability that the grid cell contains state “1” (“alive”). This is the tack I’m going to follow. I’m going to speak of cells below as being “equal” to a number between 0 and 1; when I do this, the given number is the probability, at some exact moment in time, that the cell in question is “alive”.

The introduction of probabilities like this even just by itself offers a lot of opportunity for the creation of new and elaborate cellular automata rules, say rules where each varying number of neighbors offers a different probability of survival into the next generation; I imagine the people who’ve already done so much elaborate stuff with just the addition of just new discrete colors to Life could have a field day with that.

I’m going to ignore those possibilities, though, and just maintain the rules from “Real Life” entirely unchanged– that is, a cell is created iff it has three neighbors, a cell survives iff it has three or four neighbors. These rules still work perfectly well even though now some cells are only “probably” there, and conveniently, this selection of rules means that this kind of quantum Life is in fact completely identical to normal life, so long as all the cells are equal to exactly 0 or exactly 1. The simulation only behaves differently if at least one cell, from the beginning, is programmed to be “indeterminate”– that is, to have some probability of being alive between 0 and 1. When this happens, then we can no longer calculate an exact number of neighbors for the cells in the vicinity of the indeterminate cell. We deal with this situation simply: instead of calculating the number of neighbors, we calculate the probability that the number of neighbors is “right” for purposes of the cell being alive in the next generation. That probability then becomes the cell’s “value” in the next generation. For example, let’s say an empty cell has two neighbors equal to 1, and one neighbor equal to 0.5. This empty cell has a 50% probability of having 2 neighbors, and a 50% probability of having 3 neighbors. Since the cell needs exactly three neighbors to reproduce, the probability of the cell being alive in the next generation will be 0.5.

The questions that immediately come to mind about this way of doing things are:

• Does the addition of “indeterminate” states actually have any interesting or useful effect?
• Is the addition of these indeterminate states to Life sufficient to allow the emulation of a QTM or UQCA?
• Does this actually have anything to do with quantum physics?
• Is this different in practice from the “semi-quantum life” linked above?
• Though one of my goals is for the rules to be simple, and they seem simple to me, I took three long paragraphs to describe these rules. Is the way I described the rules at all clear, and maybe is there some way I could have just summarized them in a couple of sentences?

(As far as the second question here goes, it occurs to me that what I’ve designed here may well not really qualify as “quantum life”, and maybe “probabilistic life” would be a better way of describing it. In fact, looking on Google for the phrase “probabilistic game of life”, I now find that a couple of people who’ve attempted rules along these lines: There’s this little writeup on a math site here, which appears to be describing a version of probabilistic Life with my exact rules. This author doesn’t provide an implementation, though, so I shall press on, although that site does provide some interesting proofs about the behavior of a probabilistic life that I’m not going to bother trying to understand just now. On an unrelated note, Google also turned up this odd little thing, though what it actually specifically has to do with the Game of Life is not obvious from the link.)

Anyway, let’s look at an actual implementation of this kind of quantum life and see the answers to my questions here are at all obvious.

I happened to have this simple implementation of Life sitting around I’d made awhile back, as a perl module that takes in grids containing Life starting states and spits out animated GIFs. This particular implementation was written such that on each frame, the module would make a table listing the number of neighbors each cell in the grid has, then use this number-of-neighbors table to generate the alive and dead states in the next frame. Modifying this existing module to work with quantum life turned out to be incredibly simple: Rather than the number-of-neighbors table being a simple number, I just switched it to being a probability distribution, an array of ten values representing the probabilities that that cell had zero through nine neighbors respectively. I build up this array by starting out with a distribution giving 100% probability of zero neighbors and 0% probability of any other number, and then factoring in the changes to the probability distribution caused by each of the cell’s nine neighbors, one at a time:

0 neighbors: 1 neighbor: 2 neighbors: 3 neighbors: 4 neighbors:

=

1 0 0 0 0

+ 100% alive neighbor

0 1 0 0 0

+ 100% alive neighbor

0 0 1 0 0

+ 50% alive neighbor

0 0 0.5 0.5 0

+ 50% alive neighbor

0 0 0.25 0.5 0.25

(If you squint real hard, you can probably see the binomial distribution in there.)

The probability of the cell being alive in the next generation is then calculated by

(probability of cell having 3 neighbors this generation) + (probability of cell being alive this generation)*(probability of cell having 4 neighbors this generation)

What happens when we run this?

Well, at first glance at least it doesn’t appear to work all that well. Here’s what happened when I took a simple glider and placed it on a quantum Life board, in each case varying the probability that the glider was “there” (and, in the last couple cases, varying the probability that each of the “empty” cells were actually filled). In this table I show first the starting condition, and then an animated GIF showing what happened when the Life simulation ran. In each frame, the darker the pixel, the more probable it is that there was a living cell there at that exact moment.

100%	50%	75%	90%	100%	0.001%

The 100% alive glider, as expected, behaves just as it would in normal life. The other gliders… well… just kind of instantly explode. The 90% glider manages to actually on average complete a couple of halting steps before it falls apart entirely, the poor thing, but it too smears out of apparent existence within a few more frames. This in retrospect isn’t actually surprising. When the gliders “disappear”, they’re not outright gone; instead, the blank area where they once were (and a gradually spreading portion of the rest of the blank area, as well) is equal to a bunch of incredibly low probabilities, so low they’re indistinguishable from pure white. This happens because when we run a game of Life containing indeterminate states, we’re basically simulating every single possible board configuration that could have existed at once, with some just being more likely than others. The Life rules favor non-life over life, so the vast majority of these possible games wind up just rapidly spinning out into nothingness– or, at least, the possible games that wind up containing some living cells left don’t have the life settling in some areas any more prominently than others, so the blankness is able to dominate essentially without opposition.

So this version of Life doesn’t at first glance seem to turn out to be very useful in terms of creating a usable or playable game. We could maybe fix the “everything dies instantly” problem by fiddling with the rules some in the next version, though, so maybe things aren’t hopeless. In the meantime, does what we have so far at least demonstrate quantum behavior? Well, it does seem pretty clear that the life cells participate in self-interference, at least from informal observation:

Starting state	Center line is 100% there	Center line is 99.99% there

At least self-interference is the way I interpret the sort of gradienty behavior the version on the right engages in before it fades out entirely. The probabilities don’t do that whole square-of-complex-numbers thing, and there isn’t anything resembling observation or wavefunction collapse (I could change the GIF-making function such as to basically assume the wavefunction collapse happens on every frame, by randomly populating the gray areas with life instead of displaying the probability distributions, but that would probably just make things visually confusing without conveying anything interesting), but (although I could be wrong about this) the quantum turing machine doesn’t as far as I can gather really have any of these properties either so we could maybe just shrug that off.

Even so, there really doesn’t seem to be any way you could make a turing machine out of parts that keep fading out of existence, so after encountering this everything-fades-away problem I was just about to give up and declare the whole thing a wash. Still, turing machines aside, I did manage to at least find one interesting thing about this particular quantum life implementation. We see the first glimpse of this when we try populating a Life board at random:

Okay, now here there seems to be something going on. You’ll notice that most of these areas actually don’t fade right to white, but instead stabilize at what looks like a sort of 50% gray (although, while they look like it, the stable half-gray spots aren’t 50/50 gray– each of those cells are about 34.7583176922672% there. I don’t know why it’s that particular number, but I assume it has something to do with the tension between the “survivable” thresholds of three and four neighbors.) of probability; apparently there’s some kind of sweet spot there where life boards filled to a certain density of living cells tend to keep that density more or less consistently on average. A few little bits of area stabilize toward zero, though, and the areas of zero gradually eat away at the areas of half-life until there’s nothing left. So that’s kind of interesting. But the one thing that suddenly and out of nowhere justified this entire silly little project to me is what happened when I randomly populated a board, and then “rounded” most of it– that is, I rounded everything on the randomly generated board to exactly 1 or 0, except for a thin little strip of indeterminacy running vertically down the right side. And I got this:

…WTF?

What happens here is that the “probably there” areas converge on the mysterious 34.7% half-life value and then just kind of spill outward, swallowing up all the actually-living cells as they go. Then, once all the living matter has been devoured and no food is left, the white starts eating in and the areas of half-life slowly die out. In other words…

Holy crap! It’s a zombie apocalypse simulator!

So, I’m not sure this entire exercise was altogether productive in any way, though I’m going to keep experimenting with all of this– maybe trying some alternate Life rules besides just Conway’s 23/3– to see if I can coax anything useful out of this quantum Life concept. Check again next week to see if I come up with anything. In the meantime, though, at least, I think that the final image makes the whole thing satisfying to me in two ways:

1. I at least got my psychedelic animated GIFs
2. I tried to test quantum physics, and instead accidentally created zombies. Does that mean I qualify as a mad scientist now?

If you want to run any of this stuff yourself, you can find the messy little quantum life module here and the script that generated all the images on this page here. You’ll need to rename those files to qli.pm and gliders.pl, and you’ll need Perl and GD::Image::AnimatedGif to run them; and to do your own tests you’ll have to modify gliders.pl. If you need any help with that, just ask below.

So last month, just before the elections, I was thinking about electoral shifts. With everyone pretty much convinced that the Democrats were about to take over Congress, or at least the House, I saw a lot of people making comparisons to the 1994 Republican takeover of Congress, and I saw one person make the claim that the 1994 Republican takeover wasn’t really that big of a shift compared to previous Congressional swings earlier in American history.

This made me curious. I started to wonder how one might go about judging such a thing, and started to realize that although detailed histories of the U.S. presidency abound, there really is not very much well-organized information out there about the historical makeup of the Congress.

I decided to solve this the way I solve most problems in my life: by making animated GIFs. I downloaded the rosters of the first through 109th congresses from the Congressional Biographical Directory, and then a month later, when things had settled down a bit, added the information from Wikipedia’s tentative listing of the newly-elected 110th congress. Then I wrote some perl to convert the Congressional rosters to graphs, with one colored pixel marking the party which held each seat in each of the 110 elected congresses. You can find the results below.

For starters, here’s just one big graph of everything, sorted from top to bottom by state, with senate and house seats separated. As with any of the images in this post, if you want to see it closer, you can click to zoom in:

Although this graph is to some extent cryptic since it doesn’t tell us exactly why any of these pixels change colors with time, if you look closely you can actually see many of the important events of American history reflected quite visibly in this graph. For the most obvious example, the rise and fall of the Federalist and Whig parties are clearly visible in the early part of the graph as big blobs of purple and green, accompanied by a wave of gray pixels around the 1820s, just before the Whigs appeared, marking the collapse of the Democratic Republicans before the party was reborn under Andrew Jackson. A solid line of gray pixels is also visible at the beginning of the graph, marking those heady few early years of American politics before any political parties existed at all. The Civil War is clearly visible as a long vertical black streak cutting through the graph around 1860, marking the period when the southern states simply didn’t participate in the Congress. After the Civil War most of the southern states turn very briefly red, then blue again, as blacks suddenly gained the right to vote, then lost it again with the rise of Jim Crow and the Ku Klux Klan. After this point the “solid south” phenomenon becomes incredibly marked, with the northern states in the graph a patchwork of red and blue pixels, but the southern states a solid sea of blue for a hundred years as a result of post-Reconstruction animosity toward the Republicans. In the decades after the 1950s, the great northern/southern swap as the Democratic and Republican parties in many was reversed themselves is visible as a great gradual blur of colors swapping, followed by a solid wall of change around 1994– Massachusetts and Texas are almost mirrors of one another in this period, with Massachusetts slowly turning from nearly solid red to solid blue, and Texas doing the same in reverse.

When we look at the graph by states this way, of course, shifts in Congressional control— which is more of a numbers game– are not so clear. Some of the big shifts are visible– for example stripes of red and blue are clearly visible around the beginning of the 1900s, as first the Republicans sweep congress during the Spanish-American War, then the Democrats sweep congress during the Great Depression and WWII. The 1994 Republican Revolution is visible in some states, but not others– and in those places where it does occur, it seems less like a solid switch than just an acceleration of the steady progression from blue to red in many places of the country that followed Nixon’s “southern strategy”, and the steady emergence of the “red state” phenomenon. The 2006 elections– the last column of pixels on the right– is barely visible at all.

The shifts become a little more clearly visible if we choose not to sort by state:

In the graph on the left here, sorting still occurs by state, but rather than being separated neatly the states are all just mashed together. This graph is a little hard to make sense of. More clear is the graph on the right, where pixels are instead sorted by party. Here the shifts in congressional control are quite blatant; very brief swings in power, like the Democratic powergrabs following the Mexican-American war and the Watergate scandal, become easier to see, and it’s easier to see which numeric swings were lasting and which weren’t. The “Republican Revolution” is a lot more visible on this graph than on any other, and at the very end of the graph, someone familiar with the politics of the last decade can almost chart the rise and fall of the Republican congressional majority pixel by pixel: Republican control spikes like crazy in 1994; then drops off just a little bit as voters become disillusioned with the Republicans in the aftermath of the impeachment circus; voters then warm toward the Republicans again in one final one-pixel spike, representing the halo effect of Bush’s 2004 campaign; then suddenly the numbers swing toward the Democrats again in that last final rightmost pixel.

One thing that stands out to me in this particular graph is that though the swing towards the Democrats in 2006 is quite pronounced, it’s certainly not nearly as pronounced as the swing that put the Republicans in power in 1994. Although the Democrats still hold a decent majority, and it looks like they’re about on par with where they were at what looks like the beginning of the Reagan revolution, they don’t hold nearly as much power as most of the historical Democratic majorities since FDR have. Although there are other reasons besides pure numbers to think that in this particular election the voters meant to send a message– although it’s not really visible in any of the graphs above, one of the interesting facts about the 2006 elections is that no congressional seats or governorships held by the Democrats went to the Republicans on 2006, only the other way around– in terms of pure numbers the 2006 elections were not really that big of a shift, and the Democrats are only really halfway to replicating the feat that the Republicans pulled off in the 90s. If nothing else, this means that the Democrats are going to have to govern carefully to keep control of the situation with their relatively thin majority– and will have to really convince the voters they’re doing something worthwhile with that majority from day one, because it will not take much to lose it all in 2008.

These graphs aren’t perfect. The chief problem with them is that they aren’t exactly sorted by seat. The data that I’m working off of here doesn’t show who serves in which district, only who served in what state. This means that if someone holds a particular congressional seat for 20 years, they’ll show up on the graph as a solid line of 10 horizontal pixels– but their replacement for that same seat won’t necessarily be in the exact same horizontal position as they were. Also, I don’t have records of who won the elections– Congress’s listings only showed who served during each two-year period, so if more than one Congressperson occupied the same seat during some period (for example, because one of them died and was replaced with the second), both show up in the graph. (This is why, although each state only has two Senators, many of the “Senate” lines in the by-state graph at the top are occasionally taller than two pixels.)

What I’d be curious about doing with these graphs in future is getting hold of some more specific data concerning who served in exactly which Congressional district, so the graphs can more accurately reflect the shifts within states– for example, so that if most of a state votes one way, but there’s one specific city or region that consistently votes another, it would be clearly visible. It also might be interesting, with that information in hand, to try to rework some of these graphs as colored maps, although I’ve never found a good way of making maps in software. Another interesting possibility might be implementing some sort of mouseover feature, so that by moving the cursor over any particular pixel you can see the name of the person that pixel represents.
The other thing that I’d like to try to fix about these graphs, though I’m less sure how is that they’re kind of a lot to take in all at once– they’re too tall to fit on a computer monitor, and without zooming in a lot of the features are hard to make out. This is a little bit helped on the graph that I think is my favorite, since it serves very well as a kind of “summary”– the graph where House reps are ignored and only the Senate is displayed. On this graph we get a good general idea of how people are voting but the graph is still small enough to take in at a glance, so the nature of the big party shifts by region and event are most “obvious”:

If anyone has any other suggestions for ways that these graphs could possibly be improved, I’d be curious to hear them.

As one final bonus, here’s an animated graph, with columns of pixels from left to right representing states:

So: I have made this website. You can find it at http://datafall.org/.

The idea of datafall.org is that if you have a website that is something like a blog– like, a copy of WordPress or Movable Type, or a LiveJournal, or a Blogspot account, or a MySpace page, or basically anything that uses RSS– you can add it to Datafall, and after that everything you post to your site will automatically appear at Datafall also. It’s kind of like Slashdot, except that instead of being a group blog for CmdrTaco and Zonk and the three other people who can post at Slashdot, it’s a group blog for the entire internet.

Or, if you don’t have a blog or know what I’m talking about: Datafall is an open site that (hopefully) collects the best bits of other sites, and puts them in one place for your reading pleasure.

Why I did this, and why you might care

Lately a lot of the good content and discussion on the internet has been posted in what are called “blogs”. This is a word that is supposed to be short for “weblogs” but basically just means a site where people frequently post things they wrote.

A problem with blogs, at least in my opinion, is that they aren’t very good at forming communities. Almost all blogs have comment sections, so there’s usually a little community there; but these communities usually aren’t very large, and they can sometimes be very insular. Also, most blogs have links to blogs they like and those blogs usually link back, so you sometimes get little rings of blogs that all tie together; but these usually aren’t communities so much as they are cliques. Sometimes you see “group blogs” where a couple different blogs band together, like the excellent Panda’s Thumb; but this is not common, and the tools for setting this sort of thing up don’t seem to be very good.

To me, a good internet community should be something where a whole bunch of people come together to some kind of common ground that no single person exactly controls, the way most web forums work and by-invite blog cliques don’t. When communities are open like this, you get a much wider and more interesting range of opinions, and people are encouraged to respond to things they don’t agree with instead of just shutting them out. Another nice thing about big “common ground” sort of sites is that finding the good stuff is easier– content comes to you, instead of you having to go to it. Good blogs, in my opinion, are after all kind of hard to find. On the other hand, look at something like Slashdot– it’s not very good, but it’s consistent, and that makes it easy. The links on Slashdot are usually just whatever the rest of the blogosphere was talking about three days ago, so you could get the same links by just reading a bunch of different blogs– but the links do get to Slashdot eventually, and personally, I’d just rather read something like Slashdot because it’s easier.

The problem is, though, that while some kind of big centralized site like a webforum may be what I’d prefer as a reader, the people who are actually writing good, interesting stuff prefer to do it in their blogs rather than something like a web forum. And this makes sense. Who wants to pour their heart and soul into writing something really good if it’s just going to get a Score:5 stamp in a slashdot story and then disappear forever? If you save your best writing for a blog, not only do you get more attention, it’s safer– the blogger has control over their own site, so they never have to worry about somebody else screwing it all up for them. (I’ve seen at least two collaborative writing sites fall apart, partly because the people running it couldn’t consistently keep the hardware up and running.) It’s easy enough to get people to collaborate and submit good stuff when your site is nothing but links, like the front pages of Slashdot or Fark or Digg are. But what if you want actual writing– things like news analysis, or political commentary, or interesting stories? Well, that’s what blogs are for.

I wish there was some way that you could blend the best advantages of blogs with the best advantages of something like Slashdot or a big web forum.

So I decided to try to create one.

How this works

One of the common features all blogs share is what’s called an “RSS Feed”. RSS is a way of displaying posts on a website without displaying the website itself. A lot of people use these programs called “RSS Aggregators” to read blogs. RSS Aggregators (Firefox and Safari each have one built in) keep bookmarks of all your favorite sites, and when you open the aggregator it shows you all the new posts from all of your favorite sites, all mixed together in one place.

Datafall is kind of like an RSS aggregator that’s shared by the entire internet; anyone can add their site as a bookmark at Datafall by going here. (All you have to do is give Datafall a link to your site– Datafall figures out the rest from there.) Once a site is bookmarked on Datafall, Datafall will automatically notice when the site updates, and add an excerpt of the new post, with a “Read more” link that leads to the full post on the blog where it was posted.

There are a few different ways to find posts on Datafall. Every post on Datafall has a post “type” (is it news, an op-ed, a diary?) and a post “topic” (is it about politics, computers, culture..?). By default, everything on Datafall gets posted in “Diaries” (which is basically the “anything goes” section) and doesn’t have a topic. You can move one of your posts to a different type or topic by clicking the “Edit or Moderate” link that appears under every post.

Aside from this, there is also a “Front Page” section, which is supposed to be the best of the best from all story types and topics. Like on Kuro5hin or Digg, the users vote on which stories are the best ones and worthy of going to the front page (again, by clicking the “Edit or Moderate” link under the post).

Regardless of type or topic, you can always see the newest posts on Datafall by looking at the sidebar on the right side of every page.

The hope is that Datafall will eventually work like a big collaborative RSS filter, with a bunch of feeds coming in and the very best stuff coming out on the front page, with the will of the users deciding what goes where. (Of course, since there are no users yet, all it takes to get something to the front page right now is for a single person to click on the “nominate” button.)

In principle, there are several sites that work kind of like Datafall already– sites like Feedster or Blogsearch.google.com, which take in many RSS feeds and help you find things within them. However, these are not communities. They are search engines. They do not bring different blogs together any more than Google brings different forums together, and they’re all pull, no push– you can’t get anything out of Feedster unless you already know what you’re looking for.

Datafall can be different.

Site principles

Datafall is far from finished (the section after this one describes some of the things that need to be done), and along with the work that isn’t done yet, there are also going to be a number of decisions that need to be made about how the site should work and how the community should look. As [if] the site gains momentum, these are the principles I am going to try to shape everything that happens around:

1. The site should be interesting and readable. All other goals must kneel before this one. If looking at the Datafall front page doesn’t immediately produce something worthwhile to read, then what’s the point?
2. The site should be controlled by the users. Group moderation should be used everywhere. Datafall isn’t “my” site. If I just wanted to run a blog, I’d just do that. Actually, I’m doing it already, now. Datafall, on the other hand, should be a site that exists for, and is controlled by, the people who post there. I am only one of those. Whenever it is possible for a decision about the site– about what kinds of features get implemented, about what does and doesn’t get moderated well, about how (if at all) the site is policed– to be in some way deferred to the userbase at large, it should be.
3. Filter, don’t exclude. Of course, there’s a big problem with the above idea: not all of the users are going to agree on everything. Different users might have different ideas about what is good content, or a good feature. Whenever possible, the users on the losing side of the decisionmaking process should be given some way to split away and continue on as they like. The entire point of Datafall is about bridging gaps and bringing different sites together, but it’s important to realize that this isn’t always possible, and you need to have a plan for what to do when it isn’t. If it’s decided that content doesn’t belong on Datafall (short of it actually being spam), it should be hidden, not deleted. If it reaches the point where a subset of the users wind up with a vision of what Datafall should be which is entirely opposed to that of the rest of the userbase, and it turns out there really is no way to reconcile this, the minority should be given some way to split off and carry on without the rest of us (see “groups” and “open source” below).There are two reasons for this. First off, collaborative processes can succumb to groupthink. Whether content is good or bad doesn’t have much to do with whether it is popular or unpopular– but democratic processes, like voting on which stories are the best, are better at picking out what is the popular thing to say than what is the right thing to say. This means eventually content gets excluded which does not deserve to be. The best way to avoid this is to try not to exclude content, at least not all the way. Second off, and more importantly, excluding people never works. Sad as it is to say, every site winds up accumulating people who really shouldn’t be there; but ironically enough, invariably the ones who most deserve to be thrown off the boat turn out to be the ones who are best at keeping themselves from being thrown off the boat. In a best case scenario this “certain kind of person” does this by manipulating the emotions of the people responsible for policing the site, in a worst case scenario by cheating and evading bans. The best way to deal with this, I think, is to just go ahead and give these people their soapbox, and then give everyone else the tools to avoid having to listen to it.
4. Never stop experimenting. The Internet never stops changing; you can’t survive on the internet unless you do the same. I have seen (and used) enough small sites that failed miserably to know this. Datafall should always be a work in progress, and the site should always be incorporating new ideas, even if they’re bad ones. If they turn out to be bad ideas we can just take them out again.
5. AJAX. This is a technical issue, but it’s an important one. AJAX is this new fancypants internet technology that lets webpages update without reloading. Like most things on the internet, AJAX has the potential to allow a lot of cool and interesting things, and also the potential to allow a lot of abuse. AJAX is used on Datafall in the following ways:
   • AJAX should always be used for controls. Everything on the site like reporting a bad post, or voting on a good one, is controlled by AJAX. You should never have to suffer a pageload just to change the state of something, and so far, on Datafall, you don’t– the only forms that trigger pageloads are when you’re logging in or signing up for an account, and I may even be able to remove even those eventually.
   • AJAX should never be used to navigate. That’s what pageloads are for. The “back” button is sacred and it should always do exactly what you expect.
   • The site should always work exactly the same with Javascript turned off as it does with Javascript turned on.

Future plans

Things about Datafall that should change in the near term:

1. Voting is not as robust as it should be. Right now, anyone can change any article to any section, and anyone can nominate something to the front page. I have features in place that would do this better, but they are not turned on– again, because there aren’t any users on the site yet, so right now they’d just make things needlessly complicated. Eventually the site will have something like “I liked this / I didn’t like this” counters on every story. If a lot of people like a story, it will get shown on the front page. If a lot of people dislike a story, it will get cast back down into the diary section.
2. Hilariously, although the entire site is made up of RSS feeds, Datafall itself doesn’t offer an RSS feed yet.
   
3. This is an important one– pinging. Blog engines offer ways to automatically notify sites like Feedster or Datafall when they have updated. I don’t actually even know how this works exactly. I need to find out. Right now Datafall doesn’t immediately know that one of its bookmarked sites has updated– it just checks for changes periodically. This is bad.
4. More story types— we need a “Links” section eventually, and I’m considering a “podcasts” section.
5. Deletion. Right now, if you make a post on Datafall, you can’t remove it. Nobody can delete posts but me. This is probably bad and stuff.

Things about Datafall that should change in the long term:

1. Groups. Right now, the only way to sort things on Datafall are the type and topic sections linked at the top of every page. There should be ways for users to create new types, new topics, or entire other ways of categorizing things. In principle, this should work like “Groups” on LiveJournal– LiveJournal lets you make specialized group blogs that act kind of like message boards, and that you post to as if you were making a post in a LiveJournal. Of course, you can only post to a LiveJournal group by making a post specifically to it on LiveJournal.com. Datafall groups, of course, should be able to take in posts from anywhere. Eventually this can hopefully even work such that it’s possible for users to create their own totally autonomous subsites with Datafall, with their own moderation rules and everything.
2. Ripping off Feedster and Digg. Right now, posts only enter Datafall if the person who owns the RSS feed wills it. It doesn’t have to work this way. If Datafall ever gets ridiculously large, we could add a separate “best of the internet” section that works kind of like Fark. The outputs would be voted on the same way that any other Datafall post is, but the inputs would be the entire blogosphere instead of just Datafalls’ diaries– for example, maybe Datafall users could nominate articles they liked but didn’t write. Now, given, I really don’t think this is a good idea. It doesn’t fit with any of the site’s goals, and it also introduces various difficulties (both legal and technical). However, it’s something worth considering.
3. Comment and account tracking. This, on the other hand, is something I really do want to try: Datafall bridges the gaps between sites by putting articles in a central place. However, comments on different Datafall blogs may as well be in different universes. I am curious what can be done about this. Think back to Slashdot: If you post in six Slashdot threads in one day, you can come back to Slashdot later, go to your user page, and have nice convenient links to all your posts, along with how many replies each one got. If you post in six different threads in the Blogosphere in one day, on the other hand, the only way to see what happened to them later to is to go back and track down your posts in each of those six threads. There must be a better way to do this.Right now, a Datafall account isn’t really used for anything except creating feeds. It would be interesting to try to make it so that the posts you make in the comments section of a blog that uses Datafall are automatically recognized as being part of your Datafall account. (Right now there are a couple of “shared account” services which let you access many blogs with a single signin. But as far as I know, none of them are very open or, for that matter, open source.) In addition to, or maybe instead of, this, Datafall could track comments made on Datafall blogs (some, but not all, blog engines offer RSS syndication for comments) and provide a “comments I have made on any Datafall blog” page. I think this entire concept would be something extremely useful, maybe something even more useful than the part of Datafall I’ve implemented so far. However, it would not be trivial. Each blog would have to individually support the comments features; not only is there the problem that not everyone would want to participate in this, but also there is the problem that (by the very nature of Datafall) every blog linked from Datafall is running different software. But, of course, this leads me to:
4. Blog plugins. Blog engines like WordPress or Movable type all support plugins. I would like to look into making plugins for these blog engines that makes posting on Datafall easier. A simple version of this plugin might do nothing more than add “type” and “topic” menus whenever you post a story, so you don’t have to go through the silly step of, every time you making a post, fishing it off Datafall and rescuing it from the Diary section. I don’t think this would be very hard (though, on the other hand, I don’t think I really want to do this unless people are actually interested).
5. Open source. One last thing: I want to release the code that runs Datafall as a Ruby On Rails plugin. I have not actually figured out how to do this yet. Once I have this worked out however I intend to release Datafall’s software under the GNU LGPL.

That’s about it. I hope you find Datafall useful or at least interesting. If you have any thoughts on this experiment, please leave them as a comment below.

I hate blogs.

I really do. I’ve been a very, very long-time follower of internet discussions groups of various kinds, and I’ve watched over the last three or so years as any kind of site even remotely related to technology or politics has gradually had the life sucked out of it as, one by one, every user of any worth has been gradually assimilated by the Blogosphere. And I, not wanting to follow, have gradually found myself with nowhere left to go. The discussion sites of yore, once vibrant communities, have become alternatively ghost towns or wastelands, with no one left but trolls and people who spend all their time talking about their blogs. Only Slashdot still lives, and Slashdot… kind of sucks.

It’s still not clear to me exactly what “Blogs” are supposed to be, or the “Blogosphere” for that matter. “Blogs”, despite sounding like they ought to be onomatopoeia for some kind of disgusting biological function, seem to just be websites. It’s not like there weren’t websites with frequently updated content, or people who put journals or diaries or news commentary or ideas on the internet, until four or five years ago when blogs appeared. Suck.com was perfectly mimicking the structure and nature of a blog as far back as 1995, and survived just long enough to comment on the word “Blog” as it first appeared. Supposedly, blogs are different. The two most common explanations of what makes blogs different are the appearance of “blog software”, and the idea that blogs “democratize” the internet by erasing the line between content consumers and providers. But there really isn’t anything to the “blog software”– Blog software is so incredibly easy to write I actually once wrote a blog engine by accident, and almost anything that in the 90s would have been called a “news script” qualifies as a blog engine by today’s standards. (When they’re used right, blog packages do offer a lot of innovative and frankly neat features that the high-end hardcore set gets a lot of use out of, like RSS, but not everyone uses those or is even clearly aware they exist.) The “democratizing content” thing seems a bit odd as well, since that’s the exact same sales pitch that was originally used to describe the internet itself when the media first started to notice it.

So maybe the idea is that the internet promised a global two-way street of content (where everyone participated and everyone was equal) from the very beginning, but it wasn’t really until the Blog Revolutionâ„¢ that this potential was actually fulfilled. Before blogs came along, let’s say, that promise that anyone could be a content provider was largely an illusion; this magical place where you could beam your thoughts to the entire world existed, but it was reserved for techheads, walled off by a thin but hard-to-scale barrier made of HTML and FTP sites. You had to master all these difficult acronyms and know the Secret SGML Handshake to get into the Cool Kids Club of people that were allowed to run websites in the Old New Media. But now we have “blogs”, and that’s all changed. Blog software removed that techhead barrier to entry, laying down a royal road to internet presence that anyone can master, and now that original promise that Everyone Can Have Their Own Website is fulfilled. We’re going to start the internet over, Web 2.0, and this time we’re going to get it right.

I don’t really buy that either. Probably it’s true that getting and maintaining one of those very first free websites on Geocities was reasonably beyond the technical abilities of most people. But for a rather long time there were sites where you could set up a personal page without understanding anything at all about what you were doing– mostly using free software similar or even identical to the “blog” sites of today. And the barriers to content creation weren’t a problem of the internet in general, just the web– many of the things people use blogs for today would have been provided, at one time, by mailing lists or USENET groups, and those are as easy to use as opening Outlook Express. Easy to use alternatives that effectively provided the functionality of today’s blog software existed all the way back to the 80s, and toward the end of the 90s the Geocities style sites even started to adopt simple, web-based personal website creation scripts that the proverbial grandmother could use; but people didn’t use them, or didn’t see the point, or weren’t interested, until they got to start calling them “blogs”. Even the web diary concept that one would expect is the central Blog innovation isn’t a particularly novel one. Livejournal was around for a pretty long time before there was ever a “blogosphere” to label it as part of. Until the “blogosphere”, Livejournal was just another internet community, just like the ones that had existed before that on web forums, and IRC, and USENET– and it wasn’t really all that different from any one of a dozen sites that people were using to communicate diary-like material at that point.

So how is “blog software” something new, exactly, if it isn’t the technical functionality, or the ease of use, or the format, or the communities? Because something happened there; there’s something about the difference between blog software and all the software before that that created a social movement where before there had just been an internet. All these people are running around now insisting blogs are the next big thing. Something apparently happened that inspired change in the transition between Fark and Movable Type; what?

As I see it, the one thing that actually makes blogs different is that it lets people do the same things they were doing with the Internet all along, except now they get to pretend it is important. Before, there were both important and unimportant things on the internet; but looking important was something nontrivial, a luxury reserved for people with budgets and people who know Perl. You could easily get stuff on the internet where people could see it, but since the HTML barrier meant you had to find someone else to administrate the site, you were always in effect working for someone else– you could write the most important, intelligent thing in the world, but it would never be your post, it would always be your post on Livejournal, and anyone could see that just from looking at the top of the page. Who wants to spend all that time writing something when you don’t get the credit, or the link from the front page, and somebody else’s big tacky logo is marring your work? It just doesn’t feel as special somehow.

But now, everyone from some guy with a cat to Chrysler can project the same illusion of respectability. The new blog software packages project the polish and flourish of something like Slash or Metadot or a big professionally-made news site, without the techhead difficulty and configure-these-Perl-files learning curve; and once you’re done, those big blog edifices look totally professional even if the URL is just a subdomain somewhere. That feels good. You’ve made it, you’re your own Commander Taco, your postings are beholden to nobody else’s “ban” button. You may not have any more readers in this state than if you’d just posted your thoughts periodically on some big web forum, of course. But by golly, you can look at your front page with the fancy list of prior postings, and the default WordPress theme, and the trackback where some other guy nobody’s heard of either says “so I saw the American Kumquat talking today about…” just like you were the New York Times or something, and dammit, that’s your website.

The “blogosphere” is something a bit less superficial and maybe more interesting, and it’s what happens when you take the army of identical I Am Unique And Independent websites these software packages necessarily create, and turn them on all at the same time. The name “blogosphere” seems to imply some kind of giant floating hive mind from a bad science fiction movie, and at first glance, that really is what it seems to be. Despite all the effort that’s been put into making clear site divisions between all the different bloggers, once they start writing all the blog sites kind of start to blur together; everybody links each other, everybody responds to each other, everybody builds on what everybody else has already written. When this works, this kind of gradually forms into a large cross-site exercise in collaborative journalism; where one writer would not be able to make something really stunning and well-researched by themselves (or at least would have to put a lot of work into it), when a whole bunch of writers try to attack a problem at the same time, each one contributing one little tiny piece, the final result becomes, through the magic of interferometry, something really quite impressive. The spirit of a funny cross between a capitalist fight for eyeballs and a big friendly circle jerk that the blogosphere encourages makes this possible, by ensuring that the individual contributors to the mass simultaneously compete and collaborate.

That’s when it works, mind you.

It doesn’t really wind up working all that often.

Before the blogs really arose, the tendency in internet communities was toward forming ever bigger and more monolithic single sites. To an extent you could gauge these sites by how much different *stuff* was going on within its walls, how many of its cliques had developed into full-blown subcultures. (We still get sites like this, a la Facebook and Xanga, appearing all the fricking time. But they all either are or claim to be “blog” sites, so for purposes of this particular discussion they don’t count.) These sites had intricate and multifaceted communities, and could readily create fads or community responses which spilled over into other sites or the “real world”. But they were all always focused inward. Information came in from the outside, but the focus was never quite on the information itself, the focus was always on the community’s response to the information. That was what you came there for. You could read just the start of every post to see if somebody dug up an interesting link, but you probably didn’t. There are probably more people on Slashdot who read the comments but not the article than the other way around.

Blogs, though, at least individually, are all always outward-focused. You come to a blog to see what the blogger said, but it’s almost invariably what the blogger has to say about something they saw somewhere else, what the blogger has to say about this link, what the blogger has to say about what this other blogger said, in a best-case scenario what the blogger saw this morning on the subway. This tendency means that proper communities almost never form on single blog sites; they usually form across several sites, with several blogs clumping together to form a little clique where everybody knows everyone else. Unlike the big cathedral monasteries that the old web communities formed, the blogosphere is like a million tiny islands, each linked to the two or three islands near it by little bridges. Big walled blog cities like DailyKos nonwithstanding, everything is open, everyone knows their neighbors the next island or two away, and if you just keep following those bridges you can hop from blog to blog until you’ve crossed the entire internet from one side to the other.

In theory, what the blogosphere ought to do is basically merge the entire internet into one gigantic community, one big site with a thousand faces. We don’t need the cathedrals anymore to provide a common ground for a community; the internet itself is the common ground. We can build communities just as strong, and just as complex, by just stitching all these islands together and realizing that even though they may all be on different hosting providers, in the aggregate they’re all the same thing.

That’s, again, when it works.

This is where, I think, things start to go wrong:

Let’s look at the cathedral model of internet communities again. The big Slashdot style sites. What advantages does this model offer? Well, none, really, except better organization and ease of finding things– and that can be supplied even in the island model by the use of things like Google or Feedster. The cathedral model does have some disadvantages, though. These turn out to be a bit more important.

The most important disadvantage of the cathedral model is this: Everyone inside the cathedral is basically locked inside the cathedral with every other user, which means if there’s another user on the site you dislike, you actually have to learn to get along with them. Blogs don’t have this disadvantage, and this turns out to be the chief reason why blogs don’t work. Nobody is forcing you to spend time near anyone you don’t like. If you don’t like something about a user somewhere, you can just go somewhere else. There are many islands in the sea, and you don’t have to hang out at any one of them if you don’t get along with the people there, or you don’t get along as well as you used to.

At first glance this is liberating. Every little clump of blogs, every mini-community can find its optimal userbase. Internet rivalries are something you enter into cheerfully and voluntarily, rather than being an unavoidable fact of life, and if you get tired of the rivalry you can just start banning people until it’s over. Even better, having to uproot and rearrange your internet allegiances on a whim is not a huge production in a world full of blogs, the way it is in a world full of webforums or even usenet groups. Because communities are encoded in the connections and relationships between different sites, instead of being encoded in the user database of one big site, these communities are much easier to rearrange. Switching from one web forum community to another involves signing up for accounts, setting up new profiles, starting over with a postcount of 0 and no karma. Switching from one blog community to another involves just changing what the links are on your “blogs i read!!” sidebar. Since your own blog retains everything, even your post history comes with you when you move.

After awhile this becomes addictive. The liberating realization of “hey, I don’t have to put up with these assholes, I can just go over here for awhile!” gradually melts into a realization that beyond just what kinds of communities you invest your personal attention and posts into, you don’t actually have to ever read anything at all that involves people you don’t like. Or that makes you in some way uncomfortable. Or that you don’t agree with.

And this is, though probably not the only reason, at least part of why blog communities are so cliquey as to make every single other thing that has ever happened on the internet pale by comparison. Cliquey, in fact, to the point that it effects the very content of the sites themselves.

Spend some time reading blogs and you’ll quickly realize that what conceptually seems to be a big, amorphous, continuous expanse of blogosphere is actually highly and rigidly structured. Communities span across sites, and sites bleed into one another, but the communities are highly insular, and the sites tend to only bleed into sites that the author likes. And the authors of blogs, oddly enough, tend to only like blogs that are exactly like theirs. The sense that you could just follow links from one blog to another until you’ve crossed the entire internet turns out to largely be an illusion. If you actually try this– start off at some random politics blog and start randomly clicking links on the omnipresent “blogs i like” sidebars, and just see where they take you– you’re more likely to find yourself trapped in a circular pool of sites that all link to one another, but pretty much no one else. That pool might be in some cases quite large, hitting tens or maybe even hundreds of sites. But a large fishbowl is still a fishbowl. And blog communities have generally developed a remarkable ability to form cliques that, despite there not being anything holding them together but social forces, are entirely hermetically sealed (everyone in the circle has links to lots of blogs but they’re all blogs inside the circle, everyone in the circle is always referring to things happening on other blogs but they’re all blogs inside the circle, and everything outside the circle is totally ignored except to the extent it offers something to rebut) in a way that utterly breaks the small-world effect that governs real-world social networks. This does things to the people who write and read these blogs. This is where the blog model starts to fall apart.

That part of the blogosphere which aspires to be journalism or at least op-ed is in many ways a giant experiment in how people can come to focus on the people they know until they start to mistake them for the entire world. When everything you read comes from the same pool, and everyone who reads what you write comes from the same pool, and you do this in an environment which (thanks to the illusion of being in this big open interconnected ocean) provides the illusion of incorporating the entire spectrum of discourse on the Internet, you gradually start to feel that the circle of people you talk to are the only people that matter– or even the only people that exist. The thought “well, it must be a good idea; everyone who reads my blog liked it” is just way too easy to fall into, and the more reassuring and comforting your blog circle becomes, the easier it becomes to just assume the rest of the world would just kind of feels the same way. A blogger can be bounded in a nutshell and crown himself king of infinite space, so long as he has yes-men.

This kind of intellectual tribalism leads to a habit of feeling that anyone who feels the same way you do must automatically be right– a habit that makes bloggers startlingly easy to manipulate. Case in point: Andrew Sullivan. Andrew Sullivan is “the gay conservative blogger”. These four words are all that most people bother to explore of Andrew Sullivan’s identity. Sullivan is one of those few bloggers whose sites have gotten so huge and widely-read that he’s pretty much a brand name; Andrew Sullivan means “the gay conservative blogger” the way Coke means “soda”. Sullivan’s writings and opinions are sometimes excellently-composed and insightful, and sometimes dishonest and stupid; for the moment, though, let’s not worry about what his writings actually say, and concentrate on the site itself. The site is one of the most heavily-trafficked and widely-linked sites in the entire blogosphere. It frequently comes up in arguments when people start to talk about the democratizing power of blogs. Sullivan has been showing up a lot in major media sources as a kind of spokesperson for the blog movement; the September 2004 issue of Time printed a blogging mini-manifesto by him, in which he is identified as “a member of the blogging class” and tries to write some kind of “creed” for bloggers who blog. Here’s the thing about this:

There is this idea that there is an “Old Media” and a “New Media”, and “Old Media” is an outmoded and repressive institution in which a small crowd of elites hold a monopoly on expression, and “new media” (now embodied by blogs) is a new and democratic movement in which voice is extended to all who wish to speak, demolishing the power of Old Media. If we are to accept this idea, Andrew Sullivan is as old media as one can get. Sullivan was the editor of The New Republic for six years in the early 90s; later on, right up until around the time his blog started taking off, he was a regular contributor to The New Yorker and The New York Times. The latter two are impressive on any resume, but the New Republic is probably a bigger deal. The New Republic is one of a small number of “policy rags” who not all that many people have heard of, and who not all that many people read (its circulation is about equal to the number of hits that, as of early 2005, Sullivan’s current blog would get in a single day), but whose influence is enormously disproportionate to its readership. It is kind of like how Brian Eno once claimed that the Velvet Underground only sold a couple of thousand records, but every single person who bought one started a band; the policy rags only distribute in the thousands, but almost everyone who holds actual power reads them. Not to say they’re all reading The New Republic in specific of course; but the people who actually run this country are all reading The New Republic or something like it, and anything which can get into a magazine like that has a direct private channel right into the ear of the most powerful people on earth, with the potential to make an impact that ten thousand letters written to congressmen or news corporations (and thrown away by interns) never could. In the runup to the Iraq War, when The New Republic held the distinction of being the voice of the neoconservative movement (whatever that means), The New Republic’s foreign policy writings very briefly were so in lockstep with that of the Presidential inner circle that you could practically predict what the administration was going to do next by reading The New Republic that week.

So here we have Andrew Sullivan, who for six years in the 90s ran one of these elite-of-the-elite shadow-media mouthpieces; who has connections in major media institutions most writers could only dream of; who if, if we are to accept this idea that that there’s a small cadre of entrenched old-media opinion monopolists who are the only ones in America who are given access to the national dialogue, Andrew Sullivan is right down in the core of that privileged group to the point that at one stage around 2000 he was actually publicly claiming he was being censored because the New York Times was no longer offering to print his columns. This is the person who is being selected or allowed to speak for the movement by which blogs are supposedly democratizing media as a whole, one of the chief evangelists who is bringing the gospel of the changing power of blogs to the masses. Does this not strike you as a bit odd? Andrew Sullivan is supposedly one of the blog movement’s great success stories. But from what I can see, it looks like all that’s happened in the case of Mr. Sullivan is that a media powerbroker took a break from his main career of doing op-ed writing for gigantic news publications to go slumming for a few years with this “blog” thing, and now a short few years later he’s back to writing in Time, doing essentially the same things he was doing before, except now he gets to slap the “blogger” badge on his byline (though, coincidentally, the blog itself is as I write this on an apparently temporary sabbatical). This wasn’t an elevation; it was a rebranding.

Which makes this, of course, an excellent and impressive success story for Mr. Sullivan, who is clearly doing quite well at his job as a media commentator. It is not a success story for the blog movement. The blog movement here is simply being used; it was a tool by which an established writer kept himself relevant. Perhaps this says something about what a useful tool blogs can be in the hands of someone looking for success in the fields of media or PR. It doesn’t say anything at all about the utility of blogs as an agent for change either inside or outside the media. The bloggers aren’t bothered by this; indeed, they don’t even seem to have noticed. What matters to the bloggers about Andrew Sullivan is (a) he’s a blogger (b) he’s successful and most importantly, (c) he agrees with the other bloggers. After all, if you agree that bloggers judge content based on whether it reinforces the things they already agree with, well, the different cliques of bloggers may disagree with one another about everything under the sun, but the one thing they all agree on is that blogs are important. Bloggers may reject some subset of Sullivan’s politics, but they can be counted on to rally behind the most consistent message Sullivan can be heard pushing these days, which is that blogs are changing everything and revolutionizing the media and empowering the previously voiceless. Bloggers like this message, and cannot be bothered to concern themselves with petty details such as the context of this message, or whether it makes any sense whatsoever for Sullivan (himself a living disproof of the message) to be the one promoting it. Context is something that exists outside the blogosphere. In the mindset of blogging (where the writing is always focused on things happening elsewhere in the world, but the world is always equal to your clique), the wider media and social context is something bloggers triumph over; it isn’t something bloggers exist within. The denizens of the blogosphere, apparently, don’t know or need to know what’s happening outside.

As it happens, some rather big things are happening outside of the blogosphere.

Something in the American spirit has become terribly sick. The entire mechanics of American media and society is going through a massive weird period that looks suspiciously like some kind of transition. What this is a transition to, or what from, is anyone’s guess. However, what is clear is that right now, the American national dialogue does not value honesty. American media, at this point, doesn’t so much report as it does repeat; various people have for various reasons felt for some time that the news media isn’t very good at seeking or exposing the truth, but in the last few years it’s entirely gotten to the point that the news media doesn’t even try to seek or expose the truth, doesn’t even act as if it feels that truth is something news media can or should concern itself with.

After all, truth is hard. You have to do a whole lot of work to get to it, and sometimes doing all that work to get the truth makes enemies of various kinds. Even worse, truth is risky. If you get truth even a little bit wrong, you get in big trouble and people yell at you and stuff. The news, these days, would rather concern itself with opinions. Opinions can’t be wrong. Opinions are easier, and safer, and cheaper. All you have to do to report an opinion is find someone with an opinion, and then tell people what it was. This makes all sorts of things easier. For example, you don’t have to worry about tricky things like journalistic integrity; instead, all you have to do is be balanced, which means that anytime you report an opinion, you have to also report an opposing opinion. Except it really isn’t even that hard; all you really have to do is report two opinions, and just assume that these two opinions you picked, whatever they were, are both opposing and represent the entire spectrum of thought on the issue. Except it isn’t even really that hard; it appears a lot of the time that you don’t really need to present the opposing viewpoint, so long as the opposing viewpoint is “liberal”. (Those opinions, like maybe if someone has some kind of loony communist opinion like “maybe a supreme court candidate isn’t really ‘mainstream’ if his appointment would make abortion potentially illegal where it was previously legal”, can be saved for the Letters To The Editor page or something.)

So news outlets have a pretty simple job now. Simple as the world has become for the news outlets, of course, this kind of makes things difficult for some of the rest of us. Some of us think that there are these things called “basic facts” (like “the second law of thermodynamics is not meant to be applied to an open system such as the planet Earth” or “on October 7, 2002 the President gave a speech which said in part that military action against Iraq was needed to prevent a nuclear strike on the United States of America”), and these things can be demonstrated to be either true or false and then used to make decisions. Now we find out that these things are, in fact, no longer facts at all, but just “opinions”, and moreover find out that the news media is obligated to give them equal credence as the alternative opinions (such as “the second law of thermodynamics means that life could not have evolved on Earth” or “the President never claimed Iraq had nuclear weapons”) without trying to examine or judge which alternative might be correct. This makes it kind of hard to form or argue opinions that one can consider informed, since facts apparently don’t exist anymore to be informed about.

Worse, now that facts no longer exist, there is no longer such a thing as honesty– since, after all, there’s no longer such a thing as truth to be honest about. This is not just a silly joke about wordplay. America in general seems to have lost the capacity to even talk about or understand what honesty means. For example, let’s say the President of the United States of America gets up on television and tells the world he has evidence Iraq has weapons of mass destruction. Now let’s say a few years pass and it turns out that not only did Iraq not have weapons of mass destruction, but the evidence the President said he had before was so flimsy, so full of obvious holes, and so highly contradicted by other evidence the U.S. had at the time, that it can’t really be said to have ever existed in the first place. A truly amazing number of people will look at a situation like this and conclude that the President wasn’t lying about the evidence he said he had, for the reason that the president didn’t know there were problems with the evidence he said he had. The President could have, of course, found out about these problems if he had just looked at the evidence, or asked someone else to check the evidence for problems, or just asked the simple question “now, is there any evidence which might say that Saddam Hussein doesn’t have weapons of mass destruction after all”? But he didn’t. And it was a wise choice, it turns out; so long as the President chooses not to ever concern himself with what the truth might be, he’s protected from any allegations of lying about anything, ever. He can repeat this trick any number of times: becoming willfully ignorant about the exact consequences of his decision that America doesn’t need to follow the Geneva convention sometimes, and then saying Americans are not committing torture; becoming willfully ignorant about exactly what his staff did after he gave them permission to leak classified information, and then saying his administration didn’t leak the name of a CIA operative that, it turns out later, they did. In all cases, most of a nation will claim that even though he said things that were totally untrue, if the President didn’t believe he was lying, he wasn’t lying– and the amazing thing is, they really mean it. These people aren’t just doing this to defend the President. They are doing it because they seriously feel that this is what honesty means. And what choice do they have? Most of these people are generally uninterested or unenthusiastic about knowing what the truth about things is themselves; how could they blame the President for something they themselves are doing?

This situation does not look to be changing anytime soon. Everyone who is in a position to convince America it really needs to start caring about truth is too busy taking advantage of the current situation for their own ends. America’s honesty problems do not exist in a vacuum; there are a lot of groups that are, literally or figuratively, profiting from it, and this means they have a vested interest in sustaining the problem. If you are a group which holds power, it’s a lot easier to wield that power if nobody really cares whether you tell the truth about what you’re doing. And so most groups which hold power are doing exactly that. Most of the important things in America right now are controlled by organizations which value loyalty over ethics, such as most corporations, religions, labor unions and political parties. And organizations or movements which value ethics over loyalty, such as one would normally expect the media to be, are not arising to take their place.

Which brings us to blogs. Blogs have the opportunity to change all of this, blogs are in a position to make truth and journalism matter again (as opposed to just opinions and media). So far this is not happening, and it does not look like it is actually going to. Blogs are more useful for comfortingly reinforcing opinions you already have than they are for gathering the basis for an informed opinion, and bloggers themselves are almost universally disinterested in truth (unless it’s a truth that’s uncomfortable for the other guy; those truths they care about deeply). America has a problem that it no longer cares about honesty, and the tribalism that the blog movement makes so easy means that blogs are not part of the solution; blogs are not even part of the problem. Blogs are just another symptom of the problem.

We (and by “we” I mean everybody within earshot of the blog movement) are constantly being told of late that blogs are a watershed change in how media works, that blogs will bring accountability to the press, that even if most people aren’t getting their news from blogs, blogs force the media to stay honest by applying pressure for the media to address issues it normally ignores and own to its mistakes. The idea is that normally the media would just breeze past uncomfortable issues or errors, but the collaborative fact-checking layer the blogosphere represents will refuse to drop these things until the media, due to the public attention the blogs bring to these things, is forced to revisit them. There is a problem here. This is not change. This is, in fact, exactly how right-wing talk radio worked in the 90s, to the comma. What the blogosphere is doing is not a new concept at all; it’s replicating something that’s 10 years old, except now it’s on the internet instead of the airwaves, and we now have a rabid left-wing talk radio blogosphere to match the rabid right-wing talk radio blogosphere that already existed. Instead of just Rush Limbaugh, we now have rushlimbaugh.com and DailyKos. This is not progress.

The problem is that both in the talk radio of yore and the blogosphere of today, the things that the blogosphere forces the media to address are in no way obligated to be facts. It helps, of course, but the entire theory is that an idea or allegation gets picked up by the media due to blog pressure because it’s being widely repeated; it’s no particularly easier to get a true idea widely repeated than a false one. If the blogosphere were a system that valued facts, or truth, or honesty, or one where things that are true are more likely to gain wide currency than things that aren’t, then blog pressure would indeed have a positive effect on the media outside. But blogs are themselves governed by the same principle that rules the normal media: repeat, don’t report. If you pay attention, you’ll find that the blogosphere exists in a fascinating kind of hierarchy where the people who write the little blogs read the big blogs in the morning, and simply take the opinion they decide they’re going to have that day from whichever big blog suits them most. If you very pay careful attention to the internet some days, you can actually track the spread of a single bad idea (say, a catchphrase, or an argument, or a… “meme”) as it trickles down through this hierarchy, showing up on the big megablogs at the beginning of the day, moving on to the moderate-sized blogs as the morning progresses, thriving on all of the small flyspeck blogs run by the people who read the moderate blogs as afternoon comes, and from there disseminating to the external sources like internet discussion boards, where suddenly one finds that there are a bunch of people who believe the exact same idea that appeared on the megablogs at the beginning of the day. The flow of ideas here is not of course strictly one way; sometimes the idea the megablog latched onto came from a smaller blog, or started at a flyspeck blog and had been slowly for a little while climbing its way up the hierarchy, garnering supporters among blogs at different levels. But a surprising amount of the time the day’s talking point basically originated from one of the same sources that set the tone of coverage for the normal media that day– something like Fox News. This means that what could be a marketplace of ideas that collaboratively uncovers the truth of things turns into a big daily game to see which partisan blogger army can outshout the other that day. Blogs are supposed to correct and cancel out the assumptions and groupthink that have pervaded the general media; in practice bloggers only just barely manage to correct and cancel out the assumptions and groupthink that the rival bloggers generate.

The media has become addicted to dealing in opinions instead of news or facts, and the blogosphere in its current state is not going to do anything to help this situation– “opinions as news” is, after all, the core principle of the blogging movement, and so blogs are just as bad about this as any TV news station. The cliquey homogeneity one finds within any one of the various differing partisan wings of bloggers goes beyond a simple matter of everyone echoing one another; it’s a matter of these different wings of bloggers actually living in entirely different realities. The “opinions” that have replaced what used to be basic facts split one way or another along partisan lines (Did the President ever suggest Iraq had nuclear weapons, or not? Was Iraq collaborating with Al-Qaeda? Do “liberals” “hate” “America”?), and the disconnect on these facts become just another factor separating the different cliques from one another. The fluid nature of the blogosphere has turned out not to produce an open field of borderless bridges, but a series of xenophobic little island fiefdoms that clump together into fishbowls, and shut the rest of the world out. The freedom to find or form exactly the community that’s right for you turns out, when you actually start looking at blogs, to be an opportunity to choose which groupthink you want to conform to.

But blogs are all that are left. The idioms of communication on the internet flow in phases, and right now, if you are interested in discussion and news on the internet, blogs are the form that the current phase takes. Reading the above, you may have noticed, just a teensy bit, that I’m not terribly cheerful about this. Personally, all I ever really wanted out of the internet was to have a nice conversation with someone, and good conversations are surprisingly hard to find. I have no interest in either inserting myself into a circle where everyone agrees with me on everything, nor inserting myself into a hostile blog clique (ruled by “the other side”) where I become the local effigial punching bag; and while I know at some level that I must be overgeneralizing, there must be some corner of the blogosphere that isn’t like that that I just haven’t found yet, it seems like these two extremes are the main options in the blog world. But nor (whether I’m right about the way blogs are or not) do I really want to sit and watch as the internet mediums I’ve known in the past gradually fade away entiely into ghost towns, as everyone filters away to settle in the island paradises of the blogosphere. So, I’m giving up. I’ve got a blog now. It’s got its own little subdomain and a little default WordPress layout and everything.

Here I am.

What happens now?