Posts filed under 'Vision'
So, this blog will be going into deep-freeze for a while (and I’ll regrettably have to close all comments shortly, because I don’t want to deal with the spam; email if you want to chat about what you find here). While I launch Fictionsuit, Cornucopt is going to take some time off to have an identity crisis. See, back when I first thought of the name Cornucopt, I actually envisioned it as an easier-to-install Ecore. With some push-button setup options for common social-software patterns, and in PHP. That’s about it. Oddly, as I retrace the development of Wicker, now rechristened Cornucopt, it has gotten closer and closer to Ecore. Let me give you a hazy sketch of what I think Cornucopt might look like if I get around to it after Fictionsuit, here in the post-Ruby on Rails world, where there are monorails through wide, green public parks with nary a trace of litter or cramped tennis courts.
Like Ecore, we start on the barest of wiki foundations. You have a bunch of things called nodes, or pages, or whatever, and you can add text and link them easily. Drupal does something like this, only you start with a front page and stuff like threaded comments enabled, for a more slash-cloney shape. Cornucopt’s starting structure will likely be similar to Drupal’s, only with non-threaded comments (or perhaps differently threaded) and tags enabled by default.
The interesting stuff about Ecore (or Drupal) lies in what you do from there, and how you do it. Ecore provides not only a framework for building out a site’s content, but for doing your customization and programming within the Ecore environment itself. You can define a node with a nodetype of “nodetype,” and then just start writing code into a textarea that defines what nodes of your new nodetype will do, and how they will display. You’ve got certain objects you can read or manipulate - a user object, a session object, the page’s text, and suchlike - and you just code up what you want. In theory, I think you have access to all of Perl.
So, even before I was all making out with Ruby on Rails, I was starting to think that Corny’s scheme of pagetypes was kind of a half-measure. I mean, if you want to make something new, you’ve got to write a bunch of PHP into a file, set all your permissions and stuff in there, throw the file up into a poorly named and placed directory (I was gonna get around to fixing that, honest), and then start using it and hope it works. I guess the last step is more or less immutable. But having tried to build things in Ecore, I could see that there was a better way, even though Ecore was pretty much the better way gone wrong. I wanted more things built for me, and more things that could hook together interestingly, and more things that were open to the rest of the net. Ecore doesn’t have a closed culture, exactly… but it has the Perl culture in spades. “There’s more than one way to do it, so we can’t possibly presume to do anything for you. Here’s a textarea. Good luck!”
The real reasons I decided to abandon Ecore were not the above, though. They were my unfamiliarity with Perl and the difficulty (impossibility, actually) of installing Ecore on shared hosting. Ironically, Rails isn’t very friendly with most hosting companies either. But this choice reflects my new, corrected-for-reality notion of who’s going to use Cornucopt and how. I’ve decided to commit to the fact that someone who really wants to customize a wiki is going to write code of some kind. I think they’ll find Ruby code friendlier to write than Perl code. I think Corny’s built-in options will go a decent-enough way to fulfilling other, less extensive needs that I don’t feel as bad about requiring code for the serious kung fu. And I think that Rails is hot enough that Rails hosting will spread fast.
Plenty of design hurdles remain, however. Funny enough, security of executable code isn’t a big worry; Ruby has some sandboxing built in, so I’m not really concerned about restricting them to selected objects and methods. Figuring out what methods people will need, though, might get interesting. Also, these built-in things I’m waving my hands about - what will they be, precisely? How will they be architected, and how the hell do I think I’m gonna make them all snap-together easy? And is it wise to build an application framework on top of another application framework like this?
All of these are questions I’ll have on the back burner until this fall at the earliest. First I’ve got to launch Fictionsuit, then build features I have foolishly already planned for “soon after launch,” then whatever comes after that. Then I need to see if a big generalizing machine to enable other people’s creativity is still something I want to build at all.
If not… well, all this work has been fun. I’ve learned a lot, including the ability to let go of projects without all that guilt. :-) But mostly I’ve learned about my work habits and ability to take a large project to fruition (or something like it) without having it just loom over me, ignored, like a heartbroken pet zombie. Much.
April 18th, 2005
Found this paper on something Drupal is preparing or possibly not preparing, called the Content Construction Kit. I don’t know enough about the technical underpinnings of Drupal to be able to say exactly what’s going on here, for example, but even just the beginning of the next page tells me that there is some useful thinking there for how to extend Cornucopt’s content model out to the next level.
See, it turns out to actually be pretty hard to deal with things like search. So far, searching CoCo has been limited to title searches, but once you do full text search, how do you give users and/or admins a manageable way to say what it’s appropriate to search and what isn’t? Another way of looking at search is from the perspective of application-wiki problems. How do you tell your in-page widget to do something to a given set of pages? And truthfully, even title search is complex - I’ve already special-cased Corny’s database to keep track of what’s a comment so that they don’t appear in title searches. It’s not as if I should have to tell the changes table what every pagetype is, and write special SQL code for them all. And search isn’t the only such problem - RSS feeds are starting to look similarly tough.
So I’m hoping something in this document is gonna save my ass. The hope is that I will end up with an easy way for admins and/or users to define new pagetypes without writing PHP, and in some way consistent with permissions and generally not naughty.
February 3rd, 2005
Like every implementation of Cornucopt (or at least, so I imagine), the seekrit projekt (Codename: Sleeping Princess) will be a mix of actual, formal changes to Cornucopt (that is, pagetypes and libraries that will be part of the distribution) and custom code that remains specific to the project (that is, ugly-ass hacks). The information architecture of Codename:SP is pretty much done, although I’m sure we will make changes once we have a running prototype; what’s in question is which of those IA features are appropriate to be generalized into the CoCo codebase. And, as I alluded in the previous post, a lot of those features have to do with hierarchy.
Hierarchy in wikis takes a lot of forms. The most popular is probably simple categories, which aren’t technically hierarchic at all, but are more like plain old metadata or keywords - what the Flickr folks call tags. While the tags in Flickr and del.icio.us got folks excited about what’s being called folksonomies by some, and simply “aggregated wisdom” by others, the ur-Wiki’s categories arguably got there first. (Why the excitement now, then? A little interface goes a long way.)
Besides a soft user interface, wiki categories are a soft form of hierarchy, and sometimes it’s advantageous to have something harder. Some wikis have implemented what they call namespaces, a familiar-looking way of writing hierarchy right into the title of a page (that is, with slashes: “ParentPage/ChildPage”) and having it happen magically. I am not sure why this has always sort of turned me off; maybe because it seems important to handle hierarchy with a lot of transparency and consistency. Also because this kind of hierarchy is not really a change from flat wikilinking. Which is fine, unless you want to do something like auto-generated nav, or an auto site map, and have it provide any meaningful context to the user. Most wikis have nothing that resembles a site map, and hierarchy makes this feel more important. A dedicated wikismith can of course maintain something that at least looks like an exhaustive site map, on the front page or elsewhere… but, again with the doing it yourself versus software doing it for you.
Since the very early days of Cornucopt design, before I’d written any code and back when the whole project was both more and less like a wiki, I have wanted it to have buttons named “Create Down” and “Create Sideways.” The idea being that an individual hapless web-slave, charged with creating a corporate website or just a personal one with a mix of traditional and weblog/wiki flavors, could construct the kind of tabbed, menu’d page structure we see so often without having to work at it much, or at all. I think this is what simple hierarchy in Corny is going to look like. Ideally, I’d like to leave to the HTML and CSS templates the matter of how hierarchy is displayed. That way, the difference between a chaptered-and-paged documentation look and a top-of-page nav bar with dropdown menus could just be a matter of a few clicks.
The firmest hierarchical form of all wiki forms is the sub-wiki, in which wikis contain whole other wikis. These contained wikis may or may not make pages contained in other sub-wikis, or the master wiki, linkable, searchable, or even visible. You see this most often on so-called wiki farms, Blogspot-esque services that provide free or semi-free wikis. The surrounding stuff is non-wiki in these cases - you can’t nest isolated wikis within each other like Russian dolls. But mightn’t it be neat if you could? I don’t know what such functionality would be good for, but I can’t think of a reason to disallow it either. If we want the uses of Cornucopt to surprise us, and we do, we have to add this sort of capability.
However, for the moment, the thought of integrating sub-wikis into CoCo in a flexible, removable way - including constraining search to one sub-wiki unless the admin doesn’t want it constrained, constraining links similarly, maybe doing so on a sub-wiki by sub-wiki basis instead of a global preference, and on and on - makes me want to grow a long beard and wander around muttering “doom, doom, doom.” So I think it will remain a hack for now.
November 22nd, 2004
Because I’ve promised my November free time to another project, I’m going to see if I can’t get an 0.1 release of Cornucopt onto SourceForge by the 31st. That means enabling user bans and page locks (and maaaaybe IP bans but likely not), and killing that one login bug. Seems possibly doable over the weekend.
But what I’m really here to write about is something else. I’ve been thinking some more about this wikified-templates thing, and how it could lead to what you might call “third generation” wikihood.
I have, floating around in this humid little brain of mine, some thoughts as to how pagetypes are going to have to change in the near future. It turns out that, as lovely as the object-oriented model is for me to write, it’s a bit confining. A new pagetype can, as far as I know, only inherit the behaviors of one parent class in PHP 4. (Heh - you might call it a strict-father model.) So, for instance, if I wanted to add functionality to the basic Page class that added Flickr-esque tags, but I also wanted to have tags on SoftlinkPages, I would have to duplicate the tag code, unless I’m mistaken.
I’ve decided to regress, if that’s the word, to the “plug-in” model favored by packages like WordPress and Movable Type. Instead of subclassing all of the Page code and modifying or extending it, most pagetypes will just be a bunch of functions that get called when a page of the right type does its regular Page thing. This may be easier for many developers to understand.
It also opens another door: that of actually having the PHP code that makes a pagetype work… (deep breath) reside in the wiki.
Obviously, you will not want to give just anyone access to a wiki page that gets executed as PHP code. This action on my part will raise the stakes on hard security a great deal; if there’s some hole in my code that lets people fake admin access, or whatever other kind of access that has the permission to create and modify templates, then people could inject PHP code that wipes the database, destroys files on the host computer, infects the host computer with viruses, and just generally kills kittens. Yikes.
But, he said while wiping his brow, think of the upside. Imagine if extending a wiki’s functionality were as simple as clicking ‘create new page’ and selecting the ‘pagetype’ pagetype. That’s what JotSpot is after, although they are taking the (frankly much more sane) approach of a vaguely programmatic markup language. It may be that I eventually decide that I am smart enough to take that approach, or I may build some kind of extra frippery that makes editing pagetype scripts take longer, or restrains it otherwise. Or I may just give people enough rope to hang themselves; what do I care?
I really want to find a way to do this. This is HyperCard reborn, this is a revolution in waiting. I have no reason to expect that I will get there first or at all, but promise like that is the reason I write code.
I am sounding a little wild-eyed and need to be more clear about how this all follows logically from wikified templates, and about how pagetypes, pagevars and templates would fit together in this model. Watch this space.
October 27th, 2004
On the subject of what sorts of parts of the rest of the net might be useful to wikis, Corny, or both: social networking systems.
Not the ones we know and loathe today, mind you. Systems like Friendster, Orkut, LinkedIn, MySpace, Tribe.net and on and on all have inflated opinions of their own utility, and most are closed to outside use anyway. They don’t even make good use of the rest of the net themselves; they’re walled gardens.
No, the social networking systems I’m interested in 1) haven’t been built yet, and 2) will have, in true small-pieces-loosely-joined fashion, deliberately low utility by themselves. (In fact, the ones that are trying to have even a little bit more usefulness are only drawing fire for it.)
I’m thinking that external semantic social networks could help wikis form a kind of “firm security” as an alternative to hard security, keeping some of hard security’s strengths over soft, while eliminating some of its weaknesses. See, social networks are useful at validating a self-asserted identity. How do I know you’re you? Because all the people you know say that you are.
That concept alone is far from perfect, but it’s also far from as bad as you might think. Have a look at LOAF, which describes itself as an “extension to email that lets you append your entire address book to outgoing mail message[s] without compromising your privacy,” so that “[c]orrespondents can use this information to prioritize their mail.” Random email sent from someone who knows a friend of yours, as evidenced by that friend’s presence in their address book, might be more trustable than random email from someone unknown. There are holes in LOAF as currently implemented, but the general idea is pretty intriguing. Merge it with cell phones, and the Bluetooth capability thereof, and it might get hot in here indeed.
Building such a system as part of a given wiki engine would be missing the point, in my view; the point is small pieces, loosely joined. Let someone else build a general-purpose system of some sort, then don’t be so much of a not-invented-here Wiki snob that you don’t take advantage. That’s the future.
October 27th, 2004
And wikified templates! JotSpot has inspired me to move Cornucopt’s HTML templates (the HTML files, with wacky stuff written into them, that take raw CoCo-generated data and make them into web pages, via a package called Smarty) inside of Cornucopt itself, such that mere users might be able to write their own. (And such that the templates get the advantages of wiki, like a version history, et cetera.) Combined with pagevars and a helping of what B.A. Baracus would call “jazz,” this could get us almost all the functionality of JotSpot’s forms and searches. Right now, Cornucopt’s templates live in separate files, in a little folder called ‘templates’ in the CoCo file tree. This is much handier for my current purposes than having to edit them in a browser, which is why this change isn’t scheduled for a while.
Will there be security headaches? Well, you have to really trust users to let them write arbitrary HTML. JavaScript is harmful to children and other living things. IFRAMEs are just as bad if not worse. Those are the easy ones; determining exactly what subset of HTML users really need to build applications will take some thinking. Also, the searching thing really is half the battle - I may need to implement XPath the way JotSpot does to allow people to select the pages they need to. Allowing people to create pages that create other pages could get hairy, essentially becoming a flooding attack. So I’ll have to design that carefully.
Just taking the templates into the wiki is not so difficult compared to the rest of the picture. I should really make a detailed roadmap. That was kind of a non sequitur. I want to be able to do a cartwheel.
October 15th, 2004
I’ll start this (long!) post with a dirty little secret: I’ve never really been an active user of any large wiki. The closest I’ve come is having been very active on Everything2. E2 is not really much like a wiki at all, but one thing it makes a big point of, and that it does still have in common with the Wiki Way, is its status as a kind of microcosm of the larger Internet. You can’t link outside of E2 in an E2 writeup; you can type in a URL but it won’t be hot. E2’s interface affords a lot of inlinks, and its culture and original mission statement encourage users to make a lot of internal resources rather than utilize resources at other sites. If you want to link to HyperCard, for example, you throw some square brackets and link to E2’s node on HyperCard, not the more complete page on Wikipedia. The rationale is that if you want to link to a better page on HyperCard, you (or the community in general) should make a better writeup on E2 rather than taking energy off the site.
Now, efficiency is not the king of humans, nor of the network. You don’t have to be psychotic about avoiding the duplication of effort. E2 has emotional reasons for having its own pages on things (some of those reasons are rather confused IMO, but that’s not the point). That’s fine; that’s what makes communities communities. But let’s widen the focus back out to wikis in general.
The most common forms of outward linking you see on wikis are 1) plain, old-fashioned HTML links to somewhere else (sometimes visually distinguished in the read view, sometimes written with a different syntax in the edit view, and sometimes just bare, visible URLs); 2) InterWiki links. The latter are much more systematized and “wiki-like” than the former (although plenty of folks do like to bicker about the best way to do syntax for outlinks, or for links in general). In a way, InterWiki was low-hanging fruit; if there are all these other sites that run on software that’s pretty similar to yours, you just have to do a bit of coordination to have them link to each other the way they link to themselves, then all the wikis in the world are one big wiki. That’s very cool, but it’s also a missed opportunity. Why stop at wikis?
Because it’s easier, is the largest part of the reason. But the whole point is that I’m not talking about building wikis out until they do everything. (Although it sometimes feels like that is what I’m doing with Cornucopt.) I’m talking about, how do we make better use of the rest of the net? How do we incorporate the best of “small pieces, loosely joined” without sacrificing the simplicity that we showed up for?
I’ll give you an example by hopping back to E2 for a second. A year ago I was still noding now and again, hoping to finally get up enough writeups to graduate to level 4, whereupon I’d be able to C! things. C!, pronounced “ching,” is E2’s way of letting users give a special shout-out to a node and to highlight to other users what they themselves think are the best writeups in the system. You don’t get to do it until you have enough writeups to earn it, which is what the level system was all about.
So no shit, there I was, writing like a functionary, like it was a job, for a small, insular audience and towards a confused cause, because I wanted to be able to tell people what I liked. And it dawned on me: why don’t I just tell people what I like? A del.icio.us account later, I was in business, and after a couple of posts to the E2 community diaspora, others were too. In fact, if I’d used E2’s C! system, I wouldn’t have had an RSS feed, annotation, or any of the other nice stuff I have at del.icio.us by default. E2’s information architecture sets C!s up as a carrot to reward user effort, but tastier carrots can be had for no effort. E2, like wikis, tends to forget that the rest of the net is there.
So how best to use the rest of the net on wikis? Just link to the rest of the net, might be the answer of a true-hearted wikizen. Keep it simple. But the key difference between InterWiki links and other outlinks is that you don’t have to type http://. InterWiki links incorporate the central thing about Wiki: linking is easy, or easier. (You still have to know the name of the Wiki you’re going to, and that it exists, and how it’s named in the InterWiki system, at which point you probably know enough to copy and paste a URL; that’s why I like SisterSite systems better, but moving on.) Linking is part of the interface. You don’t have to do it all yourself; the software bloody well does it for you, because that’s why we have software. When you write software, it matters that you build something in, and it matters how you do it.
These issues can be extraordinarily hard to think through, which (again) is a large part of why wikis don’t bother. They have other worries more central to their philosophy. In the weblog world (and elsewhere), you see a lot of chatter these days about Web Services, and building web applications as a layer over other web sites, and something called REST and a whole lot of other nonsense. The debate over how to apply these ideas to wikis is still pretty furious, and it’s gonna be even more of a headbreaker working out how to open all the disparate pagetypes and resources in Cornucopt to effective, non-exploitative use by the rest of the net. Figuring out how to do the reverse is not so bad by comparison, although it is giving me some agita (oh God, what about multiclassing? how the hell will I do that in a way anyone will understand? what about letting end-users use plug-ins to drop in external content? will that screw me?). But at least I have a consistent philosophy of “small pieces, loosely joined” to guide me (along with the fact that, at least half the time, Cornucopt installs are not going to be very much like wikis).
E2’s objections to outlinking are well-documented, and although wikis seem not to hate it as much, their tendency to “keep it in the family” anyway reflects a desire to privilege their own way of doing things. That’s one way to think about building any software feature: it will encourage certain behaviors and discourage others. Another way is: it will enable certain behaviors and disable others. I think the latter is what you need to focus on when your mission is to see what people do with your software. The point of opening things up is not to circumvent anyone’s ideals; the point is to take advantage of network effects we haven’t foreseen, and offer more ways to do things - more tools - as well as more emergent, unforeseen things to do. That’s also the point of Cornucopt.
September 14th, 2004
It’s looking like PhpWiki has matured technically a whole damn lot. It has WikiPlugins, which can probably be made to act like my envisioned pagetypes. They could even be more flexible than pagetypes, but, accordingly, harder to control. It also has SubPages, which is just a simple hierarchy mechanism - combine that with some key plugins and you can basically hack up something that looks a lot like Everything2. (Aping E2 isn’t the goal of the Wicker project, but it’s one useful benchmark - if we can do all that, we can probably do a lot of other, more interesting stuff too.)
So I’m tempted to drop Wicker and start hacking on PhpWiki instead. But the trouble with this comes down to implementation, as it always seems to do in Wikiland. It’s always difficult to understand other people’s code. My objections come down to interface, mostly - the specific way PhpWiki lets people into all this functionality just feels sort of wrong to me. And, as I sort of covered earlier, that seems to be why there are so many Wiki codebases. Some developer says, “that’s not the way I’d do it,” and soon there are n + 1 Wikis, where n = too damn many. I’m self-conscious about contributing to this problem.
So, as my paranoia increases about little problems in my code - security, inflexibility et alia - the temptation to cut bait and hack on something else increases. With that action would die the already-unhealthy dream of making some money off this software. I really just want my mammoth wiki-weblog-E2-like playground…
(Side note: RSS feeds for comments on individual posts really suck compared to MT’s built-in Recent Comments for an entire blog, which can be RSSed just as simply. I’ll correct this, uh, sometime.)
October 11th, 2003
For everyone that says, “I get too much email, dammit, let’s work together on a wiki,” there will be two others that say, “I hate having to go off to some web site to see what’s going on. Can’t we do this on a list?” You don’t notice this now, but I guarantee you will as Wiki usage spreads outward from the alpha geeks to the rest of the world. Businesses are trying to reduce email, sure. (They’re also trying to reduce churn, and wikis can create plenty of churn when used to tackle technical issues collaboratively. That’s only one way to use them, however.) But individuals often stumble, still, whenever they stray from it. Admit it - you know somebody who’s intelligent and gives a lot to the world, but doesn’t really understand the Bookmarks menu.
You can fight over whether the wiki should come to the people or vice versa, or you can just build nice email features into your wiki for those who want them. I plan (eventually) to do the latter. But what will they look like?
Subscription is the most obvious. Just as you can, in many message boards and weblog comment engines, check a checkbox to receive email notice when someone follows up on a comment, you ought to be able to look after a wiki page you create or edit. (Or just want to keep tabs on.) Enough people subscribe to the same page, you’re halfway to a mailing list right there.
The weirder problem is editing the wiki from outside, via email. You don’t want this to be a kludge or insecure, like many email-to-weblog gateways I’ve seen that have you put usernames and passwords in the subject line or first line of your email. You can’t use HTML email with fancy stuff done in JavaScript either, because, well, people hate that shit. What you can do, though, is create mailboxes for each page on the wiki and have them behave basically like mailing lists.
…But that would make people treat it like a mailing list. What kinds of afforrdances could we give a wiki email interface that would help keep the discourse wiki-ish? Or, if you’re feeling saucy, what other forms are possible when you combine email with a web collaboration space? And we haven’t even touched on how to bring multiple wiki pages into this. I have a feeling, though, that RSS is going to be more relevant there.
(Afterthought: hmm… automatically determining which wiki pages are closely related, by number of links? Vulnerable to attack. And it doesn’t really have anything to do with email. But people are going to need more efficient ways to slice this hunka meatloaf I’m putting together… grr.)
September 28th, 2003
Wikis are geeky; we’ve covered that. The deeper reason that wikis are geeky, though, is their very flexibility. Geeks get excited about wikis because you can make them do anything… but more casual users get un-excited about wikis because you have to make them do everything.
Weblogs do one thing and one thing only. But they bloody well do it for you! Wikis can do anything, but you’ll be doing it your damn self. I noticed this several months ago when a brief murmur about “wikilogs” went through the weblog crowd. I went to the example sites they pointed to, checked to see if they were doing anything interesting to meld anything-goes wikis with classy, well-oiled-machine weblogs… and they were wikis with a front page that was always edited by the same person. They put new stuff at the top (usually). When stuff hit the bottom, they refactored it to a new page. And I thought, “That’s it? That’s what the hype is about? This isn’t a wikilog, this is a wack-ass hedge!” (More development along wikilog lines has been done since then, and this discussion at MeatballWiki is, well, meaty.)
Maybe wikis make editing a page easy enough that people shouldn’t consider having to do that a big deal. And yeah, maybe a lot of weblogging tools are getting so complex, with their multiple categories and trackbacking, that weblogs don’t have a big usability lead. I think those problems are surmountable and not the real issue.
The real issue goes beyond just integrating wikis and weblogs. The issue is: what if we could teach wikis? What if we could tell a wiki, “when you’re on this page, don’t treat it like just any old page; make it do this specific task really well instead”? Or, “treat it like a regular Wiki page and give it this additional functionality”? If we could do that, Wiki could have a shot at being the duct tape of social software.
I remember way back when I was eleven and twelve years old, reading HyperAge, the magazine for HyperCard enthusiasts. (My having been exposed to Vannevar Bush’s “As We May Think” at this age explains a lot about my subsequent life history.) There was an interview with Bill Atkinson, if not in HyperAge then somewhere else, in which he talked about how humans couldn’t swim as well as a fish or reach things as well as a giraffe, but we could swim better than a giraffe and reach things better than a fish. Plus, we know where the ladder is kept. Then the metaphor became, well, HyperCard knows where XCMDs are kept, et cetera.
We don’t want wikis to become bloated messes, full of functions and syntaxes signifying nothing. Pluggability might be good though. Then a Wiki is a framework you can build on. (Zope is a framework too, but most shared hosting providers… frown on running your own web server binary. Plus I’ve heard that Zope has other problems, but… anyway. Implementation decisions to be frantically justified later, if ever.)
September 18th, 2003
Previous Posts