We’ve just had an interesting couple of days at the Ontogenesis Blogging a Book Meeting. I found myself adopting the position of naysayer on a few occasions (that’s usually Phil’s job, but he was running the meeting), raising questions about the process and technologies being applied. This post is an attempt to reflect on the meeting and try and identify why I was uncomfortable with the exercise, and what one might do to address the situation.
I should first make it clear that I agree with the overall aims of the exercise (see below). In addition, although this commentary is perhaps negative in a number of ways, it is intended to be constructive criticism. This was a very interesting and thought-provoking couple of days. Many thanks to Robert Stevens, George Moulton and Phil Lord for organising the meeting and providing the initial stimulus. Now, out with the knives!
Note that the opinions expressed here are mine and may not represent the views of others involved in either the ontogenesis network or this particular meeting. This may also be a slightly half-baked rendering of my thoughts and may be subject to review!
What were we trying to achieve? I think the meeting had two purposes.
Writing a number of short “encyclopedia style” articles relating to ontologies (and their use in bioinformatics).
Investigating new models for the publication process, in particular the use of a blog in order to manage the review process.
The former was to be realised through the latter. The rationale for A is clear, for B, the intention is to try and reduce some of the overhead and time delay that can be present when using traditional publishing routes. However, in my final analysis I think the difference between the kinds of short article for an encyclopedia and longer scientific papers means that the process hampered us somewhat in the production of our initial articles (“why didn’t we just use a wiki”). This is not to say that the blogging approach is not appropriate as a mechanism to support the publication process (in fact I think it might work fine if tweaked), but the jury’s still out.
The process for the meeting was roughly as follows. A number of topics for entries were identified. People selected topics that they wanted to write an entry for (in some cases this may involve multiple authors). Entries were then written as a blog entry. Once the author considered the entry ready for review, it was tagged appropriately. Reviews were also written as blog post. Through the use of the WordPress’s trackback (or was it pingback?) mechanism, by including a link to the original post in the review, the review appears as a comment in the original post.
Categories were used to indicate the status of articles (under review, reviewed, peer review), while tagging indicated content.
The idea is that through the use of the commenting mechanism, we can preserve a trail of comments and reviews, which not only provide an insight into the evolution of the article, but also provide some attribution and credit for the work of the reviewers. Note that this assumes that the process is open, with reviewer’s comments and identity visible. This is a great idea, but the meeting showed that there are some issues with the actual delivery of this using the technology.
The meeting was focused on writing short, collaborative articles with a quick turnaround (we hoped to produce a number of initial articles by the end of the meeting). However, such writing is very different from the lengthy scholarly articles that one might expect to find in a journal. Encyclopedia articles tend to be (or at least should be, imo) objective and factual rather than opinion or subjective interpretation. The kinds of review required for the activities are different. For a short article with rapid turnaround, I would like to get quick feedback about whether there are significant pieces missing, or whether content is on- or off-topic. The fact that we were all co-located also meant that I wanted/expected quick feedback — ”shouts across the room”. A review of a full paper would be lengthier and in-depth, and could also required more detailed referencing of specific sections of the original article. I’d also be happy to wait longer.
A number of times during the meeting, the question “why aren’t we doing this through a
blogwiki?” was asked. Valid question, and one which I’m not sure there really was a good answer for. The reality is that what we were trying to do falls somewhere between what’s offered by a blog and a wiki. A wiki may well have been a better environment for supporting the collaborative writing and commenting process for the encyclopedia.
Authors and reviewers needed to communicate. For example, the process as defined required authors to request reviews for articles. This was done “out of band” (e.g. without using the mechanisms of the blog). In practice, this was done by email, or simply by direct communication (as all the participants were actually co-located for the two days).
There were occasionally suggestions that reviewers/authors could communicate directly, e.g. in order to exchange information on minor typographical errors. In this case, distinguishing what kinds of communication should occur in band and out of band is important, particularly if records of the communication are intended to be part of the process — at what point do grammatical changes become substantial changes to the intention of an article?
As an aside, I found the WordPress UI a nightmare to work with. I don’t really like browser based editors, so prefer to author text using a text editor, and then cut’n’paste into the web tool. Once I’d pasted text into the text box, however, I found it messed with my underling HTML markup (adding lots of <br/> elements and stripping all my <p/>‘s out). Tables seemed problematic too, although that may be something to do with the underlying style.
In order to prevent this from just being a pointless rant, I would like to conclude with some suggestions/observations. This was a useful activity, but I would probably approach it in a different way if I was to repeat it. Below are a number of questions/points that I think need further investigation. Some of these are technology related (e.g. WordPress doesn’t do what’s needed), some are more about identifying the process and requirements.
A clearer identification of the process. What are the different steps/phases that an article will go through? This is particularly important, as I think the processes involved in writing the encyclopedia are different to those that would be in place for “journal style” reviewing.
Versioning. What is the versioning strategy that is used? Are articles edited “in place”, or should edits result in a new article? In which case, should all edits result in new articles, or can we fix typos? Who then decides what edits are “acceptable”?
What’s the role of the editor (if any)? Do we need a central controller?
Communication between authors/reviewers/editors. Mechanisms are needed that allow communication between the various actors. What is in-band and out-of-band? How much did/does the physical co-location of the participants impact on the process?
The ability to deal with different kinds of information in a review. There can be comments about the presentational aspects of the work (e.g. typos, grammatical errors and so on), as well as more substantive comments relating to the content of the work.
A clearer identification of the activities/tasks that actually required interaction and communication, and how those are managed. A number of things (soliciting reviews for example) were done “in person”. Clearly this would not be possible if participants hadn’t been co-located (although email would also work).
Some of the points above (e.g. 1 and 2) need to be considered independently of the technology being used to deliver them (although it is important to bear in mind what is possible/feasible). There was an occasional tendency to bend what we were trying to do to fit with the WordPress functionality (e.g. single categories).