I finally pushed out some embarrassingly outdated WordPress plugin updates a few moments ago.

• Updated unAPI plugin with a patch contributed by Jay Luker that removes the hard-coded "wp_" table prefix. The updated version of the plugin has been tagged as 1.4.1.
• Updated LinkPURL plugin with a patch contributed by Mark Matienzo that enables partial redirects. I made some additional tweaks to the plugin to make this feature configurable via the WordPress management UI. This has been tagged as 1.1.
• Created a new unAPI plugin branch for Mark Matienzo's Scriblio-oriented tweaks. The branch is called 1.4.1-anarchivist-scriblio and it contains the scriblio.diff file. I have yet to integrate the diffs, as the file that was patched has changed since the patch was issued. If anyone is interested in working on unAPI/Scriblio integration, please get in touch with me.

And here is my to-do list which I hope will keep me honest.

• Update OAI-ORE plugin to support version 1.0 of the ORE specification.
• Add per-post (and per-page?) resource maps that wrap all embedded images and links.
• Enable "cool URIs" for all resource maps.

It is my hope that I'll get to those sometime before the summer begins. :)

Hear ye, hear ye, techie librarians and library techies:

Birkin James Diana, representing the host, Brown University, announces a call for proposals for talks at the code4lib 2009 conference:

The code4lib conference is wonderful in large part because of what you folk share.

Head over to the Call For Proposals page and submit your idea for a prepared talk. Information about the time-frame, scope, and evaluation process — as well as the link to the submission form — can be found at:

http://library.brown.edu/code4libcon09/proposals/

Some important notes:
- Proposals can be submitted through Sunday November 23 2008.
- Just like when you vote, use your code4lib username and password.
- You initially won't be able to edit your proposal(s), but I'll have that implemented in the near future.

More information on prepared talks from the proposal page:

Prepared talks are 20 minutes, and must focus on one or more of the following areas:

* "tools" (some cool new software, software library or integration platform)
* "specs" (how to get the most out of some protocols, or proposals for new ones)
* "challenges" (one or more big problems we should collectively address)

The community will vote on proposals using the criteria of:

* usefulness
* newness
* geekiness
* diversity of topics

We cannot accept every prepared talk proposal, but multiple lightning talk sessions should provide everyone who wishes to present with an opportunity to do so.

Do consider submitting a talk if any of that sounds interesting to you.

I know precious little about rights management.  And what I do know I have gleaned from the occasional Slashdot post or Wired article.  Former colleague Grace Agnew, Associate University Librarian for Digital Library Systems at Rutgers University, has put the wraps on a book about digital rights management targeted at librarians, Digital Rights Management: A Librarian's Guide to Technology and Practice (also available in paperback):

This book provides an overview of the current landscape in digital rights management (DRM), including: an overview of terminology and issues facing libraries, plus an overview of the technology (including standards and off-the-shelf products). It discusses the role and implications of DRM for existing library services, such as integrated library management systems, electronic reserves, commercial database licenses, digital asset management systems and digital library repositories. It also discusses the impact that DRM ‘trusted system’ technologies, already in use in complementary areas, such as course management systems and web-based digital media distribution, may have on libraries. It also discusses strategies for implementing DRM in libraries and archives for safeguarding intellectual property in the web environment.

If you're a librarian or information professional looking for an introduction to DRM, an underpinning for rights management strategy, or a refresher on rights management technologies, you might consider checking it out.

For full disclosure, I was one of several reviewers of this book.

I've been using my time at RepoCamp today to get the OAI-ORE plugin for WordPress validating again.  I'm having some trouble using the validator so I say that with some diffidence.  But the latest code which is now checked in to the WordPress plugins svn repo ought to be close, if not fully conformant, to the 0.9 version of the ORE spec.

I'm not sure the plugin is really useful; it's just an Atom feed of all posts and pages in a WP instance.  I can think of some ways to make this more useful, by allowing blog authors to create their own aggregations, pulling in content outside of the particular instance.  I am certain that others can come up with even better uses.  I'm open to suggestions.

Thanks to Jay Datema for prodding me a bit, if indirectly.

About a month ago, I read on my colleague's blog that the Emory University Digital Library published a new book on sustaining digital libraries.  I've finally started reading it and figured I would post a note here.

The articles of this monograph provide resources for digital library stakeholders who seek to better understand how to effectively evolve such efforts from short-term projects to long-term sustainable programs. The monograph includes contributions from leaders in major digital libraries that have made such transitions or which are systematically considering the question of programmatic sustainability, including representatives from the National Digital Infrastructure and Information Preservation Program (NDIIPP) and the National Science Digital Library (NSDL).

I might also note that the book is available for free as a PDF.

So far I've read the introduction by the editors and the abstract from Leslie's paper, and the book looks like a high-quality read from cover to cover, with articles based on actual digital library experience.  It's a pragmatic approach for how to sustain digital library initiatives, looking beyond technical concerns towards the more challenging social and economic ones.  To some extent, we are getting pretty good at preserving bits and relationships between collections of bits — it is yet to be seen how good we will be at preserving the preservation systems themselves.

It's hard to believe but I've been at the new job for six months already, a full half-year come the 29th. Some days it seems like I've been here forever; others like I'm still a rank newb. I haven't written terribly much about what I've been up to (but I assure you I've been busy). Let me rectify that.

The Transfer Problem

Two of the projects I've been working on relate to a fairly general problem that we like to call "transfer," which revolves around, well, transferring files to and fro. Sounds simple. Is simple. That is, until you start thinking about preservation and accounting for a highly heterogeneous network with idiosyncratic nodes, esoteric storage software, and differential firewall rules. And that's where it gets interesting (and problematic). The transferring itself, or copying of files from one location to another which we call "transport," is the easiest part. We like to use common tools in our environment. It makes life easy. And so good ol' scp seems like an obvious choice to handle the job.

Since preservation is a core aspect of our "repository," a term which I use loosely, we must build certain other functionalities into the transfer process: validation, verification, inventory, backup, ingest, and so forth. Every time a file is copied to a non-transient location, we verify the file against a (SHA1) checksum and record an event for auditing purposes.

Repository Workflows

Some steps in the transfer process are routine and best handled by machines. Thus we automate them with scripts and code. Others require human intervention. This introduces another key aspect of our repository needs: workflows. No two projects will have the same workflow and yet all will have some steps in common. We're using JBoss's jBPM library for workflow management and it is more than capable of handling our workflow needs. It allows us to model complex and varied flows in a robust and not ad hoc way; it does seem preferable to me to model our workflows via jBPM's graphical editor (serialized to JPDL XML) rather than copying around blocks of code and otherwise modeling the workflow procedurally in business logic.

One of my coworkers (the author of this) designed a complete workflow system in jBPM last summer and I've taken on implementing, tweaking, and testing this system, which has required bootstrapping my sorry-arse Java skills and learning jBPM. Though I find it difficult to think in Java patterns and generally find it a burdensome environment, I'm quite impressed by jBPM. I've been working on various updates of and unit test coverage for the workflow system, which has been a crash course in a number of Java technologies and a perfect first task at LC, as it gets me into the guts of things. The Java stack we use for our workflow is highly abstracted and componentized, which is conveniently modular… but it's also Java: fairly heavy and arguably not as agile as dynamic languages such as Python.

Two Great Tastes?

So recently we've begun to think about implementing some transfer and workflow components in Jython (Python written in Java). Why? The value of Jython is as follows:

1. Ease of deployment - Deploying jar files to existing JVMs in production environments (which we do not control) is a simple task, or at least simpler than some other options.
2. Interoperability - Our stack is primarily Java-based and so interoperating with existing Java components means not having to rewrite functionality in other languages. Jython allows Python to talk to Java and vice versa.
3. Familiarity - It's Python, and we like Python. It's the closest my team has to a lingua franca and so it increases the chances of sharing code, maintaining code, and so forth.

It does not come without its drawbacks:

1. Currency - The Jython project went moribund for a few years or so and the latest stable version is now 2.2.1. Compare that to the latest version of Python: 2.5.2. I don't begrudge the Jython developers, though. I'm glad some folks picked the project up, dusted it off, and breathed new life into it. I am also glad that they're skipping 2.3 and 2.4 releases and plowing right into a 2.5 release. Because of this currency issue, some Python libraries won't work with the latest Jython and that means you're stuck looking for outdated, potentially vulnerable Python libraries, or hooking into Java libraries for the same functionality (which inevitably means more lines of code). I'm no lines of code fetishist but it does militate against the goal of agility somewhat.One does have the option of living on the edge and trying out the 2.5 branch, but that seems out of step with an infrastructure that is supposed to preserve terabytes upon terabytes of our nation's, and the world's, intellectual property. It's a responsibility I do not take lightly, as much as I'd like to be on the bleeding edge.
2. Interoperability difficulties - Talking to Java from Jython is a snap: just import java at the top of your script, and, assuming your classpath is copacetic, voila: you have access to Java libraries in your Python code! Talking to your Jython modules from Java code is, well, a little more complicated. Read on.

Despite the caveats it does seem a sane, reasonable, and potentially productive path to go down. Right? I am specifically looking to implement two workflow components in Jython: one for transport (wrapping Ant's JSch library, which provides a slick scp API) and the other for automation of ZFS filesystem/volume creation on the staging server. Nothing arcane, nothing tricky, nothing fancy. So it must be easy! Right? …

Lessons Learned?

I'm beginning to wonder about the feasibility of using Jython to make bits of our Java stack more agile. Specifically, there are three ways to get at Jython code from Java:

1. Compile to bytecode/jar via the Jython compiler (jythonc) and reference your Jython objects and methods as though they were POJOs
2. Embed a Jython interpreter
3. Instantiate a (JSR-223) script engine

Option 1 is nice because you get object- and method-level interop. However, jythonc is unsupported and will disappear. This does not seem sustainable though I might be able to limp along a while. And there are signs of hope:

Though jythonc is going away, all of the capabilities it provides will be present in 2.5 in other forms. We're adding functionality to expose Python classes as Java classes using decorators to replace the docstring class creation that jythonc provided, and we're adding static compilation of proxy classes so regular jython can run in applets and other environments with restrictive classloaders. We're definitely doing something about jythonc.

That doesn't help much now, of course, but just because jythonc goes away does not mean my jars will stop working.

The suggested methods for option 2 [1, 2] seem to be more trouble than they're worth. If the goal is more agile development for certain components, the reliance upon multiple, separate Java classes — an interface class and an object factory, in the examples listed — to get a ten-line Jython script working, this seems suboptimal, both inefficient and not straightforwardly maintainable; it seems, to me and my Java-dumb ways, rather baroque.

Option 3 [1, 2] is more appealing than option 2 as it does not rely upon these other classes specifically for Jython code. But the number of lines of Java code that must be wrapped around the Jython to get it working looks like overkill for the drop-dead simple scripts I'm writing — it might be easier, for instance, to just write the darn things in Java and be done with it. (Did I just say that?)

Conclusions

Options 2 and 3 are similar as both involve embedding Jython code, or referencing files with Jython code, and interpreting the code within Java. Generally, I worry that either option would obviate the benefit of agile Jython scripting because you wind up wrapping the code in so much Java. I offer two disclaimers to counter my objections:

1. Cleverer Java coders than myself could, I am almost certain, find ways to build abstractions (or abstractions of abstractions) to eliminate the "lines of code" and "many separate classes per Jython script" issues
2. The value of Jython in our Java environment increases proportionately with the complexity of the component — given the overhead, a short Java class seems easier and more straightforward to implement than a short embedded Jython script. On the other hand, there's value in embedding a Jython script that'd be an order of magnitude simpler than its Java analog.

At least, that's the state of my head right now re: getting Jython and Java to play nice. If I make any breakthroughs or give up entirely, I'll post follow-ups.

I am but a Java philistine and a Jython neophyte, so I remain humbly open-minded. I would greatly appreciate comments, questions, corrections, smackdowns, sagacious advice, and so on.

… meanwhile, the Code4Lib Journal has published its second issue and boy is it packed with articles; Eric Lease Morgan, Coordinating Editor of the issue, does a bang-up job on the introduction (though the title has effectively Bostonroll'd me).

Each article in this issue has a little bit of something for all who call themselves a librarian or work in a library. Each identifies some sort of library problem to be addressed, and offers one or more solutions. Many are complete with code snippets. After all, this is Code4Lib.

For example, people in public service may be interested in Edward M. Corrado and Kathryn A. Frederick’s review of database-driven subject guide applications. Kenneth Furuta and Michele Potter describe a simple help system that brings librarians running to the reference desk. Margaret Mellinger and Kim Griggs explain how library resources can be organized into course pages without the need of HTML knowledge and yet sport Web 2.0 features. Nancy Fried Foster, Nora Dimmock, and Alison Bersani shed light on participatory design.

For those of us who enjoy cataloging and metadata issues, Jonathan Gorman outlines how he modified VUFind to exploit Wikipedia and cataloging authority records to enhance information about authors in a library catalog. Chris Freeland, Martin Kalfatovic, Jay Paige, and Marc Crozier illustrate a different use of Library of Congress Subject Headings by integrating place names with Google Maps. Carol Jean Godby, Devon Smith and Eric Childress describe a technique for crosswalking just about any metadata format into just about any other metadata format.

For the systems librarian in you, Dan Scott and Kevin Beswick share how they used Linux live CDs customized as kiosk browsers to provide laptops as ‘quick lookup’ stations at their library. Andrew Darby takes advantage of the Google Calendar API to easily manage the display of library hours. Jody DeRidder exploits Google sitemap technology and static HTML pages to make content in the “deep Web” more accessible. We hope you find these articles useful, stimulating, and relevant to your daily working lives.

I am ashamed to admit that I have not yet finished the first issue, so I now have pages upon pages to read. Ordinarily when I am behind on my reading I wind up letting bits fall by the wayside but the material largely looks too good to ignore.

Congratulations to Editorial Committees past and present and to the community on the whole!

This is very rough, but here's a WordPress plugin that provides a resource map for the aggregation of all posts within an installation of WordPress. I'll be working on this some more, but for now, it does appear to work and validate (as Atom). Useful? If so, I'll zip it up and commit it to the wp-plugins svn.

Note:Ed reminds me that xsltproc can be used to transform the Atom-based resource map into RDF via GRDDL:

xsltproc http://www.openarchives.org/ore/atom-grddl.xsl http://lackoftalent.org/michael/blog/wp-content/plugins/oai-ore/rem.php

Update: The plugin has its own page.

Dorothea left a comment on a post announcing the publication of a little conference review some colleagues and I splurted out. In the announcement I lamented a bit about impact and she wisely suggested I consider depositing the review in a subject repository such as E-LIS.

We looked into our agreement with the publisher and it was actually quite permissive. (Way to go, Emerald.) And here's the review in all its open access glory.

Thanks, Dorothea!

Reading Adam Smith's D-Lib article has got me thinking about identifiers again. I don't agree with some of the assertions in the section titled "A Persistent Identifier Primer" — URIs are in fact persistent; we just break them through poor management — and so I'm led to a fundamental question: what are the good use cases for Handle (or ARK, or PURL) identifiers?

I get the need for persistent and globally unique identifiers; I'm just wondering why one needs special software with a separate URI namespace to gain persistence.

One potential use case might be resources that are outside of the organization's control — i.e., licensed content from vendors — but surely folks are using Handles for many resources that are created and managed within the organization. And I'm curious why they have decided that Handles are more durable than native URIs (the URIs to which Handles redirect), and how they deal with the problem of downstream (post-redirection) citation and bookmarking. How useful is this sort of identifier scheme if your users never even see the supposedly more persistent URI for a resource?

As a former proponent of Handles and ARKs, this may seem like a hypocritical question to pose. If I had to answer my own question, I would say that Handles seem like a good option because they save you some work and headaches in the short-term; you don't need to get together with your web team and come up with a scalable and sustainable URI policy; just assign native URIs in the usual haphazard way and generate Handles to compensate for a lack of identifier policies.

But if you're already making an organizational commitment to identifier persistence — and if you're rolling out Handles, I'd wager that's likely — why not do so by minting carefully-considered cool URIs? Less management and technology overhead and less confusion for your users are two good reasons to consider it.