Representing research outputs

While most of the SWIG discussion focussed on semantic web representation of metadata about research entities (metadata about data collections, projects, researchers etc), the group also discussed approaches to representing actual research outputs (such as datasets and annotations) using semantic web technologies. Projects discussed included:

• The Aus-e-lit project that has implemented a compound object authoring and publishing service based on the OAI-ORE semantic web model. Jane Hunter from their team discussed the advantages of using semantic web technologies to represent research data. She suggested that semantic web approaches can succintly model relationships between entities, such as the relationship between a “cleaned” dataset derived from a raw dataset. Semantic web representations also makes it easier to create visualisations of these high level relationships and present them to end users.
• The Aus-e-lit researchers have recently joined the Open Annotation Collaboration project. Social science, humanities, and crystallography communities create annotations in the course of their research, but find it difficult to share annotations between software systems. The Open Annotation Collaboration project aims to use semantic web approaches to move annotations across the boundaries of annotation clients, annotation servers, and content collections.
• The ADFI team at USQ have contributed to the Beyond the PDF project, examining semantic web approaches for linking papers to datasets and other related information.
• The Health-e-reef project used a high level ontology to unify coral reef survey observations. The ontology provided a way to unify observations from disparate data sources created by multiple independent projects and with different data strucures. The ontology models concepts such as Observations, Actors, Sites, Ecological Processes, and Measurements. James Cook University commented that they might use this ontology as part of their tropical data hub initiative.

The SWIG discussed how representing data, as opposed to metadata, using semantic web technologies introduces new scalability challenges (moving from thousands of RDF triples to millions of RDF triples). For example, CSIRO presented a paper at the 2010 eResearch Australasia conference about performance issues when querying and accessing the 30 million RDF triples in the Atlas of Living Australia dataset. CSIRO tested both open source and commercial triple stores, and observed execution times of over 8 hours for some queries, with variations of up to 9,000 times between some systems. In related work, Campbell Allen wrote a masters thesis that benchmarked RDF triple stores against a traditional relational database. He found that the relational solutions outperform the triple stores, especially for spatial queries. Campbell did observe, however, that

The Semantic Web RDF triple stores were found to be particularly suited to the data integration task at hand due the ability of ontologies to semantically define and link the data

The SWIG felt that these RDF triple store scalability issues may iron themselves out over time, observing that relational databases had to overcome similar scalability problems early in their development.

Representing metadata about research outputs

Many of the projects at the SWIG use terms from the ANDS-VITRO ontology to describe their research outputs. Some noted, however, that they had difficulty knowing how to apply the ontology and would benefit from documentation of best practice (or at least common practice) for representing research metadata as linked data.

The rest of this section provides an overview of common practice areas that might need documentation. The SWIG raised the topics below, but I have extended the discussion with more detailed examples to illustrate some points.

Best practices for using ANDS-VITRO and VIVO terms

The ANDS-VITRO and VIVO vocabularies can represent some research metadata concepts in multiple ways. For example, the VIVO ontology contains both vivo:webpage and foaf:homepage properties for describing links to webpages. Similarly, the ANDS-VITRO to RIF-CS metadata crosswalk document (available from the ANDS-VITRO google group) contains many properties for describing Agent names:

• bibo:prefixName
• foaf:firstName / foaf:givenName
• foaf:lastName / foaf:familyName
• vivo:middleName
• foaf:name / rdfs:label for display

The SWIG felt that the community would benefit from guidance on using these types of “overlapping” vocabulary terms. Peter Sefton from USQ related his experience in the ARROW project where agreeing on common usage early could have avoided interoperability problems down the track.

Supporting linked data principles

Melbourne, Griffith and QUT originally created the ANDS-VITRO ontology to represent ANDS registry objects within the VIVO system. More recently, non-VIVO uses of the ontology have also emerged, with some institutions using the ontology to create linked data representations of their research metadata. The documentation supporting the ontology, however, has inconsistent support for some of the principles championed by the linked data community. In particular, The How to Publish Linked Data on the Web tutorial suggests the following principle for choosing vocabulary terms:

In order to make it as easy as possible for client applications to process your data, you should reuse terms from well-known vocabularies wherever possible.

The ANDS-VITRO to RIF-CS metadata crosswalk document has mixed support for this principle. It supports the principle by recommending re-use of the Dublin Core description element (dcterms:description) for narrative descriptions of research data. On the other hand, the same document breaks the principle by recommending use of vivo:hasSubjectArea rather than the much more widely used dcterms:subject term for describing the topic of a research data collection.

These inconsistencies probably stem from the original purpose of the ANDS-VITRO ontology as a representation of ANDS registry objects within the VIVO system. The designers presumably did not consider linked data as a primary driver. Given SWIG interest in linked data representations, however, the community would benefit from a discussion about using more common linked data vocabularies and how these align with the ANDS-VITRO ontology.

Expanding the scope of ANDS-VITRO

The SWIG discussed expanding the scope of the ANDS-VITRO ontology.

The current version of the ontology only contains a subset of the concepts covered by ANDS registry objects. For example, it does not include properties for describing spatial and temporal coverage of a data collection, or alternative titles for registry objects. Some members of the SWIG reported how they use common linked data properties to include these concepts in their metadata (such as dcterms:spatial, dcterms:temporal, dcterms:alternative).

The SWIG also discussed expanding the ontology to cover concepts beyond those needed for the ANDS registry. For example, Newcastle University wish to model derivation relationships between data sets, such as the relationship between a “cleaned” dataset and a raw dataset. Many institutions also want to describe record keeping requirements relating to research data, such as information on how long to keep data, and how to dispose of it when appropriate.

Future work on the ontology might usefully compare these emerging practices, decide on a common approach, and document their use.

Where to from here?

This post summarises SWIG discussion of emerging practices for representing Australian research data and research metadata using linked data technologies. I hope it also highlights areas for future work: both in extending the scope of what we can describe, but also in agreeing on common ways to describe it.

Possible online forums for sharing emerging practice include this blog, the general ANDS partners mailing list and community bulletin board, and the more detailed ANDS-VITRO google group. Some of the issues, however, such as nutting out a process for maintaining the ANDS-VITRO ontology, probably require face-to-face discussion. Simon Porter from Melbourne University has suggested that the CCA-Educause conference in Sydney in April as a possible venue for another SWIG. Any takers?

Unrelated aside: this post represents my first attempt at using E-Prime to improve the clarity of my writing. I don’t think the experiment totally succeeded, but I certainly learnt a lot from the experience.

Written by Nigel Ward. Copyright The University of Queensland, 2011. Licensed under Creative Commons Attribution-Share Alike 3.0 Australia. <http://creativecommons.org/licenses/by-sa/3.0/au/>.

The project is supported by the Australian National Data Service (ANDS). ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.

Research data vocabularies created in ANDS projects

The Australian ANDS community have developed ontology and vocabulary representations that are already being re-used beyond their original projects:

• the ANDS-VITRO ontology developed by Griffith University, QUT and Melbourne University
• RDF encodings of the ANZSRC Field of Research (for), Socio-Economic Objective (seo) and Type of Activity (toa) vocabularies developed by USQ, Griffith University and QUT

Most of the projects represented at the SWIG are using terms from these vocabularies to describe their research data holdings. Most of the folks present also raised issues about how these vocabularies could be referenced and accessed.

Vocabulary publishing principles

To understand the issues, it is worth taking a step back and thinking about requirements for hosting a Linked Data vocabulary. As Duncan Dickinson pointed out previously on this blog, vocabularies are most easily re-used if they are published according Tim Berners Lee’s Linked Data principles:

1. Use URIs as names for things

2. Use HTTP URIs so that people can look up those names.

3. When someone looks up a URI, provide useful information, using the standards (RDF, SPARQL)

4. Include links to other URIs. so that they can discover more things.

I think it is also useful to consider a further principle from the W3C Cool URIs for the Semantic Web document:

5. Stability. Once you set up a URI to identify a certain resource, it should remain this way as long as possible

It turns out that the ANDS-VITRO and ANZSRC RDF vocabularies stack up well against principle 1, principle 2, and principle 4. In my opinion many of the issues raised at the SWIG pertain to support for principle 3 and principle 5. Let’s go through them one by one at assess the situation …

HTTP URIs and Linking

All terms in the ANZSRC RDF vocabularies and the ANDS-VITRO ontology are assigned HTTP URIs (meeting principles 1 & 2). The definitions in the ANDS-VITRO ontology link extensively to other URIs (meeting principle 4). The ontology re-uses established vocabularies by linking to terms in Dublin Core, FOAF, BIBO, and the base VIVO ontology.

The definitions in the ANZSRC RDF vocabularies have fewer URI linkages, but this makes sense given they are foundational and not built from other vocabularies. While it is technically feasible to make links between vocabulary terms in ANZSRC and equivalents in other subject vocabularies (such as Library of Congress Subject Headings), this would require significant effort, and I’m not aware of this being raised as a requirement. So: ANZSRC meets principle 4 to the extent that it currently makes sense to do so.

Useful information and Stable URIs: ANZSRC

The hosting of the ANZSRC RDF vocabularies supports principle 3 (Useful information). Dereferencing a URI from the ANZSRC FOR RDF vocabulary like http://purl.org/anzsrc/for#group_0501 takes you to information about the definition. Additionally, the information you get back varies depending on how you ask for it (the web server hosting the vocabulary supports content negotiation). Clicking on the link in a web browser will return a HTML page with information on the vocabulary term. Linked data browsers on the other hand can get various RDF encodings of the vocabulary.

The URIs in the ANZSRC RDF vocabularies have also been designed to address principle 5 (Stable URIs). The vocabularies use PURLs that redirect to URIs where the vocabularies are actually hosted. Currently the vocabularies are hosted by ADFI at USQ, but if they were moved in the future, the PURLs could simply be updated to redirect to the new location and folks could still refer to the vocabularies using the same PURLs.

The SWIG did discuss whether ADFI was the best place to host these vocabularies. Given ANZSRC is published by the Australian Bureau of Statistics, wouldn’t it make more sense for them to host a machine readable encoding? The ANDS folks suggested that this issue might be addressed by their “Data Connections” projects (see previous blog post).

The SWIG also discussed whether PURLs were the best URI option for identifying vocabularies because they can get in the way of content negotiation. For example, asking for an RDF+XML serialisation of http://purl.org/anzsrc/for#group_0501 returns a HTTP 302 redirect with an HTML body, rather than RDF:

GET /anzsrc/for#group_0501 HTTP/1.1
Host: purl.org
Accept: application/rdf+xml

returns

HTTP/1.0 302 Moved Temporarily
Location: http://namespace.adfi.usq.edu.au/anzsrc/for/
Content-Type: text/html; charset=iso-8859-1

I suspect the community needs to discuss this issue further. In my experience, it is possible to ignore the returned HTML and still do content negotiation on the redirected URI (http://namespace.adfi.usq.edu.au/anzsrc/for/). Some clients can even do that automatically for you (e.g. curl -L), but your mileage may vary depending on the client you are using. For this reason, some at the meeting thought it might be better to run our own PURL server that could do direct content negotiation. Once again, I think this needs further discussion.

Useful information and Stable URIs: ANDS-VITRO

The designers of the ANDS-VITRO ontology gave it a base URI of http://www.ands.org.au/ontologies/ns/0.1/VITRO-ANDS.owl# with the intent of getting ANDS to host the ontology for others to use. Clicking on the above URI, however, reveals that ANDS have (so far) declined to host the ontology (it returns a 404 Not Found error). ANDS sees the ontology as an output of one of their projects, rather than an official ANDS output. They say that

“for ANDS to host the ANDS Vitro ontology would be to both endorse it and imbue it with a degree of officialness with which we were not comfortable.”

The SWIG resolved to ask ANDS to reconsider its position in light of how many folks are using the ontology as the basis for their ANDS work. For my sins, I’ve been asked to draft a letter to ANDS requesting clarification and reconsideration of their position. I’ll keep this blog informed about progress.

To their credit, ANDS have suggested another possible way forward: they suggested that they could register a more neutral domain name such as vitro.org.au to host the ontology. This solution requires that all of us who are currently using http://www.ands.org.au/ontologies/ns/0.1/VITRO-ANDS.owl# in our metadata to update our URIs to something like http://vitro.org.au/ontologies/ns/0.1/VITRO-ANDS.owl# . This change might still be feasible at this early stage, but a decision needs to be made quickly, before too many uses of http://www.ands.org.au/ontologies/ns/0.1/VITRO-ANDS.owl# are published on the (linked data) web.

In the next installment …

This post covered the major issue discussed at the SWIG: publishing our community vocabularies. The SWIG also discussed other interesting semantic web topics including best practice for using the vocabularies, and publishing data (as opposed to just metadata) using semantic web technologies. I’ll cover these topics in (yet another) blog post.

Written by Nigel Ward. Copyright The University of Queensland, 2011. Licensed under Creative Commons Attribution-Share Alike 3.0 Australia. <http://creativecommons.org/licenses/by-sa/3.0/au/>.

The project is supported by the Australian National Data Service (ANDS). ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.

It turns out there is a lot of semantic web activity in ANDS Queensland projects. Even with just high-level summaries from those present we easily used up our allocated hour of discussion time. This post summarises the projects that were discussed at the meeting. Subsequent posts will cover the major discussion points and potential areas for action.

Updates:

• 2011-02-23: clarified QUT data capture projects, QUT contributions to ANZSRC vocabularies and fixed link to R2RML.
• 2011-03-02: further refined description of QUT data capture project.

A broad definition of “Semantic Web”

The projects represented took a broad view of what “semantic web” meant in their projects. Some are interested in ensuring we all use the same URIs to identify vocabulary terms in our metadata, some in publishing data into the Linked Data Cloud, and some in building systems that uses ontologies to represent their data model.

Interestingly, no-one at the meeting discussed using advanced semantic web technologies like inferencing or querying data via SPARQL endpoints.

ANDS “In-house” Projects

The ANDS folks at the meeting described “Data Connections” projects being pursued within ANDS.

One project involves creating “linked data endpoints” for information on NHMRC and ARC grants. Research Data Australia (RDA) already alows linkages between collection descriptions and descriptions of the activities (usually grants) that funded the work. Most ANDS projects are currently writing their own activity descriptions for activities and uploading them to RDA alongside their collection descriptions. This means that there is potential for two descriptions of the same overarching activity to be uploaded a number of times and described slightly differently each time. The project aims to reduce this duplicated effort by creating definitive URIs to identify NHMRC and ARC grants. This would allow collection records to unambiguously reference the (NHMRC and ARC) activities that funded them and for RDA users to navigate to definitive descriptions of the activities.

ANDS is working with Geoscience Australia to expose the Gazetteer of Australia as Linked Data. The gazetter provides information on the location and spelling of Australian geographical names. It is already searchable online, but the underlying data isn’t exposed in a machine-readable format. The project is investigating exposing the data in a machine readable format and linking it with the international geonames ontology. This would allow RDA collection records to unambiguously identify Australian place names and to link collection descriptions with more information about their geographic location.

ANDS also reported that a interest group has been established to examine requirements and use cases for developing Vocabulary Services for e-Research, involving the ABS, CSIRO, Geoscience Australia, and other bodies; this follows a workshop held at Geoscience Australia in November 2010. A number of folks at the meeting requested more information about the project and expressed a desire to contribute.

Monica Omodei can be contacted for more information about these projects. The January 2011 edition of the ANDS newsletter also has an article on the Gazetter project.

Tropical Data Hub

James Cook University reported that they are keen to implement a semantic layer within their tropical data hub project. They are interested in pursuing a linked data approach to exposing their data and linking it with other data and facilities. They are also keen to create a portal with a semantic layer for intelligent searches. JCU are just embarking on this project and are still shopping for technology that meets their requirements.

University of Southern Queensland / Newcastle University

As has been previously described on this and other blogs, the USQ / Newcastle RedBox metadata store project is keen to play with the linked data world. The project team reported that one of their key barriers has been the lack of definitive URIs for referencing things in the research space like ANZSRC subject terms, parties, activities or geographic places. While the RedBox team is aware that ANDS projects like those described above are addressing this issue, they couldn’t wait and so have developed some interim solutions:

• The Mint is local infrastructure for identify and describing parties at Newcastle. It pulls information from various authorities, allows for data merging / de-duplication and publishes party identifiers as URIs.
• They are storing a local (cut down?) copy of the geonames ontology to avoid load time issues they found with using the complete dataset.
• USQ created URIs for every term in the ABS ANZSRC Field of Research (for), Socio-Economic Objective (seo) and Type of Activity (toa) vocabularies. This allows them to unambiguously reference subject terms from the Linked Data collection descriptions. These URIs currently de-reference to either a human readable HTML page or a machine-interpretable SKOS representation of the vocabulary.

USQ also mentioned their work examining how to link datasets to papers in a package, as part of the Beyond the PDF initiative.

University of Queensland

Like USQ, the University of Queensland metadata store is being designed to play well in the Linked Data world. It will re-use URIs for existing entities (where they exist), provide URIs for the new entities it describes, and will be able to expose its data using an RDF serialisation.

Many of the other nationally funded projects within the UQ eResearch Lab have employed semantic web approaches:

• the Scoping Study for W4 Semantic Tagging Service report investigates the availability of suitable technologies for providing a Who/What/Where/When (W4) semantic tagging service.
• Aus-e-lit has implemented a compound object authoring and publishing service based on the OAI-ORE semantic web model. The service is useful for modelling provenance of literary resources and for online learning objects for teaching and research
• early work on the Health-e-reef project used an ontology-driven approach to integrating data from heterogeneous datasets about the Great Barrier Reef
• the Phenomics Ontology Driven Data Management Project (PODD) is developing semantic web data management solutions to meet the needs of researchers working at the Australian Plant Phenomics Facility (APPF) and the Australian Phenomics Network (APN)

Although they weren’t present at the SWIG, the Queensland Facility for Advanced Bioinformatics team at UQ was present at the ANDS community day and reported on some relevant work. Their team is using ontologies to analyse data in the Australian mirror of European Bioinformatics Institute molecular biology database with the aim populating RDA with collection descriptions of EBI data that relevant to Australian species.

Queensland University of Technology

QUT worked with Griffith University on the Metadata Exchange Hub project that adapted the VIVO open source semantic web application to collect metadata from multiple university sources and publish it to Research Data Australia (see Griffith report below).

QUT are also working on an ANDS data capture project in the bio-diversity domain that will help to create mappings between relational schemas and ontologies. The QUT team reported that they won’t be developing a sophisticated ontology: the end goal is generating ANDS compliant collection descriptions, and the research group hasn’t expressed a need for modelling complex data. The project will be storing the underlying data in a relational database and will develop a relational to RDF transformation using a technology like D2RQ or R2RML.

Griffith University

Griffith University collaborated with QUT on adapting the VIVO semantic web application to publish data to Research Data Australia. The Griffith team reported that VIVO has proven to be a good choice of technology, and will also be used for other in-house applications.

A major concrete output of this work is the ANDS-VITRO ontology: an extension to the VIVO ontology originally developed by the VIVO project. The project team collaborated with Simon Porter from Melbourne University to extend the VITRO ontology with concepts and properties needed to syndicate information to Research Data Australia. Although the ontology URI isn’t currently resolvable online, it can be downloaded directly from Griffith University.

As part of Metadata Exchange Hub project, Griffith and QUT developed OWL representations of the ANZSRC vocabularies, building on the USQ SKOS representations. They deliberately re-used the vocabulary term URIs defined by USQ to indicate that the two vocabulary representations described the same concepts. The group discussed that in the future it might be sensible for the URIs assigned to de-reference to this new OWL+SKOS representation.

In the next installment …

Hopefully this summary gives an impression of the breadth of semantic web activity in ANDS funded projects.

There is also a substantial amount of depth to some of these projects. Many of the SWIG participants commented that the ANDS-VITRO ontology and the SKOS/OWL representations of the ANZSRC vocabularies are fundamental pieces of work that are already being re-used in the community. There was a lot of discussion about the need to host and maintain this work, as well as share emerging best practices. These discussions will be the subject of my next blog post reporting on our first SWIG.

Written by Nigel Ward. Copyright The University of Queensland, 2011. Licensed under Creative Commons Attribution-Share Alike 3.0 Australia. <http://creativecommons.org/licenses/by-sa/3.0/au/>.

The project is supported by the Australian National Data Service (ANDS). ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.

Admin

[Content warning: this post contains technical details about OAI-PMH and the Atom Publication Protocol. Where possible, the technical bits have been contextualised with business requirements and illustrated with pretty pictures.]

An ingest scenario

First the ingest scenario:
At UQ we have a few ANDS funded data capture projects that aim to “create infrastructure … to collect and manage data, and to improve the way metadata about it is managed.” Each of these projects will create data repositories that manage both data and metadata about that data. The vision is that metadata descriptions from these repositories will be shared with the UQ data registry, where they may be augmented with new metadata (e.g. linked to publications and grants information). Finally, the augmented descriptions will be syndicated to the ANDS collection registry. The scenario looks something like this:

What technologies could aid the ingest of data into the UQ registry from the data capture systems? This post examines how OAI-PMH and AtomPub could support this scenario.

Ingest via OAI-PMH

RIF-CS over OAI-PMH seems like an obvious choice for supporting the scenario. It is what ANDS are using to syndicate data from institutional registries into the national Research Data Australia registry. UQ are already implementing an OAI-PMH server in our data collections registry to support syndication with ANDS. In theory, we could also build an OAI-PMH client into our registry and ask our new data capture projects to syndicate their metadata by implementing OAI-PMH servers on top of their repositories. (In fact it is more than a theory: we have prototyped this solution as part of our Health-e-reef EIF project).

While OAI-PMH seems like an attractive option for transferring metadata from data capture repositories to the UQ data registry, it isn’t perfect. Preliminary discussions with UQ data capture projects revealed that OAI-PMH actually imposes a higher implementation barrier than they’d like. The data capture projects are focussed on capturing and managing a community’s data. Syndicating metadata descriptions about that data is important, but needs to be as simple as possible so that they can focus on data management. Building OAI-PMH servers, ensuring they are always up, and ensuring they can deal with the load created by harvesting takes quite a bit of implementation effort. OAI-PMH also only supports XML based data formats. Some of our data capture communities are familiar with, and want to use simpler formats such as CSV and JSON.

For these reasons, RIF-CS (or an alternate XML format) over OAI-PMH might be a step too far for some of our data capture projects.

Publication workflows

Ingest via OAI-PMH also doesn’t support some of the publication workflows we have in mind for our registry. Taking a lead from the institutional repository community, we’d like the data registry to support versioning of metadata records, and to support publication workflows where only selected versions of metadata records are syndicated to ANDS.

A first-cut at a publication workflow for our data repository looks something like this:

This workflow says that items in the repository are either unpublished (only internally accessible), under review (on the road to being published), or published (accessible to the outside world and harvestable by ANDS). Ideally, we’d like the ingest protocol to support all of the transitions between these states (represented as arrows in the diagram). OAI-PMH can deal with create, update (version) and delete. It doesn’t, however, support notions of submit (for review), update based on review, or selective publication.

AtomPub as a possible technology

In an early discussion with one of the data capture projects, they suggested syndicating their metadata to our registry using the Atom format over HTTP. This discussion led us to investigate the related Atom Publishing Protocol (AtomPub to its friends), which looks to be a very nice fit for the workflow show above.

AtomPub is a protocol for publishing and editing Web resources that was originally developed to support posting blog entries from a web authoring tool. Since then it has been adapted to support publication and updates to other types of data. Most notably, it is used in

• GData (Google Data Protocol) that forms the basis for many of the Google APIs
• OData (Open Data Protocol) that Microsoft are using to “provide access to information from a variety of applications, services, and stores”
• SWORD (Simple Web-service Offering Repository Deposit) that was developed by the institutional repository community to ingest complex data objects

The AtomPub protocol is pretty simple: it uses the basic HTTP verbs to create and manage information resources (look away now and skip past the next diagram if you want to avoid the inner workings of HTTP):

• GET is used to retrieve a representation of a known Resource.
• POST is used to create a new, dynamically named, Resource.
• PUT is used to edit a known Resource.
• DELETE is used to remove a known Resource.

AtomPub distinguishes between draft and published resources, and with one minor extension can support the notion of resources under review. Mapping our publication workflow to AtomPub looks something like this:

(The review=yes part is a minor extension to the protocol to support transition to a review state. It was inspired by the work on SWORD v2 to support the full deposit lifecycle.)

Sure, but will it fly?

We think this approach has legs: AtomPub is a great fit for our workflow and information model; and there is growing support for it in the institutional repository community (via SWORD). We have already built a simple AtomPub interface on top of a repository storing metadata about research collections and parties (using the Apache Abdera AtomPub implementation). We can create, get, edit and delete metadata resources using both a HTML/Javascript user interface and via command line tools. Our system currently only supports published resources, but the next steps are to add under review and published states.

As well as fitting our workflow and information model, we think AtomPub can simplify our data capture projects in two main ways. Firstly, AtomPub supports simple non-XML metadata formats out of the box. We have already prototyped ingesting metadata represented as JSON, and may support CSV if the requirement arises. Secondly, AtomPub doesn’t require the data capture repositories to implement OAI-PMH servers. At its simplest, a data capture repository can submit data to our registry via a basic HTML form. We think that is a very low barrier to entry.

We will be continuing to implement our AtomPub interface over the coming months, and will share our experience in case others also want to play. We are keen to get feedback on our approach, particularly from those familiar with institutional repository ingest systems who can tell us what we are missing.

Written by Nigel Ward. Copyright The University of Queensland, 2010. Licensed under Creative Commons Attribution-Share Alike 2.5 Australia. <http://creativecommons.org/licenses/by-sa/2.5/au/>.

The project is supported by the Australian National Data Service (ANDS). ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.