Thursday 15 October 2015

Feedback on NLNZ ‘DigitalNZ Concepts API‘



This blog post is feedback on a recent blog post ‘Introducing the DigitalNZ Concepts API’ http://digitalnz.org/blog/posts/introducing-the-digitalnz-concepts-api by the National Library of New Zealand’s DigitalNZ team. Some of the feedback also rests on conversations I've had with various NLNZ staffers and other interested parties and a great stack of my own prejudices. I've not actually generated an API key and run the thing, since I'm currently on parental leave.
  1. Parts of the Concepts API look very much like authority control, but authority control is not mentioned in the blog post or the docs that I can find. It may be that there are good reasons for this (such as parallel comms in the pipeline for the authority control community) but there are also potentially very worrying reasons. Clarity is needed here when the system goes live.
  2. All the URLs in examples are HTTP, but the ALA’s Freedom to Read Statement requires all practical measures be taken to ensure the confidentiality of the reader’s searching and reading. Thus, if the API is to be used for real-time searching, HTTPS URLs must be an option. 
  3. There is insufficient detail of of the identifiers in use. If I'm building a system to interoperate with the Concepts API, which identifiers should I be keeping at my end to identify things that the DigitalNZ end? The clearer this definition is, the more robust this interoperability is likely to be, there’s a very good reason for the highly structured formats of identifiers such as ISNI and ISBN. If nothing else a regexp would be very useful. Personally I’d recommend browsing around http://id.loc.gov/ a little and rethinking the URL structure too.
  4. There needs to be an insanely clear statement on the exact relationship between DigitalNZ Concepts and those authority control systems mapped into VIAF. Both DigitalNZ Concepts and VIAF are semi-automated authority matching systems and if we’re not carefully they’ll end up polluting each other (as for example, DNB already has with gender data). 
  5. Deep interoperability is going to require large-scale matching of DigitalNZ Concepts with things in a wide variety of GLAM collections and incorporating identifiers into those collections’ metadata. That doesn't appear possible with the current licensing arrangements. Maybe a flat-file dump (csv or json) of all the Concepts under a CC0 license? URLs to rights-obsessed partners could be excluded.
  6. If non-techies are to understand Concepts, http://api.digitalnz.org/concepts/448 is going to have to provide human-comprehensible content without an API key (I’m guessing that this is going to happen when it comes out of beta?)
  7. Mistakes happen (see https://en.wikipedia.org/wiki/Wikipedia:VIAF/errors for recently found errors in VIAF, for example). There needs to be a clear contact point and likely timescale for getting errors fixed. 
Having said all that, it looks great!

2 comments:

Lars G. Svensson said...

Stuart,

I work for the DNB and stumbled over your comment, that "both DigitalNZ Concepts and VIAF are semi-automated authority matching systems and if we’re not carefully they’ll end up polluting each other (as for example, DNB already has with gender data)." We couldn't really figure out what to make of your statement about the gender data. Could you expand a bit on this so that we can get a common understanding of what is broken?

Thanks,

Lars

Lars G. Svensson said...

Hi Stuart,

I have a short question to #4. You write:
[[
There needs to be an insanely clear statement on the exact relationship between DigitalNZ Concepts and those authority control systems mapped into VIAF. Both DigitalNZ Concepts and VIAF are semi-automated authority matching systems and if we’re not carefully they’ll end up polluting each other (as for example, DNB already has with gender data).
]]
Since I work in the DNB and we are interested in supplying the best data possible, can you expand a bit on what you mean the the DNB has polluted VIAF with gender data?

Thanks,

Lars