John Wilbanks, co-author of the Panton Principles and past OKFN Advisory Board Member, just launched a petition to ask the White House to mandate free access to publicly funded research in the US. Here’s what it says:

We believe in the power of the Internet to foster innovation, research, and education. Requiring the published results of taxpayer-funded research to be posted on the Internet in human and machine readable form would provide access to patients and caregivers, students and their teachers, researchers, entrepreneurs, and other taxpayers who paid for the research. Expanding access would speed the research process and increase the return on our investment in scientific research. The highly successful Public Access Policy of the National Institutes of Health proves that this can be done without disrupting the research process, and we urge President Obama to act now to implement open access policies for all federal agencies that fund scientific research.

If more than 25,000 people sign it within the 30 day time frame, then the White House is required to consider the proposal and to give an official response. At the end of the first day there have been over 3,000 signatures.

Anyone can sign the petition – you do not need to be a citizen or resident in the US to support the initiative. If you believe in open access to research, please do consider lending your name, and encouraging friends and colleagues to do the same. You can find the petition here.

by Jonathan Gray at May 21, 2012 07:34 PM

I’m attending the lightning talks on the first day at #TNC2012. Some of the things we are hearing about (I didn’t get them all, twitter and unicorns are distracting):

• Mujina: a way of testing your SAML IdP and SP from Surfnet
• Encryption and Cryptography for Filesender: something I have talked about before from the nice people at AARnet.
• An Italian pilot to test the use of twitter to support astronomy pedagogy (note Andy McGregor and the Elevator people).
• A new approach to provisioning with SAML from Yaco, who also wrote the code for PEER.
• A presentation on Unhosted, another project looking at breaking the cycle of giving data to third party providers. Sounds a lot like UMA – SMART people!
• Peter on the (federated) TERENA Trusted Cloud Drive project that might be of interest to Eduserv cloud people.
• Another shiny R&E federation in the form of Tuakiri (New Zealand). I’ve had to register an entity with Tuakiri and can confirm they are a nice federation to join.
• A useful talk by Scott Rea on managing the risks presented by the recent CA attacks and failure. Definitely worth a look at the Dartmouth College report if you are an IT manager in R&E.
• Chris Phillips on SCIM beating up SPML. I think it is too early to say if any of these standards will be useful in cloud provisioning, but happy to sit tight wait and see at the moment.
• Bjarni talking about PageKite, another project starting to question the way the web is being used and whether it achieves the aims of ‘freedom and privacy’. This chimes with the talk I will be giving tomorrow (plug plug).
• Shooting more moons with a collaboration between JANET and PowerFolder to federate up their product(s)….which is a bit like a private dropbox on acid. So more handwaving towards the cloud folks.

I’ve been trying to find a common theme to cunningly link all of the talks. It’s difficult but I think most of the speakers were saying:

• here’s some stuff.
• we’ve done it for you.
• it works.
• go play!

by nicole at May 21, 2012 04:34 PM

Koha’s reporting tool is one of my favorite features of the ILS. This tutorial only scratches the surface of how you use the reporting tool in Koha, but it should give you a good foundation for using it in your library.

If you have an idea for a video, please just let me know and I’ll add it to my list of things to record.

Related posts:

1. Using the News Tool in Koha 3.2 and 3.4
2. Disabling Holds in Koha
3. Saving Koha Bibs to Refworks

by Nicole at May 21, 2012 03:00 PM

During recent years, the Russian Federation has undertaken a number of developments in its open data legislation strategy. This trend inspired a team of professionals to get together and start a non-profit organization, “INFOKULTURA”.

Understanding that data availability is crucial for an information society and the development of an information culture, we emphasise the establishment and promotion of Open Data concept though a number of activities, for example the Apps4Russia Contest, conferences, seminars, research and expertise provision. It is worth mentioning that not only technical capacities and access to the Internet are needed to promote the idea of informational culture as an important social topic: the lack of administrative burden, technological, legal, time and other constraints on the data availability are essential. These problems can be solved with the help of open licenses, open standards and open data sources.

Warfly – one of last year’s Apps4Russia winners

Here in Russia, there is growing interest among society and State authorities in Open Data expertise, and in responding to it “INFOKULTURA” has developed a set of proposals in support of the development, implementation and promotion of effective tools for efficient and successful information interaction between government and society.

During discussions and team meetings it was agreed that the main goals of the organization should be focused on promoting the following ideas:

• Information Culture – human activity that is associated with self-actualization and the individual initiatives manifested within the information society. Information culture is based on new data production, communication, environment change, integration of humanity into a common information space and the development of digital tools in order to create more varieties of information.
• Open Government – new modes of interaction between the government authorities and other public institutions in which the bodies that compose the state government provide a broad disclosure of its activities, provide feedback mechanisms for citizens and public institutions, and an operational set of tools for citizens and public institutions to influence the decision-making process.
• Civilian Control – the way the society can have influence with government bodies and institutions, develop the public good, and help to solve social problems.
• Plain Language – the idea that the society deserves clear communications from the government authorities – clear and concise language designed to ensure the recipient understands as quickly and completely as possible. Plain language seeks to be easy to read, understand, and use.
• Understandable Government – a new platform to broaden and ease official communications between government and society via Infographic tools, Clear visualized guidance and other comprehensive interaction mechanisms that designed to help people find out new ways to engage with State agencies, and take action to improve the way they work.

Apps4Russia Contest

In April 2012 the second Apps4Russia contest was announced at RIF+CIB 2012, the Internet industry’s main event in Russia.

The Apps4Russia contest was initiated to promote the idea of Open Data. The main goal is to encourage Russian developers to create projects based on Open Government Data, aiming to increase public benefit and improve government transparency.

In 2011 the Apps4Russia was held in the private initiative format with a prize fund of EUR 4,000. This year the prize fund has been significantly increased since it is expected that more developers from various levels, inlcuding school students, will submit to the contest.

In 2011 the following Submissions were nominated and awarded:

• “Fire tracking concept” – public monitoring of the natural resources;
• “DataPult” – interactive infographics data based on downloaded statistics;
• “Warfly: World War II aerial photography” – aerial photographs of Russian and West European cities during the period 1939-1945. All photographs are synchronized with modern landscape.

Datapult – one of last year’s Apps4Russia winners

This year all submissions will be reviewed under the following categories:

• Great — main competition for all developers. (Examples topics: anti-corruption, clear and plain statistics, environmental monitoring, Russia’s role in the global world etc.)
• Junior — school students competition (individual and group entries)
• Visualization — interactive and poster infographics competition based on Open Data resources;
• Cities and urban life environment — web-sites and applications created to improve urban life environment, through citizens and community initiatives

Call for Proposals timeframe: April – September
Summing up date – 12 September 2012 (Day of the Programmer).

Official web-site: http://www.apps4russia.ru

by Ivan Begtin at May 21, 2012 01:45 PM

Earlier this year, we announced plans to launch the School of Data. Thanks to the generous support of Open Society Foundations and the Shuttleworth Foundation, we’re now ready to go! We’re holding a kick-off sprint next week, and we invite you to get involved.

What is the School of Data?

The School of Data is led by the Open Knowledge Foundation (OKFN) and Peer 2 Peer University (P2PU). The School will provide online training for data ‘wrangling’ skills – the ability to find, retrieve, clean, manipulate, analyze, and represent different types of data.

The School of Data is a collaborative and community-orientated project, and we welcome partners and participants. We’ve already had exciting conversations with several organisations and individuals, and we look forward to drawing upon their expertise during the development of the School. We are particularly excited to welcome the Tactical Technology Collective to our sprint next week, and look forward to benefiting from their wide-ranging experience (see e.g. their drawing by numbers project). We hope many more will join us – read on to find out how you can take part!

For more information about the School of Data, please visit our FAQs

The Kick-off Sprint

Next week, a small team of us will be gathering in Berlin for the School of Data kick-off sprint. This is a great opportunity to get to know one another, and to start building materials and resources for the School.

During the sprint we will be holding a designated virtual session, and we warmly invite you to join us online!

Details of the sprint are as follows:

When: Thursday 24th May, 12pm-4pm UTC Where: Online, through our IRC channel (#schoolofdata on freenode)
How: Sign-up on the etherpad and then just drop in!

Everyone is welcome, and we will try to have a range of hands-on activities to suit everyone’s interests. We particularly welcome:

• Suggestions for content and courses
• Recommendations (or donations!) of high quality, openly licensed materials and resources, including interesting datasets to work with
• Offers to build a specific learning challenge or track
• ‘Testers’ willing to try out and improve the content available so far

If you can’t make the IRC session, you can still get involved. Sign-up to the School of Data mailing list and introduce yourself, or drop a quick note to schoolofdata [@] okfn.org.

In Berlin? Join the Social!

If you are located in or around Berlin, join us for our social! This will be held on the evening of Thursday 24th May at St. Oberholz, (Rosenthaler Straße, 72a). This is a great opportunity for anyone in the area with an interest in Open Data to meet casually for a drink and a chat.

To give us an idea of numbers, please sign-up here.

Get Involved in the School of Data

As well as the kick-off sprint, there are many ways to get involved with the School of Data.

Right now, we are looking for volunteers to:

• Contribute ideas and suggestions for the curriculum.
• Help to build individual tracks and learning challenges.
• Help to source (or donate) high-quality, openly-licensed educational resources.

And once the School is up and running, we will also be seeking volunteers to:

• Act as mentors for people who are taking part in virtual School of Data courses.
• Translate materials.
• Make the School of Data more accessible in their local area.

If you’d like to get involved in these or any other ways, please sign-up to the mailing list and introduce yourself!

You can follow us on Twitter: @SchoolofData

You may also be interested in our recent blog post about job vacancies. The Data Wrangler’s role in particular will be closely involved with the School of Data.

Finally, if you are able to contribute towards the ongoing maintenance of the School, please do feel free to donate via our secure PayPal account.

by Laura Newman at May 21, 2012 01:10 PM

by thmmylibrary at May 21, 2012 09:14 AM

In a long-awaited move [1], Google has announced that its search will now be "semantic." They don't actually mean "semantic" in the sense of the semantic web, although there are similarities. While what Google is doing may not formally follow the W3C standards for the semantic web, there is no doubt that they are performing acts of "data linking" that make use of the concepts of linked data. The W3C standards for linked data are designed for openness, so that data from disparate communities can come together. Google has no obligation to play well with others and, as we saw with the development of schema.org, is in a position to make its own rules, many of which are known only within the giant Google-verse. They call their technology a "knowledge graph" and talk about "things not strings." I've used this same phrase myself in numerous presentations on linked data.

Google has always been about using links between things on the web to determine its brand of "relevance" of a web resource to a search query. By using existing linked data, via large stores of links like Dbpedia, Wikipedia, Freebase, and presumably others, Google can now expand its offerings from a single list of results to additional information about the topic that might be the intended topic of the searcher. I say "might be" without any irony; whether in a web search engine or a library catalog, the communication between the searcher's mind and the device that provides results is always only approximate. What the additional data provides is not only more context but a more ample explanation of the topics that have been retrieved. No longer do users have to guess from snippets the meaning of the results in the result set, but they can see a Wikipedia-like entry that not only gives them more information, but it contains links to other sources of information of the topic.

Snippet

"Knowledge Graph" result

"Knowledge graph" detail

Imagine that you have an 18-year-old user who finds a novel on your library's shelf by Oliver North. The user looks up the author in  your catalog and sees that this person has written a few other books, but oddly always with a "co-author." Is someone so inept worth reading? Now imagine that your catalog also presents the user with the context: Ollie North, Iran Contra, and related persons. Suddenly the user sees where North fits into US history, has a chance to find out what an interesting character he is, and the books take on a whole new meaning.

by Karen Coyle (noreply@blogger.com) at May 21, 2012 09:03 AM

The technique described in Using the sub-property ladder works well to “dumb-up” raw, level 0 data from MARC21 fixed-length data fields to interoperate with metadata from other schemas. Unfortunately, it cannot be used with most MARC21 variable data fields (tags) and subfields. We cannot simply dumb-up a subfield to the level of its parent tag because most tags have more than one subfield; the meaning of a tag is a combination of the meanings of its subfields and tag-level data  is a composite of subfield-level data.

There is another technique we can use to bridge the semantic gap between a subfield and its tag: tags generally can be treated as “aggregated statements”, where the value of a tag is a literal string, or statement, which is composed of the values of subfields.

For example, a record may contain a tag 260 (Publication, Distribution, etc.) with subfield a (Place of publication, distribution, etc.) = “Edinburgh :”, subfield b (Name of publisher, distributor, etc.) = “Castle Press,”, and subfield c (Date of publication, distribution, etc.) = “2012.”. The contents of the tag, “$aEdinburgh :$bCastle Press,$c2012.” can be turned into a tag-level value, “Edinburgh : Castle Press, 2012.”, by substituting a space for each subfield indicator ($) and code pair. We can then use a tag-level property with the label “Publication, Distribution, etc. (Imprint)” and URI “m21plus:T260″ to publish the metadata statement “This resource – has Publication, Distribution, etc. – ‘Edinburgh : Castle Press, 2012.’” as an RDF triple.

The instructions for deriving the tag-level value or aggregated statement from the subfield values are known as a syntax encoding scheme (SES). This is part of the Dubin Core abstract model, allowing specific SESs to be used in an application profile. There can be many different ways of deriving the value; the example above works because MARC21 subfields contain embedded punctuation that delineates the component parts when the subfield encoding is removed. This simple SES allows a MARC21 record to conform to the syntax prescribed by the International Standard Bibliographic Description (ISBD) for compound statements. Unfortunately, this makes it difficult to apply any other SES to the subfields without first removing the punctuation.

It would be much better if the instructions for adding ISBD punctuation to MARC21 data were embedded in an SES. Then a different SES could produce “Published in 2012 by Castle Press in Edinburgh” rather than “Published in 2012. by Castle Press, in Edinburgh :”. This is the approach taken by ISBD itself, and there is clearly an opportunity here for collaboration between the MARC21 and ISBD communities. The same approach is envisaged for RDA.

The aggregated statement technique is also very useful when a MARC tag is repeated. Using tag 260 again as an example, a record may contain multiple publication statements for intervening publishers, where the tag’s first indicator has value “2″. If there are two such tags, then there may be two or more publication places and two or more publisher names, for example “$32001-2005$aEdinburgh :$bMudhut Publishing” and ”$32006-$aEdinburgh :$bCastle Press” (subfield 3 is for Materials specified). A linked data representation of the record needs to keep the places, names, and dates correctly associated so that they don’t get mixed up, for example “Mudhut Publishing” with “2006-”. The tag-level RDF property (m21plus:T260) can be used with an aggregated statement to keep the level 0 data associated with the correct repeat of the tag, avoiding the use of blank nodes in the RDF graph of a specific record.

RDF graph of MARC21 Publication statement data

As the graph shows, the two Publication statements must have URIs so that they can link to the correct subfield values. The URIs identify the literal strings of the aggregated statements, and are instances of an SES; all SESs are sub-classes of the class of literal strings. A blank node, on the other hand, has no URI and uses a local identifier to make the links; such links appear broken in a non-local environment.

To sum up, it seems useful to represent MARC21 tags as RDF properties associated with a syntax encoding scheme. We intend to add these properties to the Open Metadata Registry. Specific encoding schemes can then be assigned using an application profile. There must be many examples of instructions for processing tag subfields for output and display which can form the basis of suitable encoding schemes.

by Gordon Dunsire at May 20, 2012 10:08 PM

by dan@coffeecode.net (Dan Scott) at May 20, 2012 05:57 PM

Seit Google vor einigen Tagen den “Knowledge Graph” vorgestellt hat, rumort es in der Semantic Web Community. Klaut Google doch einfach Ideen und Techniken die seit Jahren unter der Bezeichnung “Linked Data” und “Semantic Web” entwickelt wurden, und verkauft das ganze unter anderem Namen neu! Ich finde sowohl die Aufregung als auch die gedankenlose Verwendung von Worten wie “Knowledge” und “Semantic” auf beiden Seiten albern.

Hirngespinste von denkenden Maschinen, die “Fakten” präsentieren, als seien es objektive Urteile ohne soziale Herkunft und Kontext, sind nun eben Mainstream geworden. Dabei sind und bleiben es auch mit künstlicher Intelligenz immer Menschen, die darüber bestimmen, was Computer verknüpfen und präsentieren. Wie Frank Rieger in der FAZ gerade schrieb:

Es sind „unsere Maschinen“, nicht „die Maschinen“. Sie haben [...] kein Bewusstsein, keinen Willen, keine Absichten. Sie werden konstruiert, gebaut und eingesetzt von Menschen, die damit Absichten und Ziele verfolgen – dem Zeitgeist folgend, meist die Maximierung von Profit und Machtpositionen.

In abgeschwächter Form tritt der Irrglaube von wissenden Computern in der Fokussierung auf “Information” auf, während in den meisten Fällen stattdessen Daten verarbeitet werden. Statt eines “Knowledge Graph” hätte ich deshalb lieber einen “Document Graph”, in dem sich Herkunft und Veränderungen von Aussagen zurückverfolgen lassen. Ted Nelson, der Erfinder des Hypertext hat dafür die Bezeichnung “Docuverse” geschaffen. Wie er in seiner Korrektur von Tim Berners-Lee schreibt: “not ‘all the world’s information’, but all the world’s documents.” Diese Transparenz liegt jedoch nicht im Interesse von Google; der Semantic-Web-Community ist sie die Behandlung von Aussagen über Aussagen schlicht zu aufwendig.

Laut lachen musste ich deshalb, als Google ein weiteres Blogposting zur Publikation von gewichteten Wortlisten mit einem Zitat aus Goethes Faust beginnen lässt:

Yet in each word some concept there must be…

Im “Docuverse” wäre dieses Zitat durch Transklusion so eingebettet, dass sich sich der Weg zum Original zurückverfolgen ließe. Hier der Kontext des Zitat von Wikisource:

Mephistopheles: [...] Im Ganzen – haltet euch an Worte! Dann geht ihr durch die sichre Pforte Zum Tempel der Gewißheit ein.

Schüler: Doch ein Begriff muß bey dem Worte seyn.

Mephistopheles: Schon gut! Nur muß man sich nicht allzu ängstlich quälen; Denn eben wo Begriffe fehlen, Da stellt ein Wort zur rechten Zeit sich ein. Mit Worten läßt sich trefflich streiten, Mit Worten ein System bereiten, An Worte läßt sich trefflich glauben, Von einem Wort läßt sich kein Jota rauben.

Die Antwort von Google (und nicht nur Google) auf den zitierten Einwand des Schülers gleicht nämlich bei näherer Betrachtung der Antwort des Teufels, wobei das “System” das uns hier “bereitet” wird ein algorithmisches ist, das nicht auf Begriffen sondern auf Wortlisten und anderen statistischen Verfahren beruht.

In der Zeitschrift für kritische Theorie führt Marcus Hawel zu eben diesem Zitat Goethes (bzw. Googles) aus, dass Begriffe unkritisch bleiben, solange sie nur positivistisch, ohne Berücksichtigung des “Seinsollen des Dings”, das bestehende “verdoppeln” (vgl. Adorno). Wenn Google, dem Semantic Web oder irgend einem anderen Computersystem jedoch normative Macht zugebilligt wird, hört der Spaß auf (und das nicht nur aufgrund der Paradoxien deontischer Logik). Mir scheint, es mangelt in der semantischen Knowledge-Welt an Sprachkritik, Semiotik und kritischer Theorie.

by jakob at May 20, 2012 01:49 PM

by ranti at May 20, 2012 05:17 AM

by ksattler at May 20, 2012 01:31 AM

by heycarey at May 19, 2012 08:49 PM

This is a brief observation about the central role that error costs must play in any discussion of orphan works policy. I made it at the Berkeley orphan works conference last month. By popular request (okay, by one person’s request), I’m putting it online here.

The first, and most obvious, errors are those made by copyright owners whose works become orphaned. Works don’t end up orphaned unless there’s been a mistake by the copyright owner. They make mistakes about whether they’re copyright owners, about whether they’re findable, and about whether there’s a potential audience interested in their works.

But these errors interact with errors made by potential users of the works. If users knew with certainty whether copyright owners would emerge and object to possible uses, there’d be no orphan works problem, because every search would lead either to genuine negotiations or to use without fear of suit. False negatives that expose users to the risk of being sued and copyright owners to mistaken uses; false positives chill use without benefitting copyright owners.

And finally, there are error costs in the judicial system, which magnify the effects of errors at the previous two stages. They award remedies more than sufficient to compensate copyright owners, or they fail to award sufficient remedies. And the same problems face any system for dealing with orphan works: it could mistakenly declare that works are orphan when they’re not, or vice versa, and that searches were diligent when they weren’t, or vice-versa.

The point is that the ubiquity of errors isn’t just an incidental feature of the orphan works debate: it’s the defining reality that causes there to be an orphan works problem at all, and with which any response to the problem must grapple.

by James Grimmelmann (james@grimmelmann.net) at May 19, 2012 03:46 PM

Increasingly, I’m using Google Docs with collaborators. Yesterday, one of them pointed out the new “Research” search tab within Google Docs. (Tools->Research). I’m a bit surprised that your searches don’t show up on your collaborators’ screen. I’m particularly surprised that sharing searches doesn’t seem possible.

Google Docs' new 'Research' tab promotes search within Google Docs.

Apparently, it is pretty new. More at the Google Docs blog.

by jodi at May 19, 2012 01:38 PM

This week I taught a workshop in FL for PLAN on WordPress. I was asked what Plugins I use/recommend so I thought I should share a list. I went through all of my sites and grabbed the link from the plugins page that said ‘Visit Plugin Site,’ but I recommend searching the WordPress Plugins database for these and installing them from there.

Depending on the type of site I use all of the following on my WordPress sites:

• Akismet: http://akismet.com
• FeedBurner FeedSmith: http://wordpress.org/extend/plugins/feedburner-setting/
• Google XML SiteMap: http://www.arnebrachhold.de/redir/sitemap-home/
• SexyBookmarks by Sharaholic: http://www.shareaholic.com/tools/wordpress/
• TrollGuard: http://www.trollguard.com
• TweetMeme Retweet: http://tweetmeme.com/about/plugins
• Ultimate Google Analytics: http://www.oratransplant.nl/uga
• Widget Context: http://konstruktors.com/projects/wordpress-plugins/widget-context/
• WordPress Database Backup: http://austinmatzko.com/wordpress-plugins/wp-db-backup/
• WordPress Editorial Calendar: http://stresslimitdesign.com/editorial-calendar-plugin
• Yet Another Related Posts Plugin: http://yarpp.org
• All in One SEO: http://semperfiwebdesign.com
• Capability Manager: http://alkivia.org/wordpress/capsman
• Category Posts Widget: http://jameslao.com/2011/03/24/category-posts-widget-3-2/
• Contact Form 7: http://contactform7.com
• Google Analytics Dashboard: http://www.ioncannon.net/projects/google-analytics-dashboard-wordpress-widget/
• NextGEN Gallery: http://alexrabe.de/?page_id=80
• Printer Friendly and PDF: http://www.printfriendly.com
• RSS in Page: http://www.titusbicknell.com/wordpress/rss-in-page
• Super Search: http://www.canhaswebsite.com/super-search
• Ultimate Google Analytics: http://www.oratransplant.nl/uga
• WordPress Dashboard Twitter: http://wpdashboardtwitter.com
• Google Maps for WordPress: http://xavisys.com/google-maps-for-wordpress/
• Job Manager: http://pento.net/projects/wordpress-job-manager-plugin/
• Unfiltered MU: http://wordpress.org/extend/plugins/unfiltered-mu/
• WordPress Importer: http://wordpress.org/extend/plugins/wordpress-importer/

Related posts:

1. Speeding up WordPress Dashboard
2. WordPress Automatic Upgrade
3. Menus in WordPress 2.7

by Nicole at May 19, 2012 12:32 PM

In academic libraries, there are usually so many levels of priorities. There are the priorities of the university. There are the priorities of the library. Each unit probably has its own priorities, as does each individual. Ideally, these all sync up nicely, where an individual can show how their priorities mesh with library’s and university’s priorities. However, it’s not always easy for the library to support all of those university priorities. That’s often because the library doesn’t have the people-power or financial resources to do everything well. So the library has to choose whether they follow every university priority in a superficial way, or whether they focus on the priorities that they can accomplish well in light of limited resources. Neither is a completely satisfying choice.

At my library, and really at the University a a whole, there is definitely a tug-of-war going on between the original access mission of the University and the growing importance of research. Clearly both are important and both require library support. My colleagues are deeply committed to both roles, but it’s frustrating when you know you can’t do it all as well and completely as you’d like. You can’t develop a vibrant scholarly communications and data management program AND have a comprehensive program of outreach and instruction to the neediest students when the same people are involved in both. And yes, we’re doing all of those things, but not to the extent that we’d like to. Having been at a small place before, we certainly dealt with those limitations too (we still don’t have an institutional repository at Norwich), but the expectations of the academic community were lower because we weren’t a large research institution. And in light of budget cuts, I’m sure many, many academic libraries are feeling similarly frustrated by what they can’t do (or do enough of).

And this tug-of-war is seen in the instruction program as well. We can’t do all of the teaching we’d like given our staffing, so we have to prioritize. But how? With the growing research priority, do we focus more on faculty outreach and graduate-level instruction? With the focus on Freshman retention, do we put more time and effort in teaching first-year students? We have a strong liaison program and a ton of teaching goes on in upper-level undergraduate classes, especially those that are core to majors (like research methods). This is fantastic! I remember when I got to Norwich, very little library instruction was going on outside of the lower-division classes and we worked hard over the years to get information literacy instruction integrated into core courses in the majors. PSU has been there for a long time. Is that less important than reaching Freshman or more? Or is there, as I suspect, no one right answer to that question?

So how do we set priorities? How do we determine how much focus to put on each thing we do? A colleague recently showed me stats on what percentage of the total enrollment is each class (Freshman, Sophomore, Junior, etc.). Do we use that to determine our instructional priorities? Do we say “sophomores make up x% of student enrollment, so we will provide x% of our teaching in 200-level classes?” It’s certainly a concrete way of making decisions, and probably as good as any, but I don’t feel like needs and priorities translate so easily to exact numbers and percentages. We still need to take into account Univerity priorities, student needs, what classes are the most valuable to be involved with, and in what classes can we make the greatest difference. If someone comes up with a formula for figuring this out, they deserve some kind of award.

Another thing we talk a great deal about is using learning objects to augment and/or replace the one-shot. And I’ve started to wonder where is the best place in the curriculum to implement this? Should we replace Freshman-level instruction with online learning modules because most students are not really at an emotional/intellectual space yet where they are capable of serious research or do we focus on face-to-face instruction because they need the high-touch approach? Do we employ learning objects in upper-division classes because the students are more self-motivated once they’re in their majors, or is that the critical time to connect with them because the sort of research they’re doing is higher-level? Do we stop teaching grad students face-to-face because of their much higher motivation level, or is that the perfect reason to focus on them? I don’t know if there have been studies on this, but it would be interesting to figure out at which level does it make the most sense to provide face-to-face instruction and at what level would students benefit most from learning objects. It seems like most suites of learning objects designed to replace face-to-face instruction happen at the Freshman level, but that might just be because there are so many sections of the same few courses and it’s easier to create something that works for many, many, many classes.

None of these issues is unique to my University; in fact, I’d argue that in a world of rising materials costs and shrinking budgets, they’re pretty darn universal. Even at little old Norwich, where the student/librarian ratio was so much smaller, we had to prioritize. It got to a point where I had to start cutting down on the number of history classes I was teaching, because it was taking up such a disproportionate amount of my time (although I really enjoyed it!). So, at your institutions, how have you determined what to prioritize in terms of library instruction? When demand for your services exceeds supply, what do you stop doing? Where have you replaced face-to-face instruction with other lower-touch models and why?

by Meredith Farkas at May 19, 2012 04:56 AM

Click here to view the embedded video.

Russel Houghten‘s Open Horizon is part skate film, part time lapse, and mostly awesome.

Then somebody pointed to this Jimmy Plmer/Z-Flex video that shares a number of features with Houghten’s work, but is less ambitious in scope. At least they did a behind the scenes video that shows the sweet Red camera and rails.

by Casey Bisson at May 19, 2012 03:54 AM

Those following me in my various social circles are probably already sick of hearing about this, but unglue.it launched yesterday.  Unglue.it, started by Eric Hellman (aka @GlueJar) and other folks w/ connections to Code4Lib, is a effort to release copyrighted books to the world.  Working with right-holders in a kickstarter style to raise enough money to license an ebook  a creative commons non-commercial license.  It's a way of "front-loading" profits so the author can be compensated for their work, but the world gets access.

They also have a mechanism for adding books to a "wishlist" that will give them an indication of works that people want and what right-holders they should track down.

This is a brilliant way to deal with some of those really important and hard to find out of print books.  For example, I've wishlisted a very good biography by Greg Rickmann of Philip K. Dick called To the High Tower.  It's a work that I stumbled across that's sought out by a small circle of Philip K Dick fans. 

It's not clear if there's enough demand for another printing, but unglue.it offers a chance that it could be made easily available again while also giving the author further profit he's not going to see of this long-since sold out book otherwise.  I have had the good fortune to read it due to the fact I work at a major library that has a ton of access of books, but I know many a sci-fi fan that doesn't have the resources I do.

I also must admit that I'm interested to see if this model works.  I've thought about trying to do a small hobby side business of making value-added public domain works and perhaps doing something similar to unglue.it with near-orphaned copyright works.  However, tracking down right-holders has proved troublesome enough that it's remained in my large bucket o' ideas I'd like to do someday.  I'm hoping unglue.it takes off enough that it'll create a infrastructure that might make it easier to do projects of this nature.

Here's what I'm pledging to:

by Jon Gorman at May 19, 2012 12:19 AM

Yesterday, Judge Baer held an hour-long hearing in the HathiTrust case. Although most of the time was spent on procedural matters, the Authors Guild’s lead attorney, Edward Rosenthal did a very effective job leveraging them into substantive points.

The first problem for the court was a discovery dispute. Some of the plaintiffs live far from New York and have objected to having their depositions taken there, and a fourth, J.R. Salamanca, is in ill health and bedridden. After some discussion not worth recounting, the defendants’ attorneys agreed to take the deposition of Salamanca’s literary agent instead, and the two sides agreed on logistical arrangements for the others.

The most significant consequence of the deposition skirmish is that the close of discovery has been effectively pushed back. It had been scheduled to be finished by May 20, which is self-evidently impractical now that some depositions won’t even happen until next week. Instead, it now appears that discovery will last until June 8. This fact puts pressure on the schedule for summary judgment. Judge Baer had asked for the motions for summary judgment to be fully briefed by July 20. But allowing the necessary time for each side to respond to the other’s papers means that the actual motions would need to be filed in mid-June, i.e. uncomfortably close to the end of discovery. Judge Baer at one point asked parties if they could finish their briefing by the start of July so he could “put it under his pillow” when he goes away for the month. They agreed to go off and discuss the schedule, but I’d be quite surprised if the summary judgement deadlines were moved up.

And this scheduling tempest will spill over beyond its teapot: it seems likely to shape how the case will be argued. Joseph Petersen, appearing for the HathiTrust, tried to suggest that a quick ruling from Judge Baer on the motions for partial judgment on the pleadings (HathiTrust’s on associational standing and the Authors Guild’s on the applicability of copyright defenses) would help winnow the issues in the case, making for more narrowly focused summary judgment motions. Judge Baer wasn’t buying. He said, gruffly, that he was inclined to hold over these issues and decide them together with the summary judgment motions. This isn’t good news for HathiTrust, for reasons shortly to become apparent.

The first phase of the substantive oral argument dealt with HathiTrust’s motion to have the Authors Guild and other associations removed from the case for lack of standing (leaving only the individual plaintiffs). W. Andrew Pequignot delivered the argument in a style familiar to anyone who’s watched a moot court. He give a clear, but completely wooden, summary of HathiTrust’s argument against the associations, focusing on the argument that each copyright plaintiff must prove individual ownership of the works on which they sue, so that an association would need to present individual facts for every one of its members. The judge tried to ask him what practical difference associational versus individual standing would make if HathiTrust ended up losing on the merits, a question which raises subtle questions about the scope of a possible injunction, but Pequignot didn’t engage with the question.

Ed Rosenthal then gave the Authors’ Guild reply, and showed why he’s the chair of his firm’s IP and litigation groups. The defendants copied ten million books, he said, in an act of “preemptive mass digitization,” and now they want to look at individual books in evaluating standing. The response to Rosenthal’s point, if there is one, is that the Copyright Act really does require proof of individual ownership, a requirement that has nothing to do with whether the infringer is accused of copying one book or a million. Rosenthal could have replied by saying that this would leave copyright owners without a way to challenge mass infringement, and the defendants’ natural surreply would have been that individual lawsuits would be more appropriate. But that last point was precisely the question Pequignot ducked—thereby not only ceding much of the standing issue but also the rhetorically intuitive high ground.

That mattered, because Rosehtnal used the standing issue as a pivot to his argument that HathiTrust’s copying was substantively impermissible under the Copyright Act. Having set up the issue as a mass challenge to mass digitization, he was ready to roll with his argument that Section 108 provides the only relevant permission for copying here, permission that HathiTrust has far exceeded in copying books wholesale rather than retail. Thus, he claimed, the associations were the perfect plaintiffs to mount a program-wide challenge.

HathiTrust’s next moot court argument came from Allison Roach, who argued that no one had standing to challenge the Orphan Works Program since no identifiable books with copyrights owned by any of the plaintiffs had been made available or were in imminent likelihood of being made available through the OWP. Judge Baer was skeptical, saying that he was bothered that the libraries did “all of this” before there was an opportunity for plaintiffs to complain. Roach said was that no books had been made available, only a list of candidates, and that the plaintiffs were asking for an injunction against the entire Orphan Works Project without concrete facts about specific books it would infringe.

Rosenthal’s response here was a little less vivid. He emphasized that the University of Michigan had set up a mechanism for its orphan works. Some plaintiffs found their books on the list; the University suspended the program. If, he argued, this meant there was no right to object because there was currently no program, then there would never be a circumstance in which the program’s legality could be addressed. Any copyright owner who tried to object would be defined out of the class of copyright owners with standing to object, and this couldn’t be. (His point illustrates why the standing argument may be too clever by half when it comes to the Orphan Works Program, and why suspending the program might end up being ineffective in insulating it from judicial review.)

This brought the court to the plaintiffs’ motion for judgment on the pleadings that the libraries couldn’t raise fair use, Section 108, or other Copyright Act defenses. Here, Rosenthal led off by arguing that Congress passed a specific statute with directions for libraries, which the defendants disregarded. He then acted annoyed that the defendants, in their responses (see the bottom of our page on the case) characterized this as a broader attempt to stop libraries from claiming fair use, ever. No, Rosenthal said, the plaintiffs don’t argue against other library uses, just that they can’t digitize every book. They chose to scan in a large project, and the burden should be on them to justify that project. Once again, it was an oral advocacy gem.

Joseph Petersen then gave a rebuttal that ran through HathiTrust’s brief. The plaintiffs, he said, tried to argue that libraries have no fair use rights, but only the specific rights granted in Section 108. When shown how absurd it would be to claim that libraries alone in society have no fair use rights, the plaintiffs changed course and argued that the case isn’t about library copying in general, but only about this program. And this, he said, showed why this issue wasn’t appropriate for the “rule 12” context (i.e. a motion for judgment on the pleadings): it obviously depends on specific facts about the libraries and what they’re doing. He then recounted, quickly, some of the libraries’ arguments about the symbiosis between Section 108 and fair use, about the noncommerciality of the project, and about the text of Section 108.

He was followed by Daniel Goldstein, on behalf of the National Federation of the Blind. He ran through some of the history of accessibility of books to the blind, and emphasized that digitizing books brings the number of accessible titles from tends of thousands to tens of millions. Now, blind and visually disabled students can access HathiTrust’s digital database (when they provide appropriate certification of their disability). They’re the only group that has access to the database, but now they have equal access to the books themselves as sighted students would. He used this to argue that the plaintiffs’ assertions about categorical exclusions from fair use and other copyright defenses would tell libraries that they can’t make the copies for the print-disabled that they need to to comply with the Americans with Disabilities Act and the Rehabilitation Act.

All in all, yesterday’s skirmish was a minor one in the arc of the case. The discovery disputes were sorted out, and the schedule will be. Because Judge Baer strongly signaled that he’ll put the immediate motions off until he considers the summary judgment motions, that just puts the interesting and important issues off until the even more interesting and important summary judgment ruling.

Still, the skirmish was a clear win for the Authors Guild and its co-plaintiffs. Rosenthal made common-sense arguments about standing that—from the audience at least—seemed like they were persuasive to Judge Baer. He leveraged his responses to the defendants’ motions on standing to bolster his own argument on the applicability of fair use. And because Judge Baer is likely to hold the present motions over, he put the defendants in the difficult position of arguing that they are entitled to a blanket fair use defense at the same time as they argue that fair use is a fact-specific inquiry requiring individual participation.

The defendants’ decision to press the standing issues, at least in the way they did, now appears like a mistake. Both at the hearing and in the case overall, the plaintiffs have been able to use their responses to the standing motion to wrong-foot HathiTrust and take control of the case’s timing. As readers of this blog know, I don’t think much of the plaintiffs’ own judgment on the pleadings motion, but I have to give them and their lawyers credit for using it at the hearing to define the narrative of the case on their terms. They chose their counsel well.

The other matter on display yesterday is how different Judge Baer is from Judge Chin. Where Chin’s attitude is generally thoughtful and gentle, Baer tends more towards the gruff and the impatient. (It may not have helped that the hearing was sandwiched between three criminal matters and an afternoon of conferences, and that Judge Baer’s, schedule as he announced, had no room for lunch.) His fast-track schedule for the case is an indication of where Baer’s priorities lie, and my sense is that he saw the hearing more as a way to keep the case moving properly than as an occasion for deep reflection on the issues.

Assuming no curveballs, the next major dates in these cases will be in mid-June, when a variety of major motions will fall due. Motions for summary judgement will be due June 14 in Authors Guild, the visual artists’ motion for class certification will be due June 13, and summary judgment motions in the HathiTrust case will arrive somewhere around then, too, depending on what the parties agree to.

by James Grimmelmann (james@grimmelmann.net) at May 18, 2012 08:57 PM

The planned install this weekend has a number of enhancements designed to make your life better, as a developer. Of course, I say this with a caveat because it still feels like step 2 of 132 that we have envisioned for having all the services look, act and smell the way we want them to. But, “walk before you run” is what we keep telling ourselves.

What you need to know:

Principle to Principal

Small but important change for all WorldShare (WMS) APIs except NCIP:

read more

by alicesneary at May 18, 2012 07:23 PM

Courtesy of Seth Shaw and Ben Goldman, who pointed this out to me, I’d like to take note of an email program that has very specialized, but potentially important, use: the YPOPs program.  This is a Windows application which can be used to establish an IMAP or POP3 Connection to Yahoo email account, from which you would be able to download email for preservation purposes.

The project seems to be dormant, but there is a project website at http://ypopsemail.com/   and a sourceforge site at http://sourceforge.net/projects/yahoopops/.

by Chris Prom at May 18, 2012 06:44 PM

by AndyP at May 18, 2012 03:44 PM

You could say “this is a service to watch” but that would be missing the point. Yesterday the ‘Unglue.It‘ service launched as a way to crowdsource the funding of a fee to authors to release their own works under a Creative Commons license.

This is new model that doesn’t have an equivalent in the analog, print world or the existing digital, ebook world. In the analog world, the right of first sale reigns supreme; as a purchaser of the book, you have the right to do just about anything you want with that copy: you can give it to a friend, you can sell it at a garage sale, you can donate it to a library. Since it only exists as a physical object, once you give it up you no longer have access to it.

In the digital world, exact replicas are as easy as clicking and dragging a file to a flash drive, posting it to a website, or e-mailing it to friends. And you still retain the original copy. The concern for this sort of proliferation of copies is why many (most?) publishers insist on using various forms of DRM. DRM is a royal pain in the butt, though, (and publishers know it) because most forms don’t allow you to convert a file to be read across various readers (e.g. from Kindle to Nook). And you don’t really “own” the file; the DRM software “phones home” resulting in cases where files are removed from devices and files that are permanently locked because “home” has shut down.

Enter the Unglue.It model. Rights holders can set a price that, when reached, they promise to release a digital version of the work that is unencumbered by DRM under a Creative Commons License. (The
— CC BY-NC-ND 3.0">
— CC BY-NC-ND 3.0">
— CC BY-NC-ND 3.0">worldwide non-commercial, no-derivatives license is the default.) Readers that want to see that happen pledge an amount that they choose towards the campaign. If the accumulated pledges reach the price by the campaign deadline, readers’ credit cards are charged, the author receives their money (minus a commission fee to the Unglue.It service), and posts the ebook somewhere for anyone to download and read. Rights holders can also offer incentive premiums, such as signed posters from cover illustrators, virtual author visits, and opportunities to interact with the author as they create a new work. If the amount isn’t reached by the deadline, the campaign fails — no one’s credit card is charged and the book isn’t released.

The economics of this are fascinating. Publishers will let titles go out-of-print when they think the market for new copies reaches a point where maintaining the inventory is no longer cost effective. In some cases, print-on-demand from devices like the Espresso Book Machine have filled that void for single copies, but no one is really sure what the value of a work is worth as time marches on. The Unglue.It model enables us to see what the value for some of these works really are after their initial publication by creating a market for wholesale, after-first-run, unlimited copy licenses. It will be interesting to see how these first few campaigns go and how the market settles out.

The Unglue.it funding process serves two very important ends – it helps writers and content providers like myself get paid for the work we produce while at the same time making that work freely available to readers all over the world. No one else, anywhere, is doing anything like this and I couldn’t be more excited about being a part of this ground-breaking launch.

So, rather than to be “something to watch”, go ahead and register at Unglue.It, participate in one of the campaigns or add books to your wish list to encourage other rights holders to join, and see where this new model takes us.

by Peter Murray at May 18, 2012 03:40 PM

The renowned Dr. Pangloss greatly enjoyed last night's inaugural dinner of the Storage Valley Supper Club, networking with storage industry luminaries, discussing the storage technology roadmap, projecting the storage market, and appreciating the venue. He kindly agreed to share his notes, which I have taken the liberty of elaborating slightly.

• The following argument was made. The average value to be obtained from a byte you don't keep is guaranteed to be zero. The average cost of not keeping the byte is guaranteed to be zero. Thus the net average value added by not keeping a byte is guaranteed to be zero. But the average value to be obtained from a byte you keep is guaranteed to be greater than zero. The average cost of keeping the byte is guaranteed to be zero. Thus the net average value added by keeping a byte is guaranteed to be greater than zero. So we should keep everything. Happy days for the industry!
• The following numbers were quoted. The number of bytes to be stored is growing at 60%/yr. The cost of storing a byte is growing at -20%. Thus the total cost of storage is growing at 60-20=40 (100+60)*(100-20)=128%/yr. And IT budgets are growing 4% a year. Happy days for the industry!

Tip of the hat to Jim Handy for correcting my math.

by David. (noreply@blogger.com) at May 18, 2012 10:25 AM

The University of Illinois Archives is current searching for a full-time Archival Operations and Reference Specialist.  The position will report directly to me and will have responsibilities for overseeing the American Library Association Archives and for providing reference services for University Archives.  Although this is initially a two-year appointment, there is a possibility that it could be extended or made into a continuing appointment, depending on funding.  It offers the ability to work with a wide range of archival functions, from pre-custodial work through access.

A copy of the position description is available at https://jobs.illinois.edu/default.cfm?page=job&jobID=18473.

If you are interested in learning more about this job, please contact me via email.

by Chris Prom at May 18, 2012 02:02 AM

Unglue.it launched at noon yesterday. It was a good day, which means nothing awful happened. That's what launching a new website is like.

Here are some stats for our first 12 hours:

• We had 2,151 unique visitors
• We served 9,042 page views
• We signed up 86 new ungluers, to total 665
• We generated 68 pledges for 5 campaigns totaling $1,098
• 543 tweets were sent about unglue.it

Hope Springs.... Unglue.it Prepares to Launch Effort to Make Books Free Via Crowdsourcing

• 74 added ungluers
• $693 more pledges
• 1761 unique visitors
• 5506 page views
• high traffic blog posts from Audrey Watters, Cory Doctorow, and Хабрахабр

by Eric Hellman (noreply@blogger.com) at May 18, 2012 01:44 AM

I found this online, and it’s a perfect example of a bad habit I’ve been trying to clean up for some time. When I’m trying out software that needs a MySQL database, I’m used to create database foo; but not creating a specific user for that database. Sure, if it’s in the install steps it’s easy... Read more »

From HOWTO create a normal MySQL user on fak3r by fak3r

1. HOWTO: replicate, backup, copy or move a mySQL database
2. MySQL Cheat Sheet
3. HOWTO: configure MySQL's my.cnf file

by fak3r at May 17, 2012 07:04 PM

Eric Hellman of Openly Informatics fame (subsequently bought by my employer OCLC) has launched his new project: Unglue.it. The site uses a crowd-sourced funding (or “crowdfunding”) model to raise enough money to pay book authors to open up their books as ebooks for free. As described on the site:

Unglue.it is a a place for individuals and institutions to join together to give their favorite ebooks to the world. We work with rights holders to decide on fair compensation for releasing a free, legal edition of their already-published books, under Creative Commons licensing. Then everyone pledges toward that sum. When the threshold is reached (and not before), we collect the pledged funds and we pay the rights holders. They issue an unglued digital edition; you’re free to read and share it, with everyone, on the device of your choice, worldwide.

This follows the model of sites like Kickstarter.com, where individuals pledge various amounts to support projects. Like KickStarter, Unglue.it offers various rewards pegged at specific pledge amounts as compensation to contributors. Also like KickStarter, each book “campaign” on Unglue.it has an end date.

For example, the campaign for Riverwatch by Joseph Nassise is well underway, but nearly $25,000 must be raised in 43 days to open it up for everyone. One of the benefits offered is a 45-minute Skype video chat for your  library or reading group for $150.

I’ll be watching this with interest, to see if it is indeed possible to open up books for everyone this way. Who knows? I may even be pledging.

by Roy Tennant at May 17, 2012 05:04 PM

One of the outcomes I’m looking for in this series of visits is unintended consequences. Did things happen that we didn’t predict, or plan for in the project? Did things change for the better, or worse? Here in Peru, I’ve been surprised and impressed at how the project has spun off across the country. A small pilot project to visit schools to promote libraries and librarianship as a profession. Collaborations with municipal governments on the development of new regional libraries. Workshops to sensitise mayors to the public library of today. An association that is supporting libraries and library workers across the country, whether they are members or not (in Latin America, many associations are hamstrung by legislation that requires associations, or Colegios, to only admit professional graduates as members).

The project has also had an impact on the library association serving Peru’s Amazon region and peoples. Under threat from resource exploitation and displacement, the association is a critical advocate for preservation of cultural heritage, traditional information, and lifestyles in the country’s northern jungles. The challenges they describe in preserving lifestyles and culture remind me very much of an excellent comic book produced by Survival International a few years ago, “There you go” about the problems associated with imported approaches to sustainable development. If you haven’t seen it, I highly recommend it.

This has been a welcome surprise because we didn’t set out to have an impact on this association as well. They’ve made use of the workshops, and I’m looking forward to sharing what they are doing with others – I cannot think of another association that has such a specific, dedicated advocacy purpose although of course there are many working on traditional knowledge and cultural issues around the world.

Change in Peru is on the up, with more sustainable and environmental initiatives being promoted by the government and a focus on inclusion, both of which are positive for libraries. For more on this, see Beyond Access’ blog, which talks about their recent visit here.

I heard on the weekend while cycling around Barranco and climbing the Temple  of the Sun at Pachacamac that pride in local culture and patrimony is “trendy” amongst Peruvians right now (pride in local culture, visits to local sites, purchase of local products), which does bode well for cultural preservation initiatives.

Now, for me, after a great few days in Peru during what they call “Winter” (it’s tshirt weather) it’s time to head some 13,000km North East. Next stop: Ukraine!

by Fiona at May 17, 2012 01:28 PM

Last year CrossRef and DataCite announced support for HTTP content negotiation for DOI names. Today, we are pleased to report further collaboration on the topic. We think it is very important that the two largest DOI Registration Agencies work together in order to provide metadata services to DOI names.

The current implementation is documented in detail at http://crosscite.org/cn. The documentation explains HTTP content negotiation as implemented by both Registration Agencies and provides a list of supported content types.

An example application of HTTP content negotiation is a citation formatting service. You can try it at http://crosscite.org/citeproc. This service will accept DOIs from both CrossRef and DataCite, unlike the previous formatting service which accepted only CrossRef DOI names (http://citation.crrd.dyndns.org). This is possible because CrossRef and DataCite support a shared, common metadata format. When you input a DOI into the formatting service, it doesn't know where the DOI was registered. The service will make an HTTP content negotiation request to the global DOI resolver specifying which format of the metadata should be returned in the HTTP Accept header. The global DOI resolver will notice (Accept header!) that this is not a regular DOI resolution request; it will turn to CrossRef or DataCite accordingly for the relevant metadata instead of redirecting to a landing page. The format of metadata is shared between both registration agencies so the formatting service can interpret it without knowledge of the DOI origin.

In summary HTTP content negotiation lets you process a DOI's metadata without knowledge of its origin or specifics of the registration agency.

If you have any problems, email us at tech@datacite.org or labs@crossref.org. For general discussion please kindly leave a comment below.

May 17, 2012 01:17 PM

At universities across the country, you’ll find soda and snack vending machines in the hallways of higher learning. I wouldn’t doubt you might also find some machines selling beer. But, here’s something different: in the lobby of Virginia Tech’s College of Engineering, they’ve got a vending machine that is one of the coolest things I’ve seen: it delivers creations, 3D printed ones.

It’s called the DreamVendor, a mysterious sounding name, which actually pretty accurately describes it. It’s a one-of-a-kind, interactive 3D printing station to enable Virginia Tech students to freely and quickly fabricate prototypes for their academic, and even personal, design projects.

I think this offers an excellent example of how 3D printing could be done at public libraries. Anyone want to take on the challenge?

(via 3D Printer)

by Fiacre at May 17, 2012 02:50 AM

Here the steps I just took to install metaproxy (which requires yaz and yaz++) on Red Hat Enterprise Linux 6.2.  The reason for this exercise is because Indexdata’s RPMs don’t work for 6.2 (the versions of boost-devel and icu-devel they require seem to only be available in 5.5).  Since I expect Indexdata to eventually release 6.2 compatible RPMs, I installed all of this into /opt/local (so it’s easy to remove — of course, if you’re already using /opt/local, you might want to try somewhere else).  Also, this assumes you’ll put a metaproxy.xml in /opt/local/etc/metaproxy/, so keep that in mind.

1. yum install boost boost-devel icu icu-devel libxml2 libxml2-devel gnutls gnutls-devel libxslt libxslt-devel gcc-c++ libtool
2. Install yaz:
   1. wget http://ftp.indexdata.dk/pub/yaz/yaz-4.2.33.tar.gz
   2. tar -zxvf yaz-4.2.33.tar.gz
   3. cd yaz-4.2.33
   4. ./configure –prefix=/opt/local
   5. make
   6. make install
3. Install yaz++
   1. wget http://ftp.indexdata.dk/pub/yazpp/yazpp-1.3.0.tar.gz
   2. tar -zxvf yazpp-1.3.0.tar.gz
   3. cd yazpp-1.3.0
   4. ./configure –prefix=/opt/local/ –with-yaz=/opt/local/bin
   5. make
   6. make install
4. Install metaproxy
   1. wget http://ftp.indexdata.dk/pub/metaproxy/metaproxy-1.3.36.tar.gz
   2. tar -zxvf metaproxy-1.3.36.tar.gz
   3. cd metaproxy-1.3.36
   4. ./configure –prefix=/opt/local –with-yazpp=/opt/local/bin/
   5. make
   6. make install
5. cd /opt/local
6. mkdir etc; mkdir etc/metaproxy; mkdir etc/sysconfig
7. Copy this gist as /etc/rc.d/init.d/metaproxy
8. chmod 744 /etc/rc.d/init.d/metaproxy
9. Copy this gist as /opt/local/etc/sysconfig/metaproxy
10. chkconfig –add /etc/rc.d/init.d/metaproxy
11. /etc/init.d/metaproxy start

by Ross at May 16, 2012 08:08 PM

by ksattler at May 16, 2012 07:48 PM

As a web developer, I cringe at deprecated code and try my best to keep up to date, which right now means familiarizing myself with HTML5 and CSS3. In reflecting on how best to update our website, I realized that with a CMS, naturally some things are out of my control.

Giving Up Control & Relying on Developers

Whether it’s the core or plugins, users of a CMS are reliant on its developers to keep things up to date. Is that lost of control worth the benefits? Generally, I would say yes, but that doesn’t stop me from wishing that the technology that we use to adopt new specifications.

WordPress & HTML5

Image Tags & Properties

I think it’s interesting that in HTML5 there is now the figure and figcaption elements. If they are taken advantage of, I think it definitely helps to parse information in a webpage and to identify text that is directly related to images.

One thing that does bother me about WordPress (which actually has noting to do with HTML5) is that it forces users to have a title, and leaves alt text blank by default. I don’t know what the best solution may be, but I would propose to insert the title text into the alt text by default and then allowing the user to change it. If they want to leave it blank, then there should be a checkbox to mark it “intentionally left blank” or something. Perhaps this could be an admin option, but I would definitely want something like that since I would really like to force our users to have alt text, but I don’t want to touch the WP core obviously.

Text Formatting Tags

It’s a bit of a minor thing and while some may argue the usefulness of the different semantic tags, users of the rich text editor would have no notion that they’re using <strong> instead of <b> or <em> instead of <i>. While I admit that even I struggle on the appropriate use of each (I have to look it up every time I think about it), if we want to see widespread adoption, then we need to get users to think about their writing and what they intend to do when using any of strong, em, b, i.

Tables

While we avoid tables and it should never be used for layouts, users will still want to insert tables to display data without resorting to an image. I’ve always wondered that WordPress doesn’t have a table insertion button even under the kitchen sink. What worries me is that then users who have a basic knowledge of HTML will insert it themselves using the HTML view with improperly formed code.

Layout & Forms

You might wonder why I’d lump the two, and that’s because, other than (using the default) comment form, both of these are dependent on a WordPress setup.

Forms will generally depend on the plugin. Similarly, whether the layout is in HTML5 is very dependent on the theme, along with many elements of accessibility.

Unfortunately, while HTML5 themes are relatively easy to find, most form plugins do not tell you whether they are using HTML5 or how much of it.

Why Not Adopt HTML5

I do realize that while there are a number of advantages to HTML5, especially in terms of structure,  it’s still in development. Working in an educational institution, it’s also more work and sometimes difficult in some cases to ensure backwards compatibility.

In particular, screen readers do not necessarily support all the new HTML5 elements and will frequently ignore whole chunks of text or have difficulty with reading links, etc. Even the newest versions of screen readers do not necessarily recognize elements and properties designed to make webpages easier for screen readers to interpret.

I would like to think that since WordPress talks about trying to be accessible that anything in the WordPress core will be updated once there is widespread adoption not only among browsers, but also screen readers. Obviously, adoption will take time though. For example, many form input types have been adopted by most browsers, but has not been adopted by IE at all (will be in IE10).

One can only hope that adoption will pick up once various part of the HTML5 specifications are ‘cemented.’

by Cynthia at May 16, 2012 06:50 PM

by vedmonds at May 16, 2012 04:56 PM

by mleggott at May 16, 2012 12:15 AM

by verwirrung at May 15, 2012 08:57 PM

by verwirrung at May 15, 2012 08:57 PM

We have a great panel this year for Top Tech Trends at the ALA Annual Conference in Anaheim! The panelists will describe changes and advances in  technology that they see having an impact on the library world, and suggest what libraries might do to  take advantage of these trends. Presentation of LITA Awards and Scholarships will take place prior to the Top Tech Trends program.

When: Sunday, June 24, 2012 – 1:30pm – 3:30pm
Where: Anaheim Convention Center, Ballroom A

The panelists:

•     Stephen Abram, Gale Cengage Learning
•     Lorcan Dempsey, OCLC
•     Meredith Farkas, Portland State University Library
•     Clifford Lynch, CNI
•     Nina McHale, Arapahoe Library District, Colorado

by mprentice at May 15, 2012 08:47 PM

by jrochkind at May 15, 2012 08:30 PM

The May 2012 edition of the Evergreen newsletter focuses on the April International Conference in Indianapolis, Indiana.

You can read the full text of the newsletter by visiting the following Evergreen wiki page.

To submit your own entries for the June newsletter, you can email Amy Terlaga at terlaga@biblio.org.

by Amy Terlaga at May 15, 2012 06:29 PM

Last week saw a big (well big for library data nerds) announcement from OCLC that they are making the data for the Virtual International Authority File (VIAF) available for download under the terms of the Open Data Commons Attribution (ODC-BY) license. If you’re not already familiar with VIAF here’s a brief description from OCLC Research:

Most large libraries maintain lists of names for people, corporations, conferences, and geographic places, as well as lists to control works and other entities. These lists, or authority files, have been developed and maintained in distinctive ways by individual library communities around the world. The differences in how to approach this work become evident as library data from many communities is combined in shared catalogs such as OCLC’s WorldCat.

VIAF’s goal is to make library authority files less expensive to maintain and more generally useful to the library domain and beyond. To achieve this, VIAF seeks to include authoritative names from many libraries into a global service that is available via the Web. By linking disparate names for the same person or organization, VIAF provides a convenient means for a wider community of libraries and other agencies to repurpose bibliographic data produced by libraries serving different language communities

More specifically, the VIAF service: links national and regional-level authority records, creating clusters of related records and expands the concept of universal bibliographic control by:

• allowing national and regional variations in authorized form to coexist
• supporting needs for variations in preferred language, script and spelling
• playing a role in the emerging Semantic Web

If you went and looked at the OCLC Research page you’ll notice that last month the VIAF project moved to OCLC. This is evidence of a growing commitment on OCLC’s part to make VIAF part of the library information landscape. It currently includes data about people, places and organizations from 22 different national libraries and other organizations.

Already there has been some great writing about what the release of VIAF data means for the cultural heritage sector. In particular Thom Hickey’s Outgoing is a trove of information about the project, which provides a behind-the-scense look at the various services it offers.

Rather than paraphrase what others have said already I thought I would download some of the data and report on what it looks like. Specifically I’m interested in the RDF data (as opposed to the custom XML, and MARC variants) since I believe it to have the most explicit structure and relations. The shared semantics in the RDF vocabularies that are used also make it the most interesting from a Linked Data perspective.

Diving In

The primary data structure of interest in the data dumps that OCLC has made available is what they call the cluster. A cluster is essentially a hub-and-spoke model with a resource for the person, place or organization in the middle that is attached via the spokes to conceptual resources at the participating VIAF institutions. As an example here is an illustration of the VIAF cluster for the Canadian archivist Hugh Taylor

Here you can see a FOAF Person resource (yellow) in the middle that is linked to from SKOS Concepts (blue) for Bibliothèque nationale de France, The Libraries and Archives of Canada, Deutschen Nationalbibliothek, BIBSYS (Norway) and the Library of Congress. Each of the SKOS Concepts have their own preferred label, which you can see varies across institution. This high level view obscures quite a bit of data, which is probably best viewed in Turtle if you want to see it:

<http://viaf.org/viaf/14894854>
    rdaGr2:dateOfBirth "1920-01-22" ;
    rdaGr2:dateOfDeath "2005-09-11" ;
    a rdaEnt:Person, foaf:Person ;
    owl:sameAs <http://d-nb.info/gnd/109337093> ;
    foaf:name "Taylor, Hugh A.", "Taylor, Hugh A. (Hugh Alexander), 1920-", "Taylor, Hugh Alexander 1920-2005" .

<http://viaf.org/viaf/sourceID/BIBSYS%7Cx90575046#skos:Concept>
    a skos:Concept ;
    skos:inScheme <http://viaf.org/authorityScheme/BIBSYS> ;
    skos:prefLabel "Taylor, Hugh A." ;
    foaf:focus <http://viaf.org/viaf/14894854> .

<http://viaf.org/viaf/sourceID/BNF%7C12688277#skos:Concept>
    a skos:Concept ;
    skos:inScheme <http://viaf.org/authorityScheme/BNF> ;
    skos:prefLabel "Taylor, Hugh Alexander 1920-2005" ;
    foaf:focus <http://viaf.org/viaf/14894854> .

<http://viaf.org/viaf/sourceID/DNB%7C109337093#skos:Concept>
    a skos:Concept ;
    skos:inScheme <http://viaf.org/authorityScheme/DNB> ;
    skos:prefLabel "Taylor, Hugh A." ;
    foaf:focus <http://viaf.org/viaf/14894854> .

<http://viaf.org/viaf/sourceID/LAC%7C0013G3497#skos:Concept>
    a skos:Concept ;
    skos:inScheme <http://viaf.org/authorityScheme/LAC> ;
    skos:prefLabel "Taylor, Hugh A. (Hugh Alexander), 1920-" ;
    foaf:focus <http://viaf.org/viaf/14894854> .

<http://viaf.org/viaf/sourceID/LC%7Cn++82148845#skos:Concept>
    a skos:Concept ;
    skos:exactMatch <http://id.loc.gov/authorities/names/n82148845> ;
    skos:inScheme <http://viaf.org/authorityScheme/LC> ;
    skos:prefLabel "Taylor, Hugh A." ;
    foaf:focus <http://viaf.org/viaf/14894854> .

The Numbers

The RDF Cluster Dataset http://viaf.org/viaf/data/viaf-20120422-clusters.xml.gz is 2.1G gzip compressed RDF data. Rather than it being one complete RDF/XML file, each line has a complete RDF/XML document on it, which represents a single cluster. All in all there are 20,379,541 clusters in the file.

I quickly hacked together a rdflib filter that reads the uncompressed line-oriented RDF/XML and writes the RDF as ntriples:

import sys
 
import rdflib
 
for line in sys.stdin:
    g = rdflib.Graph()
    g.parse(data=line)
    print g.serialize(format='nt').encode('utf-8'),

This took 4 days to run on my (admittedly old) laptop. If you are interested in seeing the ntriples let me know and I can see about making it available somewhere. It is 2.8G gzip compressed. An ntriples dump might be a useful version of the RDF data for OCLC to make available, since it would be easier to load into triplestores, and otherwise muck around with (more on that below) than the line oriented RDF/XML. I don’t know much about the backend that drives VIAF (has anyone seen it written up?)…but I would understand if someone said it was too expensive to generate, and was intentionally left as an exercise for the downloader.

Given its line-oriented nature, ntriples is very handy for doing analysis from the Unix command line with cut, sort, uniq, etc. From the ntriples file I learned that the VIAF RDF dump is made up of 377,194,224 assertions or RDF triples. Here’s the breakdown on the types of resources present in the data:

Here’s a breakdown of predicates (RDF properties) that are used:

I’m expecting these statistics to be useful in helping target some future work I want to do with the VIAF RDF dataset (to explore what an idiomatic JSON representation for the dataset would be, shhh). In addition to the RDF, OCLC also makes a dump of link data available. It is a smaller file (239M gzip compressed) of tab delimited data, which looks like:

...
http://viaf.org/viaf/10014828   SELIBR:219751
http://viaf.org/viaf/10014828   SUDOC:052584895
http://viaf.org/viaf/10014828   NKC:xx0015094
http://viaf.org/viaf/10014828   BIBSYS:x98003783
http://viaf.org/viaf/10014828   LC:24893
http://viaf.org/viaf/10014828   NUKAT:vtls000425208
http://viaf.org/viaf/10014828   BNE:XX917469
http://viaf.org/viaf/10014828   DNB:121888096
http://viaf.org/viaf/10014828   BNF:http://catalogue.bnf.fr/ark:/12148/cb13566121c
http://viaf.org/viaf/10014828   http://en.wikipedia.org/wiki/Liza_Marklund
...

There are 27,046,631 links in total. With a little more Unix commandline-fu I was able to get some stats on the number of links by institution:

The 301,345 links to Wikipedia are really great to see. It might be a fun project to see how many of these links are actually present in Wikipedia, and if they can be automatically added with a bot if they are missing. I think it’s useful to have the HTTP identifier in the link dump file, as is the case for the BNF identifiers. I’m not sure why the DNB, Sweden, and LC URLs aren’t expressed URLs as well.

One other parting observation (I’m sure I’ll blog more about this) is that it would be nice if more of the data that you see in the HTML presentation were available in the RDF dumps. Specifically, it would be useful to have the Wikipedia links expressed in the RDF data, as well as linked works (uniform titles).

Anyway, a big thanks to OCLC for making the VIAF dataset available! It really feels like a major sea change in the cultural heritage data ecosystem.

by ed at May 15, 2012 05:57 PM

LITA is pleased to announce the availability of a new webinar, “Social Networking the Catalog: Community Based Approaches to Building Catalogs and Collections,” presented by Margaret Heller (Dominican University) and held June 7, 11:00 am – Noon CDT. This presentation will introduce the Read/Write Library Chicago, a new model for libraries that exists to illuminate and create connections between people, materials, and institutions in the city of Chicago. Participants will learn about new trends and features in social reading and cataloging, social library catalogs and integrated library systems, and crowdsourcing platforms. The catalog builds a social network, and the social network in turn builds the catalog. We love the read/write internet. It’s time to create the read/write library, where everyone has a voice in the collections, catalog, programming, and mission. This model is extensible to more conventional libraries through a multitude of tools and approaches that are already widely used.

Additional information and registration are available at http://www.ala.org/lita/learning/online/socialcatalog

by mprentice at May 15, 2012 05:54 PM

by mcbrydemlis at May 15, 2012 03:35 PM

Hello everyone,

Evergreen 2.2 rc1 was just released today, 15 May 2012. This is the
release candidate. The Evergreen community hopes that Evergreen 2.2.0
will follow in just about two weeks, depending as always on feedback from
those who contribute their feedback after testing.

This release includes various bug fixes, please see the full list of
changes.

The 2.2 series includes many new features over the 2.1 series, including
the Template Toolkit OPAC (TPAC) and too many others to count.

Please report any new bugs on Launchpad.

I would like to particularly thank Thomas Berezansky, Ben Shum, Jason
Stephenson and Dan Scott for assisting in innumerable ways with the
mechanics of publishing this release candidate. I am surely neglecting a
couple of other folks whose help was invaluable, but at least they have
their karma.

Thanks everyone!

by Lebbeous Fogle-Weekley at May 15, 2012 03:10 PM

Editorial by Laurence Lannom, CNRI

May 15, 2012 01:18 PM

Article by Natasha Simons, Griffith University, Australia

May 15, 2012 01:18 PM

Article by R. Niccole Westbrook and Dan Johnson, University of Houston Libraries; Karen Carter, Rutgers School of Communication and Information; Angela Lockwood, Texas Women's University School of Library and Information Studies

May 15, 2012 01:18 PM

Article by Christopher A. Lee, Alexandra Chassanoff, and Kam Woods, University of North Carolina, Chapel Hill; Matthew Kirschenbaum and Porter Olsen, University of Maryland

May 15, 2012 01:18 PM

Article by Andras Holl, Konkoly Observatory, Budapest, Hungary

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

May 15, 2012 01:18 PM

For those looking for yet another reason to join us for OKFestival in Helsinki this September, the OKFestival Core Organising Team is proud to announce the inspiring public outcomes of our unconventional First Call for Proposals – and to request your participation for our Second Call to share your ideas in Finland.

As we’ve noted previously, because OKFestival is the first event of its kind, combining Open Knowledge Conference and Open Government Data Camp together for a week-long celebration of action and collaboration, we decided to take a risk by opening up over 2/3 of the week’s programme to you as festival participants.

So last month, we released the First Call for Proposals, crossing our fingers expectantly as we did it. A few of us on the Core Organising Team (photo) were, admittedly, a tad worried – would global communities rise to the challenge? Or would we be left alone in cyberspace without even a programme to our name? We presented the festival to audiences at FREE CITY in Tallinn, at Re:Publica in Berlin and to local stakeholders in Finland. And we waited in anticipation.

In the end, we didn’t have to worry at all. The response to our First Call for Proposals was both overwhelming and encouraging. Open knowledge and data enthusiasts around the world did take the reins – and now, a month later, we have a groundbreaking, action-focused programme planned in co-operation with citizen teams of Guest Programme Planners all over the world. For a summary of the Open Knowledge Festival planning process in 14 slides, see our first Slideshare presentation here.

As you'll see above, the First Call for Proposals allowed the Core Organising Team to determine the most important themes and salient ideas, the subjects of which are highlighted through our 13 guest-organised Topic Streams of 2012:

1. Open Democracy and Citizen Movements
2. Open Government Data
3. Open Cities
4. Open Design, Hardware & Manufacturing
5. Open Cultural Heritage
6. Open Development
7. Open Research and Education
8. Open Geodata
9. Open Source Software
10. Data Journalism and Data Visualization
11. Gender / Diversity in Openness
12. Open Business and Corporate Data
13. Open Knowledge and Sustainability

The breadth of these topics is quite diverse - indeed, the variance is somewhat unprecedented for an event of this kind. Going through the topics above and learning more about how their Guest Programme Planners are determining the programming on the Public Planning Wiki, it's hard not to feel a sense of excitement about what's to come.

For the Second (and last!) Call for Proposals, we encourage ideas that further enrich each of these themes with new perspectives. We want your lightning talks, lectures, panel discussions, workshops, hackathons and all things in between. Let's fill Helsinki's streets with innovative new ideas, new collaborations between civil society and government, and new projects that provoke openness in unexpected ways.

It is our hope that together, these themes will illustrate the importance of diverse understandings within open knowledge and open data communities - and we look forward to seeing even more of you get involved in this inspiring process.

The Second Call for Proposals is here. Deadline for submission is June 1st - go to okfestival.org for details. And feel free to mix and remix the Slideshare presentation above for your own uses - it's meant to be shared!

by Kat Braybrooke at May 15, 2012 12:05 PM

I started blogging a little over three years ago. I found that it was a great way to organize my thoughts and it gave me an excuse to talk to people and ask questions about things that interested me. It became an extended conversation with so many readers about the future of libraries and the role of books and readers in our changing society. Also polarons, faster-than-light neutrinos, and log-normal distributions.

by Eric (noreply@blogger.com) at May 14, 2012 07:29 PM

The Koha News Tool allows librarians to post news items to the OPAC and the staff client. In Koha 3.4 the news tool can also be used to add news items to your circulation receipts or slips. This tutorial will walk you through the simple steps of adding and viewing news items.

If you have an idea for a video, please just let me know and I’ll add it to my list of things to record.

Related posts:

1. Koha Offline Circulation Tool
2. So much Koha news today
3. Adding a Child Patron in Koha 3.2

by Nicole at May 14, 2012 03:00 PM

Using OAI-PMH to populate the “Catholic Portal” is not straight-forward, and this posting outlines some of my investigations in this regard.

Introduction

As you may or may not know, OAI-PMH is a “standard” protocol designed for harvesting metadata. It only understands six commands (or in OAI-PMH parlance, “verbs”). These commands are sent to remote computers in the form of URLs, and the remote computer is expected to respond in the form of specifically shaped XML streams. These commands include:

• Identify – Lists who manages the repository and what type of content it contains.
• ListMetadataFormats – Lists the various metadata schemes used to describe the repository’s content. At least one of these schemes must be Dublin Core.
• ListSets – Specifies how the repository’s content is subdivided. There can be zero or more of these subdivisions.
• ListIdentifiers – Returns a list of keys pointing to specific records in the repository.
• ListRecords – An enhanced version of ListIdentifiers, this verb downloads whole records, not just identifiers.
• GetRecord – Given a specific identifier, this verb retrieves a single record.

Through a conversation of these verbs and the returned XML streams, metadata between computers can be exchanged. It is then up to the computer doing the harvesting to implement some sort of cool and interesting service with the harvested content. Here at Catholic Portal Central we want to index the metadata and provide immediate access to remote digitized content.

Investigations

At least three Catholic Research Resources Alliance (CRRA) members have OAI-PMH repositories: Duquesne University, Boston College, and Loyola University Chicago. Using a little Perl script, I most recently investigated the content of the repositories of Boston College and Loyola University Chicago. Through this process I learned what metadata formats they supported, what sets were used to subdivided their collections, and output Dublin Core metadata from a few selected sets.

The harvested Dublin Core metadata was typical of OAI-PMH repositories: thin, a bit ambiguous, and somewhat inconsistant across repositories. It was thin because many of the Dublin Core elements are left unpopulated. It is ambiguous because many of the fields are repeated, and the values of repeated elements are of different types. For example, a description field may be empty, contain an abstract of the work, the full text of the work, or the process used to digitize the material. It is inconsistant because things like dates, names, and subject entries are formatted differently. In some places names are listed in first name/last name order. Other times it is last name/first name order. Dates can be anything from “February 12, 2012″ to “2012-02-12″ to “Twelfth Century”. None of this is new the world of OAI-PMH. It is typical.

All is not lost. There are patterns to this apparent randomness. Using my script I can sometimes output titles, descriptions, subject headings, and URLs of digitized objects. For example, here is such a list from the Loyola University Chicago repository:

item: 46

key: oai:content.library.luc.edu:coll6/45

title(s): Letter to the Secretary of the Literary Agency of London, 1908
title(s): Catholic Women Poets

identifier(s): cudahy219e3

identifier(s): 003_kayesmith_1908;pg3.jpg

identifier(s): http://content.library.luc.edu/u?/coll6,45

subject(s): Shelia Kaye-Smith; poets; women poets; Catholic poets

subject(s): Local

description(s): third page of letter requesting appointment

description(s): does not suit you any other time up to 4 15 will do Would you kindly send a reply to me c o Miss F E Walters Girton College Cambridge With apologies for troubling you believe me Yours faithfully Sheila Kaye Smith

description(s): Master file scanned at 600 dpi RGB in reflective mode from original document using MicroTek ScanMaker 1000XL

description(s): http://www.luc.edu.archives

type: image

From this output it becomes apparent that the first title is the title of the artifact, the third identifier is the URL of the digitized object, the first subject field is a delimited list of keywords, the first description is a sort of abstract, and the type field contains a value denoting what kind of digitized thing is in question. Thus, the output follows a pattern, and computers are very good at patterns, therefore a computer program could easily be written to read this particular OAI-PMH output and stored in the Portal’s index.

Next steps

My next steps are two-fold. First, I will harvest and index some of the metadata from selected Loyola University Chicago OAI-PMH sets. Second, I will let colleagues from various CRRA committees (specifically the Digital Access Committee as well as the Collection Committee) peruse the results. In the end I hope to get feedback on how to proceed. Should I index more content? Less? None? If more, then how should records be displayed, and exactly how ought the Dublin Core metadata be mapped to VuFind’s underlying Solr index fields?

All of this work is entirely feasible. At the same time it is not enormously scalable. Hand-crafting the parsing of OAI-PMH output, and handcrafting how it all gets mapped to Solr’s index is time consuming and fragile. The Portal Home Planet can easily do this work for no more than a dozen different repositories, but after that some other means of production will need to be examined.

by Catholic Portal at May 14, 2012 02:59 PM

Call me crazy, but I think the secret life of checkout slips is fascinating.
Some moms use their foot-long slips filled with children’s books as a master list, crossing off items as they’re returned. One regular patron I knew kept every checkout slip she ever received. Upon returning items, she’d ask us to cross off the titles on the original slip and initial it. This behavior was the result of a typical “I returned that”/“Not according to our computers” interaction.

And, of course, countless slips are used as bookmarks or refrigerator-mounted notices or simply left in dust jackets for weeks. However small, these slips are touchpoints—ways that people interact with us—and collectively we’re pumping out thousands of these things daily.

Likewise, in some small way, we’re representing ourselves through these little scraps of paper. Yet, most of us are churning out slips that could be easier on the eye and more helpful to our users.

This isn’t something to keep you up at night, but it’s still worth thinking about, because details matter. All of these little touchpoints add up to create people’s experience of our libraries. And dispensing ugly checkout receipts illustrates that we haven’t spent enough time sweating these details. Even worse, this inattention is at the root of complaints about hard-to-use websites and repeated questions about where the restrooms are.

What is a good checkout slip?
To answer this we have to know what a checkout slip is supposed to do. As I see it, there are a few core ­functions:

• remind people when items are due (patron need)
• remind people what items they have checked out (patron need)
• facilitate the return of materials (library need)

Beyond that, some slips have secondary functions:

• facilitate renewing items (patron need)
• promote library events (library need)
• broadcast policy changes (library need)
• alert people to holiday hour changes (patron need)

With these factors in mind, we can now think of some other factors surrounding the design of an ideal checkout slip:

• They should respect people’s privacy.
• They should include the library’s name and branding.
• They should be easy to read. This includes obeying graphic design basics as well as not cutting off item titles, etc.
• Ideally, they’d show some personality and/or be friendly.
• Item types could be helpful to patrons trying to locate a misplaced item.

Checking out the fun
After compiling the functions’ lists, I started to think about whether there was a way to make checkout slips more fun, or whether that was a terrible impulse. More seriously, I considered what would be the minimum amount of information required to make an item easily identifiable and other basic considerations, such as why there is a due date listed for each item when most items share a due date with others. With all of these things in mind, I took a crack at designing a checkout slip.

There’s nothing very different about this design, but I reckon it is a bit easier to use when hanging on a refrigerator than the current crop. Aside from sensible typography, the only thing notable is that items are grouped by due date rather than listing a due date for each item.

I really like the idea of a checkout slip that includes an extra bit that’s specifically meant to be displayed on a refrigerator or corkboard, though such a design could add about three or more inches of length per due date. It might be cumbersome, but consider how much better this communicates your library’s philosophy.

Just remember: the details matter, especially when these checkout slips are the most visible output of your library that most users will see.

This first appeared “The User Experience,” a column I write for LJ.

by Aaron Schmidt at May 14, 2012 12:35 PM

Dan Olds at The Register comments on an interview with co-director of the Wharton School Customer Analytics Initiative Dr. Peter Fader:

Dr Fader ... coins the terms "data fetish" and "data fetishist" to describe the belief that people and organisations need to capture and hold on to every scrap of data, just in case it might be important down the road. (I recently completed a Big Data survey in which a large proportion of respondents said they intend to keep their data “forever”. Great news for the tech industry, for sure.)

But a Big Data zealot might say, "Save it all—you never know when it might come in handy for a future data-mining expedition."

by David. (noreply@blogger.com) at May 14, 2012 10:00 AM

RDFa and HTML5 microdata, are, I think, basically interchangeable.

RDF and microdata both use the same fundamental triple data model. Please note that schema.org is just a specific set of vocabularies that can be used with HTML5 microdata,  HTML5 microdata goes beyond this. schema.org is a pretty good microdata tutorial though, if you remember you don’t have to use it’s vocabularies.  Here’s the actual microdata spec. Here’s a good microdata tutorial that pre-dates schema.org and is not schema.org-specific.

You can take pretty much anything that’s RDF, from any vocabularies, and use an RDFa style approach to express (basically) the same semantics is in HTML5 microdata  instead.

This is a good thing for RDF, because there’s no good way to do RDFa in HTML (or anything but xHTML which is basically an abandoned approach — RDFa needs XML namespaces).  You can go from (any) html5 microdata to RDF too — although there are a couple gaps I’ll discuss at the end.

First, let’s show how you’d do RDFa-style RDF semantics expressed in HTML5 microdata. Let’s take the complete example from the RDFa wikipedia article, as it’s small but makes us actually use a pretty complete complement of microdata features. There are in fact a couple weird details I’m not sure about.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"
    "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    version="XHTML+RDFa 1.0" xml:lang="en">
  <head>
    <title>John's Home Page</title>
    <base href="http://example.org/john-d/" />
    <meta property="dc:creator" content="Jonathan Doe" />
    <link rel="foaf:primaryTopic" href="http://example.org/john-d/#me" />
  </head>
  <body about="http://example.org/john-d/#me">
    <h1>John's Home Page</h1>
    <p>My name is <span property="foaf:nick">John D</span> and I like
      <a href="http://www.neubauten.org/" rel="foaf:interest"
        xml:lang="de">Einstürzende Neubauten</a>.
    </p>
    <p>
      My <span rel="foaf:interest" resource="urn:ISBN:0752820907">favorite
      book is the inspiring <span about="urn:ISBN:0752820907"><cite
      property="dc:title">Weaving the Web</cite> by
      <span property="dc:creator">Tim Berners-Lee</span></span>
     </span>
    </p>
  </body>
</html>

Here’s the same thing, using the same vocabularies, with HTML5 microdata. (yes, contrary to some belief, you can mix and match more than one vocabulary in microdata too, although you’ve got to spell out the complete URI for all but one in any given scope.

<html lang="en">
  <head>
    <title>John's Home Page</title>
    <base href="http://example.org/john-d/" />
    <link rel="http://xmlns.com/foaf/0.1/primaryTopic" href="http://example.org/john-d/#me" />
  </head>
  <body itemscope itemtype="http://purl.org/dc/elements/1.1/"  itemid="http://example.org/john-d/#me">
    <h1>John's Home Page</h1>
    <p>My name is <span itemprop="http://xmlns.com/foaf/0.1/nick">John D</span> and I like
      <a href="http://www.neubauten.org/" itemprop="http://xmlns.com/foaf/0.1/interest"
        lang="de">Einstürzende Neubauten</a>.
    </p>
    <p>
      <span itemscope itemtype="http://purl.org/dc/elements/1.1/" itemprop="http://xmlns.com/foaf/0.1/interest" itemid="urn:ISBN:0752820907 ">
      My favorite
      book is the inspiring <cite
      itemprop="title">Weaving the Web</cite> by
      <span itemprop="creator">Tim Berners-Lee</span>
     </span>
    </p>
  </body>
</html>

Mismatches and missing semantics

While the fundamental approach is compatible, there are a few mismatches and semantics lost or less clear in html5 microdata. Here are some I feel like noting, there may be others.

• I’m not sure I did the right thing with the <link> in the <head> section — html5 has kind of an odd fork in it between maintaining more or less backwards compatibility with old-style <link> and <meta> (in <head>, using ‘rel’), and microdata style (in <body>, using ‘itemprop’).  There are weird things with ‘rel’ only being allowed in certain places and ‘itemprop’ in others; you also are never supposed to have both ‘rel’ and ‘itemprop’.  So anyway, I’m not sure what a proper way of expressing a relationship with the document as the subject is, in microdata, may have done something not right here.
• RDFa takes XML’s namespaces to express vocabularies. RDFa’s namespace+name is analagous to microdata’s itemtype+itemprop.  But.
  • In microdata, you can do something dear to RDF’s heart, and express the predicate URL as a literal absolute URL  – which is what you have to do to mix namespaces/vocabularies, and that’s really just fine. You can also do the equivalent of a namespace (in an itemtype) and a non-URI bare name belonging to that namespace (in an itemprop), but you only get one namespace at a time like this.
  • But also, RDFa, via XML, is quite clear that you concatenate a namespace and a bare name to get the complete URI. We used this same convention when putting our RDFa into microdata, which works because itemtype’s are always URIs too.  – but it’s just a convention, microdata isn’t clear about that, and microdata examples often use itemtype URI examples that clearly weren’t intended like this. Like schema.org: itemtype=”http://schema.org/Book” + itemprop=”bookFormat” concatenated == “http://schema.org/BookbookFormat”. Um, that’s not quite sensible, not what anyone’s looking for… although it is a legal URI….
• microdata makes  a lot ‘easier’ to use what the RDFistas call ‘blank nodes’ — nodes whose ‘subject’ lacks a specified URI. Idiomatic microdata actually generally has a bunch of those, including the top-level one(s).   The microdata spec tries to tell you that you can only use an `itemid` for certain vocabularies that establish it’s use — ideally, I think this would be opened up, and even encouraged. The semantics should be made more clearly compatible with RDF — the itemid it is an identifier for the ‘itemscope’d thing, that is the ‘subject’ URI of any itemprop’s in that itemscope, that should be made clear.
  • I personally think allowing idiomatic blank nodes is a good thing for microdata, making it more usable, letting people get started with the minimal semantics for their use cases, not making them spend time on metadata design/control they don’t need yet.  Even if RDFistas disagree, I suggest they focus on making it easier to avoid blank nodes — more idiomatic, more encouraged by docs, more generally legal — and give up on making it hard or impossible to have blank nodes in html5 microdata.

Whither RDF/RDFa

(That’s “whither”, not “wither”. Hopefully).

There are probably other rough spots than the ones I’ve identified. And the one’s I mentioned include some tough ones (the itemtype+itemprop==URI issue).

But by and large, HTML5 microdata’s fundamental model is RDF compatible.  Hopefully the RDFistas are focused on figuring out how to lessen the impedence mismatches, if neccesary by lobbying the html5 working group to make minimized interventions.  Hopefully they’re not still stuck on an xhtml/rdfa/why-didn’t-they-do-things-our-way train, because that train isn’t leaving the station.  Instead though, they can contribute to sanding off a few rough spots in microdata to make it quite capable of doing what they want (and, if they’re right, everyone else will eventually realize they want too). Work on tools to turn microdata to RDF, too, hopefully.

microdata could actually be the a great thing for RDF.  If handled correctly, it should be possible to express full RDF semantics in microdata — microdata can be the RDF-in-HTML-markup standard that RDFa wanted to be. (microdata’s designers clearly knew about RDF/RDFa and were influenced by it). It’s also possible to leave a lot of semantics out when writing microdata — but often in ways you could do with RDF/RDFa too, lots of blank nodes, etc, RDF/RDFa just tries to make it inconvenient and non-idiomatic.

While the RDFistas may be rueing that microdata makes it so easy to not have completely specified triples with no blank nodes everywhere — I think the flip side of this is actually what will allow it to possibly get more uptake, and be an easy start on the road to RDF, if RDF plays it’s cards right.   That because you have to think through the complete vocabularies and semantics less, you can get started with just the semantics you need, and not be forced to do more up front metadata design than you need for your identified use cases, or more than you can afford or have the skills to do. That, and some the immediate use cases in ‘Google will use it!’ of course. But if Google had tried to say they used RDFa (didn’t they once, maybe, sort of?), I don’t think it would have gone anywhere — RDFa is just too overwhelming.

by jrochkind at May 14, 2012 04:59 AM

In the past few years, I’ve received numerous emails from the Institute for Cultural Diplomacy announcing upcoming conferences and educational programs. The messages say, “If you do not wish to receive emails from the ICD in the future, please send us an email to info@culturaldiplomacy.org indicating this.” I have, six times, spread out over half a year. It didn’t work. Twice, I cc:ed Mark Donfried, the ICD’s “Director and Founder,” over whose name the emails are written. I never received a response from him, just more spam. I called the ICD’s office, in Germany, and asked to be removed from their list. The woman who answered the phone promised I would be. She lied: the email continued.

This is unethical behavior, inconsistent with the values the ICD supposedly represents. It’s disrespectful, dishonest, and disreputable. I doubt that any of my readers run in ICD circles, but if you do, please think hard about what it says about the ICD as an organization.

by James Grimmelmann (james@grimmelmann.net) at May 13, 2012 06:58 PM

On Friday, the long-awaited decision in the Georgia State e-reserves case (a.k.a. Cambridge University Press v. Becker) dropped. By way of context, the case is a challenge by three academic publishers (Oxford University Press, Cambridge University Press, and Sage Publications) against Georgia State University’s e-reserves policy. The publishers sued in April 2008, in a lawsuit funded by the Association of American Publishers and the Copyright Clearance Center, claiming that the e-reserves policy went far beyond the bounds of fair use. Georgia State, as a state university, invoked the doctrine of sovereign immunity, the practical implication of which is that the publishers can only obtain injunctions against future infringements, not damages for past infringements. Since it also tightened up its e-reserves policy in December 2008, it also successfully argued to the court that only the uses made under the new policy should be relevant to any potential injunction.

There was a trial a year ago, and then long silence from the court. Now we know why it was taking so long: the opinion is 350 pages. That number is a little misleading, in that over two thirds of the opinion are dedicated to a highly methodical copyright ownership, infringement, and fair use analysis of seventy-four separate claims of infringement, using standard templates and highly repetitive language. Having now dug through the details, I’d like to offer a few observations.

First, over a third of the claims didn’t even make it to the fair use stage at the heart of the case. In many cases, the publishers were unable to prove to the court’s satisfaction that they owned the copyright in the portions of the books that were copied and uploaded. Sometimes they couldn’t produce a timely registration certificate and there were proof problems with originality; sometimes they couldn’t find a work-made-for-hire agreement or copyright assignment from the authors of individual chapters in edited volumes. The court was unsympathetic: no documented chain of title, no lawsuit. There’s a looming e-rights mess, loosely akin to the robosigning mess around ownership of securitized mortgages: in both cases, the putative owners don’t have all their papers in order. This opinion either recognizes or contributes to the mess, depending on your point of view.

Other claims dropped out before the fair use stage because they were uploaded to the e-reserves system but never downloaded by students. The court dismisses these from the lawsuit as de minimis, explaining that these uses by the University, while technical implicating the copyright owners’ exclusive rights, don’t affect the incentives for authors to create. This puts more teeth in the de minimis doctrine in copyright: it goes beyond the view that de minimis means “not substantially similar.” It also strengthens the argument that “internal use” copies never used to reach an to an audience that reads them for their content don’t infringe. Think, for example, of the HathiTrust’s archive of scans from Google Books.

(As an aside, the e-reserve logfiles played a key evidentiary role in the case. Specific users were never identified, but if a file had a total hit count of two, it’s unlikely that students actually read it. This stands in contrast to other cases, like American Geophysical, which was tried by sampling: the parties selected a single scientist at random, examined his files looking for photocopies, and treated him as representative of a cohort of 500. Here, the logs permitted an analysis of the copying done for numerous faculty members—presumably all those who assigned any excerpts from any of the plaintiffs’ books.)

When the court did reach fair use, it held across the board that two of the four factors favored Georgia State. The purpose of the use, while not transformative, was nonetheless for highly favored educational purposes by a nonprofit institution. And the nature of the works was consistently informational.

On the third factor, the amount copied, the court repudiated the Classroom Guidelines, calling them “not compatible with the language and intent of § 107.” It noted that the numerical limits in the Guidelines are so stringent that not one of the excerpts at issue in the case would fit within them. It was particularly uninterested in the Guidelines’ position that copying not “be repeated with respect to the same item by the same teacher from term to term,” which the court described as “an impractical, unnecessary limitation.”

Instead, the court fashioned its own quantitative test. For books of nine or fewer chapters, the court set a threshold of 10% of the total page count; for books of ten chapters or more, the threshold was a single complete chapter. (The chapter-based rule creates an odd incentive for publishers to create books with a surfeit of tiny chapters.) Copying of any amount under this threshold, the court held, would be treated as “decidedly small.” In practical terms, this ended up being a one-sided bright-line rule: copying of less than 10% or one chapter always ended in a fair use win for Georgia State.

Finally, the fourth factor, the effect on the market, favored the publishers whenever CCC was offering a digital license for copying the book in question, and favored Georgia State whenever there was “no evidence in the record to show that digital excerpts from this book were available for licensing” as of the date of infringement.” In practice, this was another one-sided bright-line rule: no digital license meant an instant win for Georgia State. The court repeatedly emphasized that students would not have bought the assigned books as a substitute for the excerpts posted on the e-reserve system.

This treatment of licensing is likely to have significant implications. On the one hand, it suggests that libraries may have a freer hand to make expanded uses of orphan works, since by definition, no one will be licensing them. And on the other, the court didn’t consider photocopying licenses to be a suitable substitute for digital licenses. This will put significant pressure on publishers to turn on digital licensing.

Only in seven instances did Georgia State use more than 10% or one chapter of a book that was available for digital licensing. When this happened, the court took a more detailed look at the specifics of the book’s licensing market and the portion copied. Generally, this turned on whether the book made significant revenues via licensing: if so, the use was unfair. (In one instance, the court did a “heart of the work” analysis under factor three to find no fair use because the professor had assigned chapters that “essentially sum up the ideas in the book.”)

Thus, the operational bottom line for universities is that it’s likely to be fair use to assign less than 10% of a book, to assign larger portions of a book that is not available for digital licensing, or to assign larger portions of a book that is available for digital licensing but doesn’t make significant revenues through licensing. This third prong is almost never going to be something that professors or librarians can evaluate, so in practice, I expect to see fair-use e-reserves codes that treat under 10% as presumptively okay, and amounts over 10% but less than some ill-defined maximum as presumptively okay if it has been confirmed that a license to make digital copies of excerpts from the book is not available.

The most interesting issue open in the case is the scope of any possible injunction. Given that Georgia State won on sixty-nine out of seventy-four litigated claims, while the publishers won on only five, I expect that the any injunction will need to be rather narrow. But given how amenable the court’s proposed limits are to bright-line treatment, it is likely that the publishers will push to write them in to the injunction.

My bottom line on the case is that it’s mostly a win for Georgia State and mostly a loss for the publishers. The big winner is CCC. It gains leverage against universities for coursepack and e-reserve copying with a bright-line rule, and it gains leverage against publishers who will be under much more pressure to participate in its full panoply of licenses.

by James Grimmelmann (james@grimmelmann.net) at May 13, 2012 04:25 PM

I hope many of you have seen the excellent and fun WebGL GLSL sandbox at http://glsl.heroku.com/. This live editing of shaders is an excellent learning tool, as it allows you to watch the consequences of any changes you make.

I am constructing some simple OpenGL ES scripts as a learning resource for anyone new to it, in part as a way to help me learn it too. As part of this, I’ve written a little script that gives a similar kind of experience, but on the commandline, with a Raspberry Pi. It overlays the render display over the top of the framebuffer window, and reloads the shader any time you save the script.

You can run this from the terminal (as I do in the video later on) or from within LXDE (X11). (You may wish to replace ‘nano’ with ‘leafpad’ in glsl_sandbox.sh if you are running it in LXDE)

Prerequisites:

* Install pyinotify

$ sudo apt-get install python-pyinotify

* Get the repository:

$ wget https://github.com/benosteen/pyopengles/zipball/master -O pyopengles.zip
$ unzip pyopengles.zip
  Inflating benosteen-pyopen....
  ...

cd into the repository and you are ready to go!

Usage of ‘glsl_sandbox.sh’:

$ bash glsl_sandbox.sh [NAME_OF_SHADER_FILE]

The repository has a few demo shaders that are known to work included for you to try – they are copied from the WebGL sandbox site (http://glsl.heroku.com/)

‘basic.glsl’ – From http://glsl.heroku.com/e#2423.0

‘leds.glsl’ – From http://glsl.heroku.com/e#2450.0

‘raymarch.glsl’ – From http://glsl.heroku.com/e#2171.0

Pass the script any new filename, and it will create a new shader from the template and save it to that location.

This uses the “nano” text editor by default, as that is installed on the reference debian image, but if you look in the script, it is not hard to change to your preference

(NB Ctrl-O, followed by enter to save the file, and Ctrl-X to quit nano.)

Here is a video of it in action, as it is quite hard to describe in words.

by benosteen at May 13, 2012 02:10 PM

I have written before about some issues relating to RDA and RDF. Today I want to actually consider some things we should consider that should cause us to question the concept of "RDA in RDF."

For many decades we have been using relational databases to store our bibliographic data, bibliographic data that we create and exchange using the MARC format. Doing so was not by any means natural or intuitive because there is nothing about the structure or content of the MARC record that lends itself to being stored and managed in a relational database. The results were often awkward, inefficient, and unsatisfying.

Part of the reason for this is the unitary and flat nature of MARC. In spite of the long history of creating separate authority files, each MARC record is a complete and closed document with no actual connections to data outside of itself. While some database implementations for MARC do create relational tables for headings, the degree to which a MARC record can be separated out into tables is minimal and gains us very little in terms of the functionality of an RDBMS.

The underlying problem, however, is not in the structure of the MARC record but in the content of our catalog records. Moving from the card to a database for our data requires more than adding mark-up coding around the catalog data; to do so successfully requires re-thinking the data in terms of relational database principles. There are two basic principles to relational database design: repetition and combination.

To design for relational databases you look at your data to see what elements will be repeated in many different records. Rather than carrying those data elements in multiple records, you create a separate database table for each repeating element, and you store that element once. For example, if you are creating a database of mailing addresses, you see quickly that elements like state and zip code will appear in multiple records. You therefore create a table of state names and one of zip codes, and perhaps even one that links zip codes to city names. In this way, your database carries the string "Mississippi" only once, and that string is replaced in the records with a database pointer that uses much less internal storage. Ditto the zip code. And if the zip code is associated in a table with a city name, you also only store city names once, and each address record needs only a pointer to the zip code, not a city name. In fact, with a zip code you can get the city and state, and your design might look like:

by Karen Coyle (noreply@blogger.com) at May 13, 2012 11:42 AM

After yesterday's blog post, I thought I'd have a go at narrowing down my definition of a "separate search".

If a user enters some search terms, and then uses 2 facets to refine the search before clicking on a result, I was classing that as 3 separate searches — what niggled me overnight was that that approach might inflate the facet use statistics …after all, 30.6% of all searches used at least one facet felt a little high given that I'm forever hearing staff moan that students never use the facets, no?!

For today's blog post, I've removed all searches that didn't lead to a result click. (There's a small caveat that my jQuery code currently doesn't capture a result click for links to the OPAC where the user clicked on the availability message (highlighted in red below) — this is because my jQuery code that captures the result clicks runs once the page has loaded, but before the AJAX'd availability information has been retrieved. When I get some time, I'll see if I can find a way around that.)

So, let's see how much of a difference that makes to yesterday's stats…

• 29.4% of searches used at least one facet to refine the results
• 10.4% of searches were refined using the content type facet (e.g. newspaper articles, book reviews, books/ebooks, journal articles, etc)
• 7.8% of searches were refined to just items with full text available online
• 9.4% of searches were refined by publication date
• 5.6% of searches were refined to just articles from scholarly publications (including peer-review)
• 3.7% of searches were refined using the language facets
• 2.5% of searches were refined using the subject term facets
• 2.1% of searches used a Boolean operator, with nearly all of them being AND

So, that overall figure for the % of searches which used at least one facet hasn't dropped by much from yesterday's figure of 30.6%.

Anyone who follows me on Twitter will know that I like to cheekily mock the importance of Boolean and the data from the last 7 days reveals a few things:

1. no-one who used a Boolean NOT in their search clicked on a result
2. only 0.07% of searches (that's just 7 searches in every 10,000!) used a Boolean OR, which is arguably the most useful operator to use
3. unless you're using a search that includes one of the other Boolean operators, the use of AND is pretty much redundant as it's the default Boolean operator in a search (i.e. the search "dogs AND cats" is the same as "dogs cats")… so why are we telling students to use it in Summon?

After poking a bit of fun at someone for entering a 356 word search query yesterday, I can reveal that the longest search in the last 7 days that resulted in a result click was 98 words (it was a paragraph copied and pasted from a journal article).

I guess the big question here is why the disconnect between the "students don't use facets" mantra and the actual usage data?

Finally, I thought I'd figure out how many results are clicked on after a search…

by Dave Pattern at May 13, 2012 09:54 AM

I have an Acer Liquid E phone running Android. All of the photographs it takes are timestamped in the internal EXIF metadata to 8 December 2002. Turns out there's a bug in the camera app: it seems that instead of using "insert correct date here" in libcamera.so somehow the December 2002 date got hardcoded in.

The timestamp on the file itself is correct, however, so I wrote this script to use that to edit the EXIF times. It uses ExifTool, which is probably in your package manager:

#!/bin/bash

for I in 2002-12-08*jpg
do
  TIMESTAMP=`stat --printf "%y" "$I"`
  NAME=`echo "$I" | sed 's/2002-12-08 12.00.00-//'`
  NAME="acer-$NAME"
  echo "$I --> $NAME ($TIMESTAMP)"
  cp -p "$I" $NAME
  exiftool -P -DateTimeOriginal="$TIMESTAMP" -CreateDate="$TIMESTAMP" $NAME
done

For example, a file named 2002-12-08 12.00.00-460.jpg timestamped 2012-04-10 19:30 would have the DateTimeOriginal and CreateDate EXIF fields corrected to to 2012-04-10 19:30 and the file would be renamed to acer-460.jpg. The original file is left untouched.

It worked for me, and it won't delete your files, so use it if it helps. Make sure that whenever you copy files off your Acer phone you use cp -p to preserve the original timestamp. Otherwise your photos will have their internal dates set to today!

by wtd at May 12, 2012 06:26 PM

I discussed the utility of the sub-property relationship in Getting to higher MARC branches, Netting more MARC fruit, and Adding MARC fruit to the cornucopia. Coincidentally, Bob DuCharme posted Simple federated queries with RDF which outlines the same technique and provides additional information on its use for resource discovery. Those posts are somewhat technical, and I tried to lighten up in my presentation Turtle dreaming at the recent Dublin Core Metadata Initiative (DCMI) seminar Five years on. This post is another attempt to demonstrate in a non-technical way (I hope) how useful and powerful the sub-property relationship can be.

A metadata attribute, like ‘title’, that is to be used for linked data in the Semantic Web is usually represented in Resource Description Framework (RDF) as a property. A property can be used as the predicate part of a triple: “Subject – predicate – object”, where ‘Subject’ is what the triple is about (e.g. a resource), ‘predicate’ is the aspect of the subject, and ‘object’ is the value of that aspect for the specified subject. For example:

“This resource – (has) title – ‘Using the sub-property ladder’”

is a single metadata statement in triple format. We can think of this as conforming to the triple template:

“Specified resource – (has) attribute – value”.

Note that prefixing the predicate with ‘has’ turns it into a verbal phrase and renders the statement in (near) natural language.

We can also make meta-metadata statements in triple format. These are ‘data about metadata’ rather than ‘data about data’, and are often referred to as ontological triples to distinguish them from data triples such as the example above. The triple template for one type of meta-metadata statement is:

“Specified RDF element – (has) relationship – Other specified RDF element”

Note that a relationship between metadata elements is also represented in RDF as a property. In particular, ‘sub-property’ is a pre-defined relationship between two RDF property elements, giving the ontological triple:

“Property 1 – (is) sub-property of – Property 2″

Furthermore, such relationships can embed semantic rules that can be processed automatically by software known as ‘semantic reasoners’ or just plain ‘reasoners’. The rule embedded in the sub-property relationship is: If “P1 – (is) sub-property of – P2″, then any data triple using P1 as its predicate can generate another data triple using P2 as its predicate, with the same subject and object. Let’s call this kind of ontological triple a mapping triple, because it effectively maps one property to another.

Suppose we have two attributes ‘title’ and ‘varying form of title’. I can create the mapping triple:

“‘varying form of title’ – (is) sub-property of – ‘title’”.

If we have a data triple:

“This resource – (has) varying form of title – ‘Pat presents cataloguing for beginners’”

then a reasoner will automatically generate the data triple:

“This resource – (has) title – ‘Pat presents cataloguing for beginners’”

In a similar way, we can create the mapping triple:

“‘title statement’ – (is) sub-property of – ‘title’”

and from the data triple:

“This resource – (has) title statement – ‘Cataloguing for beginners’”

generate:

“This resource – (has) title – ‘Cataloguing for beginners’”

So what? Further suppose that the ‘title’ attribute is from the DCMI metadata terms, and the ‘varying form of title’ and ‘title statement’ attributes are from the MARC21 tags 245 and 246. So a MARC21 record for the resource might contain the set of data triples:

“This resource – (has) 245 [title statement] – ‘Cataloguing for beginners’”
“This resource – (has) 246 [varying form of title] – ‘Pat presents cataloguing for beginners’”

A reasoner can generate the set of data triples:

“This resource – (has) [DC] title – ‘Cataloguing for beginners’”
“This resource – (has) [DC] title – ‘Pat presents cataloguing for beginners’”

In other words, we have generated a DC record from a MARC21 record. Or we have generated a title index for the MARC21 record. Or both.

Let’s add an RDA attribute and an ISBD attribute mapping to the mix:

“[RDA] ‘title proper’ – (is) sub-property of – [DC] ‘title’”
“[ISBD] ‘has title proper’ – (is) sub-property of – [DC] ‘title’”

The data triples:

“That resource – (has) [RDA] title proper – ‘Cataloguing for geeks’”
“Another resource – [ISBD] has title proper – ‘Does cataloguing have a future?’”

can generate the corresponding DC triples, and we end up with:

“This resource – (has) [DC] title – ‘Cataloguing for beginners’”
“That resource – (has) [DC] title – ‘Cataloguing for geeks’”
“Another resource – (has) [DC] title – ‘Does cataloguing have a future?”
“This resource – (has) [DC] title – ‘Pat presents cataloguing for beginners’”

So now we have a title index to metadata from multiple heterogeneous sources. And the beginnings of a set of records in Dublin Core format.

Note that the attribute which is the sub-property must be entirely narrower in its semantics than the related super-property. If we create the mapping triple:

“‘title’ – (is) sub-property of – ‘varying form of title’”

then we generate the data triple:

“This resource - (has) 246 [varying form of title] – ‘Cataloguing for beginners’”

which is incorrect.

As a result, a data triple generated by a sub-property mapping triple is usually ‘dumber’ than the original data triple; detail is lost because the generated triple uses an attribute which is broader in meaning than the original. This ‘dumbing-up’ is necessary to produce interoperable metadata from different schemas – but data is not permanently lost because the original triple is still available for use in other applications. Needless to say, data triples created with broad attributes cannot be “smartened-down”, at least on their own.

The sub-property relationship can be chained. We can create a new attribute property, MARC21 ‘title’, which could be used in an application for making a title index to MARC21 records, as already mentioned. This new attribute is a super-property of all the MARC21 title-type attributes, and is also a sub-property of the DC ‘title’ attribute:

“[MARC21] ‘title statement’ – (is) sub-property of – [MARC21] ‘title’”
“[MARC21] ‘varying form of title’ – (is) sub-property of – [MARC21] ‘title’”
“[MARC21] ‘title’ – (is) sub-property of – [DC] ‘title’”

Doing this does not affect the previous mapping triples relating each MARC21 title-type attribute directly to the DC ‘title’ attribute, although it  makes them redundant because this new set of mapping triples generates exactly the same data triples at the DC level from the MARC21 originals.

Different application can therefore re-use and, if necessary, augment the sub-property chains for each of the high-level core attributes found in most bibliographic metadata schemas, such as title, author/creator/agent, subject, target audience, etc. The chains form a net(work) of mappings, or map, which can automatically dumb-up triples from any level of semantic granularity to any higher level.

We should only have to publish such maps or part-maps once, openly so that anyone can use them and add to them. If the professional communities develop the maps first, much effort will be saved and much authority imparted. This requires collaboration and action real soon now – the ISBD Review Group and the Joint Steering Committee for Development of RDA have started with the development of a mapping between the ISBD and RDA element sets.

These maps should remain valid forever, so the effort is worth expending. The original data triples use the original properties based on the schema attributes at the time and they will be valid “for their time”, in the same way that many catalogues are likely to contain records created under the Anglo-American Cataloguing Rules, with its ‘general material designation’ attribute long after the successor standard RDA: resource description and access has been adopted with its ‘content type’ and ‘carrier type’ attributes.

And mappings from the MARC21 element sets will show, we hope, that it may not be necessary to convert the entire contents of every MARC21 record as a result of the Bibliographic Framework Transition Initiative!

But the professional communities lack a framework to help them collaborate as a super-community. A network of mappings is more (socially) efficient than an aggregation of one-to-one mappings between pairs of schemas. We need (name)spaces to add intermediary attribute properties and publish the mappings; we need protocols for managing semantic change as schemas evolve; we need lightweight protocols for authorizing mappings; we need systems for ensuring the long-term preservation of RDF element sets and mapping triples.

by Gordon Dunsire at May 12, 2012 06:17 PM

[ update: slightly revised stats are available here! ]

We've just started collecting in-depth data about how students are searching Summon (keywords entered, facets selected, etc) and I thought some of you might be interested in an early analysis from the last 7 days (just under 40,000 separate searches by 2,807 students)…

• On average, students used 4.5 keywords per search (the mode is 3 keywords and the majority of searches used 3 keywords or less — view graph) [1]
• 30.6% of searches used at least one facet to refine the results [2]
• 11.7% of searches were refined using the content type facet (e.g. newspaper articles, book reviews, books/ebooks, journal articles, etc)
• 9.5% of searches were refined to just items with full text available online
• 9.2% of searches were refined by publication date [3]
• 7.2% of searches were refined to just articles from scholarly publications (including peer-review)
• 3.4% of searches were refined using the language facets [4]
• 2.6% of searches were refined using the subject term factes
• 2.3% of searches used a Boolean operator, with AND being by far the most common (2.23% of searches) [5]

notes:

[1] – One student copied & pasted the following 356 word title & abstract into the search box!

Peter J. Shaw, David J. Rawlins Article first published online 2 AUG 2011 DOI:10.1111/j.1365-2818.1991.tb03168.x 1991 Blackwell Science Ltd Issue Journal of Microscopy Volume 163, Issue 2, pages 151–165, August 1991 Additional Information(Show All) How to CiteAuthor InformationPublication History SEARCH Search Scope Search String Advanced >Saved Searches > ARTICLE TOOLS Get PDF (1119K) Save to My Profile E-mail Link to this Article Export Citation for this Article Get Citation Alerts Request Permissions Share Abstract References Cited By Get PDF (1119K) Keywords Confocal microscopy;three-dimensional fluorescence microscopy;point-spread function;deconvolution;computer image processing SUMMARY We have measured the point-spread function (PSF) for an MRC-500 confocal scanning laser microscope using subresolution fluorescent beads. PSFs were measured for two lenses of high numerical aperture—the Zeiss plan-neofluar 63 × water immersion and Leitz plan-apo 63 × oil immersion—at three different sizes of the confocal detector aperture. The measured PSFs are fairly symmetrical, both radially and axially. In particular there is considerably less axial asymmetry than has been demonstrated in measurements of conventional (non-confocal) PSFs. Measurements of the peak width at half-maximum peak height for the minimum detector aperture gave approximately 0·23 and 0·8 μm for the radial and axial resolution respectively (4·6 and 15·9 in dimensionless optical units). This increased to 0·38 and 1·5 μm (7·5 and 29·8 in dimensionless units) for the largest detector aperture examined. The resulting optical transfer functions (OTFs) were used in an iterative, constrained deconvolution procedure to process three-dimensional confocal data sets from a biological specimen—pea root cells labelled in situ with a fluorescent probe to ribosomal genes. The deconvolution significantly improved the clarity and contrast of the data. Furthermore, the loss in resolution produced by increasing the size of the detector aperture could be restored by the deconvolution procedure. Therefore for many biological specimens which are only weakly fluorescent it may be preferable to open the detector aperture to increase the strength of the detected signal, and thus the signal-to-noise ratio, and then to restore the resolution by deconvolution. Get PDF (1119K) More content like thisFind more content like this article Find more content written by Peter J. ShawDavid J. RawlinsAll Authors ABOUT USHELPCONTACT USA

…sadly, Summon failed to find a result for that as we don't subscribe to the article!

[2] – Normally, you search Summon by entering your keywords then, after the results appear, you select facets to refine your search and each facet selection invokes a new search. So, if you ran a search and then select 2 facets, that will be logged as 3 separate searches (1 without any facets, and 2 with).

[3] – Mostly, the publication date facet is being used to limit the search to the X most recent years.

[4] – The vast majority of the content in our Summon instance is in English and, apart from one search that refined the results to just Italian, every use of the language facet was to refine the results to English only.

[5] – Boolean operators have to be entered in UPPER CASE in Summon, with an invisible AND being implict in any multi keyword search that doesn't include Boolean. Looking at the searches queries that included a Boolean operator, 6% were entered entirely in upper case, implying that the user wasn't conciously invoking a Boolean search.

by Dave Pattern at May 12, 2012 01:24 PM

Equinox Software, Inc. is excited to announce the development of link checker functionality in Evergreen. Evergreen currently has no built-in mechanism for verifying the validity of URLs stored in MARC records. The ability to verify URLs will be of particular benefit to locations with large electronic resource collections. The requirements for this project are being developed in partnership with NRCan Library and Statistics Canada Library. The technical specifications for this project will be shared with the Evergreen Community once they are ready. Equinox developers estimate that coding will be completed no later than the end of the third quarter of 2012.

Once the coding is finished, the code will be submitted to launchpad, where another developer will need to review and approve it. Once it has been signed off on by another developer, it can be included in the next major release of Evergreen. End user documentation will also be made available to the Evergreen Community. For additional information, contact George Duimovich, NRCan Library, or Suzannah Lipscomb, Equinox Software.

Add to:

by slipscomb at May 11, 2012 07:08 PM

by jrcwiok at May 11, 2012 06:47 PM

by mindhiker at May 11, 2012 04:46 PM

I'd love to have a law named after me, so here goes:

Dave's Law

users should not have to become mini-librarians in order to use the library

If you ever find yourself needing to invoke Dave's Law, please let me know

by Dave Pattern at May 11, 2012 11:52 AM

Following the fun we had at March’s Meet-up ‘launch’, we will be having another gathering of people interested in open data next Wednesday 16th May. Hosted by the Wash Bar, Edinburgh, from 19.00, come and join us to discuss ideas, projects and plans in relation to openness.

Lightning Talks will include Federico Sangati on crowdsourcing and education, ahead of his presentation at Dev8ed later this month, and a sneak preview of the hackathon that Open Biblio will be running 12-14th June in collaboration with OKFN’s Open GLAM and Cultural Heritage Working Group and DevCSI.

If you would like to give a lightning talk (informal 2-3 minute presentations) about anything related to open data or knowledge, contact naomi.lillie [@] okfn.org.

Sign up here and we’ll see you there!



For this and other events in Edinburgh and the rest of Scotland, sign up here.

by Naomi Lillie at May 11, 2012 08:30 AM

This is the fifth and last in a series about using R to look at reference desk statistics recorded in LibStats. Previously:

• Ref desk 1: LibStats
• Ref desk 2: Questions asked per week at a branch
• Ref desk 3: Comparing question.type across branches
• Ref desk 4: Calculating hours of interactions

I've been making some other charts showing other kinds of ratios and calculations but I'm going to skip to one last pair of charts where I bring in the number of our students to figure out how many students we help with research help each week and for how long.

First, a brief review of the four branches of the York University Libraries system we're looking at:

• Scott is arts, humanities and social sciences, and the building includes the map library, the archives, and music/film library
• Bronfman is business
• Frost is on the Glendon campus in another part of the city and handles all of the students there
• Steacie is science, engineering and health

(Osgoode is law but they don't use LibStats so we'll forget about them for now.)

I calculated how many "home students" each library has. Bronfman handles everyone in the business school and in the administrative studies program in another faculty. Steacie handles everyone in the science and health faculties (except psychology, which is handled at Scott). Frost handles everyone at Glendon. Scott handles everyone else. The York University Factbook let me look up how many students were in each faculty, and I did a bit of adding and subtracting and figured out:

• Scott has 34,388 "home students"
• Bronfman has 6,050
• Frost has 2,677
• Steacie has 10,018

That's 53,133 students total, as of last fall. (We have about 43 librarians and archivists, for a ratio of 1235 students to each librarian, which is one of the worst in Canada.)

You can figure out something very similar for your library, probably.

With those numbers, we're all set for some more work in R.

First, I make a libstats.bigscott data frame, which gloms together all of the reference desk activities that happen in the Scott Library building (which as I said contains three smaller libraries) into one. This is necessary to group together all possible arts/humanities/social sciences questions. These lines below rename certain library.name fields by saying, for example for SMIL, for every entry in this data frame where library.name equals "SMIL", make library.name equal "Scott." Nice example of vector thinking in R.

> libstats.bigscott <- libstats
> libstats.bigscott$library.name[libstats.bigscott$library.name == "SMIL"] <- "Scott"
> libstats.bigscott$library.name[libstats.bigscott$library.name == "ASC"] <- "Scott"
> libstats.bigscott$library.name[libstats.bigscott$library.name == "Maps"] <- "Scott"
> libstats.bigscott$week <- as.Date(cut(as.Date(libstats.bigscott$timestamp, format="%m/%d/%Y %r"), "week", start.on.monday=TRUE))

Next, use our old friend ddply to count how many research questions are asked each week.

> research.users <- ddply(subset(libstats.bigscott,
                                 question.type %in% c("4. Strategy-Based", "5. Specialized")),
                          .(library.name, week), nrow)
> names(research.users)[3] <- "users"
> research.users$user.ratio <- NA
> head(research.users)
> library.name       week users user.ratio
1     Bronfman 2011-01-31    48         NA
2     Bronfman 2011-02-07    80         NA
3     Bronfman 2011-02-14    42         NA
4     Bronfman 2011-02-21    61         NA
5     Bronfman 2011-02-28    53         NA
6     Bronfman 2011-03-07    59         NA

Now, another probably heinous non-R way of dividing the number of users (or, actually, questions) each week by the number of "home students":

> for (i in 1:nrow(research.users)) { 
    if (research.users[i,1] == "Bronfman"          ) { research.users[i,4] = research.users[i,3] / 6050  }
    if (research.users[i,1] == "Frost"             ) { research.users[i,4] = research.users[i,3] / 2677  }
    if (research.users[i,1] == "Scott"             ) { research.users[i,4] = research.users[i,3] / 34388 }
    if (research.users[i,1] == "Steacie"           ) { research.users[i,4] = research.users[i,3] / 10018 }
  }
> library.name       week users  user.ratio
1     Bronfman 2011-01-31    48 0.007933884
2     Bronfman 2011-02-07    80 0.013223140
3     Bronfman 2011-02-14    42 0.006942149
4     Bronfman 2011-02-21    61 0.010082645
5     Bronfman 2011-02-28    53 0.008760331
6     Bronfman 2011-03-07    59 0.009752066

user.ratio there is what we're after. It looks low, doesn't it? Multiply it by 100 to get a percentage. It's still low.

The y-axis is per cent, so this shows that usually through term time we see give research help to under 1% of our students. There are a few weeks in some branches where it gets above that, but it's never above 1.5%.

That really surprised me. I have no idea what the numbers are like at other universities. If you figure it out for where you work, let me know. Perhaps one per cent is a common figure? Could it be five per cent at some universities? It would have to be a small university, I think, or have a lot of librarians.

Know that we know how many students we help with research, I wondered how long we spend helping them. More calculations in R, using ref.desk.spent, the function I defined in the last post to add up an estimate of how much time is spent at the desk. Here we break it down by branch by week, create a research.time.bigscott data frame, which I then merge with research.users so I can divide to create the research.mins.ratio which is what I'm after:

> research.time.bigscott <- data.frame(library.name = factor(), week = factor(), research.mins = numeric())
> branches <- c("Scott", "Frost", "Bronfman", "Steacie")
> for (i in 1:length(branches)) {
    branchname <- branches[i]
    for (j in 1:length(weeks)) {
      spent <- desk.time.spent(ddply(subset(libstats.bigscott,
                                            library.name == branchname & week==weeks[j] &
                                            question.type %in% c("4. Strategy-Based", "5. Specialized")),
                                     .(time.spent), nrow))
      rbind(research.time.bigscott,
            data.frame(library.name = branchname, week = weeks[j], research.mins = spent)) -> research.time.bigscott
    }
  }
> research.users$week <- as.factor(research.users$week) # Necessary for merge to work cleanly
> research.time.bigscott <- merge(research.time.bigscott, research.users, by=c("library.name", "week"))
> research.time.bigscott$research.mins.ratio <- research.time.bigscott$research.mins / research.time.bigscott$users
> head(research.time.bigscott)
  library.name       week research.mins users  user.ratio research.mins.ratio
1     Bronfman 2011-01-31           758    48 0.007933884            15.79167
2     Bronfman 2011-02-07          1340    80 0.013223140            16.75000
3     Bronfman 2011-02-14           595    42 0.006942149            14.16667
4     Bronfman 2011-02-21           997    61 0.010082645            16.34426
5     Bronfman 2011-02-28           775    53 0.008760331            14.62264
6     Bronfman 2011-03-07           901    59 0.009752066            15.27119
> xyplot(research.mins.ratio ~ as.Date(week) | library.name, data = research.time.bigscott,
         type = "h",
         ylab = "Length of average research interaction (minutes)",
         xlab = "Week",
         main = "Average length of research interactions (Scott includes ASC/Maps/SMIL)",
         sub = paste("From Feb 2011 to", up.to.week),
         abline=list(h=15, lty=3, col="lightgrey"),
        )

In this xyplot command I throw in an extra abline to draw a dashed light grey line along y=15 to help point out that generally we spend about fifteen minutes on each research interaction.

The Steacie library stands out from the others, and there are some peaks here and there, but overall we spend on average about fifteen minutes on each research interaction with students.

Put those two charts together and it shows that during term time we spend on average about fifteen minutes a week giving research help to each of under one per cent of our students.

by wtd at May 11, 2012 05:44 AM

(All of this is accurate, so far as I know, in Rails 3.2.3. If you are reading this later in future Rails versions, mileage may vary).

Rails 3 introduced plugins-as-gems, and the special case of Engines. An Engine is basically  a library of code that can define it’s own views, controllers, models, assets, etc, in it’s own codebase, that will be available for the Rails app. (An Engine doesn’t actually need to be defined in it’s own gem, it can be defined anywhere that ends up in the load path. but it’s own gem is typical).  You can have Rails generate a skeleton for an Engine plugin as gem, with `rails plugin new enginename --full`.  (Without the –full, it’d be a less powerful plugin without full Engine features — actually it ends up being pretty much just an ordinary gem).

A “plain” engine (as opposed to ‘isolated’ engine we’ll discuss later) basically “inserts” controllers, views, and models into the host app — they’re added to the load paths to part of the host app same as any locally defined controller, view or model.

Additionally, routes defined in your $engine/config/routes.rb will be automatically included in the host app. I’m not sure if they’ll be included before or after host app routes; route definition order matters in Rails3 routing.

Name collisions?

If there’s a name collision, the thing with the same name in the host app will usually ‘win’, and the one in the gem will be in accessible to most code (in gem or in host app).  If there’s name collision between two gems, it probably depends on load order (what order they’re referenced in the Gemfile, usually).

This is pretty much what you’d expect to happen, so long as the host app version really wins, I think it’s “right”.  (With helpers specifically, things can sometimes get confusing and not behave how you expect. I now can’t find the message I think I sent to the rails-user listserv on this at some point, and maybe it’s been changed/fixed in recent versions of rails.)

You can put your models, views, and controllers in module namespaces just exactly the same as you can if you were adding em to any Rails app, in order to try and prevent namespace collision. They’ll work just exactly the same way — the point of an Engine is the stuff in an engine is in the host apps load paths just the same as if it was really in the host app source locations.

Avoiding routing name collisions can be handled the same way, in a ‘plain’ engine, using the Rails3 router :namespace function, or any of the other related router functions (:as, :module, :path, etc.)

Some Engines handle routing by not including routes in $engine/config/routes.rb, where they’ll be automatically loaded by Rails, but instead loading routes into the host app using their own logic, so it can be done just so. This is especially useful for routes that should be changed by host app configuration. For instance, Devise and it’s `devise_for` method that the host app calls manually in it’s own routes.rb.

Isolated Engines: Rails 3.1

Rails 3.1 introduced the “isolate_namespace” directive, which you can add to your engine module.

The one main effect this has is actually on routing. $engine/config/routes.rb are not added to the host app’s routing.  Instead, Rails creates a little Rack mini-app out of your engine (or maybe any Engine already is this?), with your engine’s routing in it, so that host app can mount the Engine into the host app’s own routing, using the standard Rails routing ‘mount’ directive for Rack apps. See the Engines guide (or the edgeguide version, with slightly expanded information).

It also makes the engine’s $config/routes.rb behave a bit differnet as far as default routing params, assuming all routes are :namespace’d, making sure  the routing helper methods are available to your Engine’s controllers and helpers (and at the right method names), etc.

On top of this, it changes how rails generators work inside your engine. You can use rails generators inside an engine to add controllers and models. In a ‘plain’ engine, if you call `rails generate controller foo`, it’ll add an $engine/controllers/foo_controller.rb, just like any rails app.  It’ll add an `$engine/views/foo` directory and an `$engine/helpers/foo_helper.rb`. Just like an app.

In an Engine with `isolate_namespace`, if you call `rails generate controller foo`, it’ll namespace everything it generates for you:  `$engine/controllers/$enginename/foo_controller.rb` will contain a controller whose class is EngineName::Foo.  Similarly, view folder in `$engine/controllers/$enginename/foo`, etc.

Isolated engines are convenient for many cases.  You can have Rails generate a new skeleton for an isolated engine with `rails plugin new enginename --full --mountable`

There’s one aspect of them, though, that you may or may not want — and is fortunately pretty easy to change, giving you what I’ll call a Semi-Isolated Engine.

More Isolation Than you Might Want: Controller inheritance

There’s one aspect of isolated engines that ends up being a bit confusing — It’s actually not caused by the `isolate_namespace` directive in the Engine, but purely by the Rails generators — in fact, purely by the `--mountable` arg to `rails plugin new engine_name --full --mountable`.

Let’s look at how controller inheritance works.

If you use the `rails generator controller` to generate in your engine, if you look at it you’ll see that it’s defined as < ApplicationController — inheriting from the class called ApplicationController — just like a controller in a normal app. But your engine gem doesn’t have an ApplicationController (at least it ought not to, at least not a top-level-namespace ::ApplicationController) — what’s it inheriting?  Well, it’s inheriting from the ApplicationController in whatever host app it happens to be running in.

This means common logic in the host apps ApplicationController is available to engine controllers. (Say, a current_user? method; the engine would obviously need to document it’s conventions).  It also means all the helper methods loaded into the host app in a way that they apply to all controllers, will be available to engine controllers/views.  It also means that, by default, the default rails template layout for controllers in the engine is the host app’s `application` layout — or any other default layout specified in host app ApplicationController.

Sometimes that’s all actually nice, but sometimes you want more isolation. If you generate an engine with `rails plugin x --full --mountable`, you get it.  But how you get it is a bit confusing at first.

mountable/isolated generation of Engine::ApplicationController

If you generate a `mountable` (ie, isolated) engine, and then you use `rails g controller` to generate a controller, you’ll see it’s still defined as `< ApplicationController`. And yet it doesn’t actually inherit the behavior of the host app ApplicationController — it’s got no logic from host app ApplicationController, no helpers, won’t find it’s layout, etc.

What’s going on? It’s a different ApplicationController.  When you generate an engine with rails –full –mountable, it generates an EngineName::ApplicationController to $engine/controllers/$engine_name/application_controller.rb.

Because of the way Rails constant lookup works, it’s finding this ApplicationController.

And it generated a layout in your engine too at $engine/views/layouts/$engine_name/application.html.erb.

That’s the layout used by all your engine controllers, by default too.

multiple ApplicationController’s, really?

While this level of isolation is perhaps useful for many (most?) Engines, I question the decision to ‘override’ the ApplicationController class name and count on ruby constant-lookup in namespaces to get to the right one. ruby namespaced constant lookup is notoriously confusing, and changes from ruby version to version not always in documented ways.  I think it’s just asking for developer confusion and bugs.

Fortunately, it’s only a feature of the Rails generators (both the ‘rails plugin new‘ and `rails generate controller` within an isolated_namespace engine). Got nothing to do with actual rails runtime logic.

If you want to do it differently, no problem.  Go change $engine/controllers/application_controller.rb to, say, engine_name_controller.rb instead, and the layout to engine_name.html.erb.  All of your engine controllers should now “< EngineNameController” instead of “< ApplicationController“.

You’ve got the exact same behavior, just with less confusing and error prone names.

Sadly, `rails g controller` in an isolated_namespace engine will still generate “< ApplicationController“, you’ll have to manually change it each time you use the generator.

Now, for the Semi-Isolated Engine

Okay, now we can get to the actual point. While isolating controllers like this can be useful sometimes, sometimes it’s not. You might still want the routing isolation that “isolate_namespace” gives you, and the convenient change in behavior of the rails generators under that condition.

But you do want your engine controllers to inherit from the host app ApplicationController. No problem!  Just change that engine ‘main’ controller to “< ApplicationController”. You could do that even without the name change we discussed above, by properly scoping to top-level namespace, but that would lead to the confusing (but correct!) EngineName::ApplicationController < ::ApplicationController.

Less confusing if we changed the name as recommended above, say if your engine is the Widgetizer, Widgetizer::WidgetizerController < ApplicationController.

Now,

• any logic in the host app ApplicationController is available in engine controllers.
• Your engine controllers are by default using your engine’s ‘main’ layout instead of the host app’s — just delete the engine layout and they’re by default using the host app’s, that’s it!  (Delete $engine/app/views/layouts/widgetizer/application.html.erb, or $engine/app/views/layouts/widgetizer/widitizer.html.erb if you changed the names as recommended).
• If you have logic which you do want available to all engine controllers but shouldn’t be in teh host app, just add it to your intermediary engine main controller, right? Because SomeEngineController < EngineController < ApplicationController. (With Rails 3.2+ hierarhical view lookup, all views can be looked up through this chain, not just layouts).
• Because of isolate_namespace, the host app is still not automatically given the helpers in the engine — great! (If you want to manually expose engine helpers in the host app, see advice in the Engine Guide).
• Helpers in the engine are a bit more confusing. Since engine controllers subclass the host app ApplicationController, helpers from the host app areavailable in engine controllers. In some cases this is useful, in most others it probably won’t cause a problem.
  • If there is name collision between helper methods in host app and engine, when called from within an engine controller, the engine helper method ‘wins’. Which is great. (The engine helper can even call ‘super’ to get access to the host app version, although there are few cases where an engine helper could rely on ‘super’ existing.) However, this is reliant on details of how and in what order Rails include’s helper modules into controllers, something that’s changed in past rails versions, I’d be a bit cautious of relying on this continuing to work, sadly.

So there you have it, the “Semi-Isolated Rails Engine”, a design that works well for me for certain kinds of engines. It’s a testament to Rails 3.x nice, clean, flexible, consistent, well-designed architecture that we don’t need to fight with Rails actual runtime logic at all to do this, we don’t even need to change it, we just need to make different choices than the Rails engine generators make. If someone wanted to, they could even make their own generators that behaved this way for a ‘semi-isolated rails engine’.

by jrochkind at May 10, 2012 08:17 PM

Bibliomation, Inc., Connecticut’s largest library consortium, is sponsoring the integration of Syndetic Solutions by Bowker with Template Toolkit OPAC (TPAC) in Evergreen.  Equinox developers will be writing the code for this project.  TPAC will be able to support cover images, reviews, summaries, table of contents, excerpts, and author notes from Syndetic Solutions.  Once the code is written, it will be  submitted on launchpad, where another developer will need to review and approve it.  Once the code is signed off on by another developer, then it can be submitted for inclusion in the next major release of Evergreen.  For more information, contact Amy Terlaga at Bibliomation / terlaga@biblio.org or Suzannah Lipscomb at Equinox / slipscomb@esilibrary.com

Add to:

by slipscomb at May 10, 2012 05:26 PM

​ How do we measure social progress? Academics and international institutions have struggled with employing measures of human development which go beyond GDP per capita: education, health the the economy, but then what values do we attach to these?

In countries like Italy stark regional differences have dominated over time. Particularly in times of fiscal austerity when the country attempts to recover from an economic crisis with major social consequences, seeing how and why the South and the North differ is an important step in a consensus-building process to find solutions and realise collaboration with the citizens.

The Open Economics Working Group of the Open Knowledge Foundation released YourTopia Italia – an application which gives the users a chance to input their priorities in eight categories of socio-economic progress:

• Labour Market
• Education
• Health
• Environment and Energy
• Science and Research
• Household Income and Inequality
• Public Safety
• Social Life

Each category is comprised of sub-indicators e.g. Neighbourhood Safety, Income Inequality, Problems with Air Quality or Friends Networks. While the Northern regions fare rather well in most indicators, which are highly correlated with income per capita, Social Life seems to be better in the Italian South, where more people get married, fewer people separate and more people meet friends in their free time.

YourTopia Italia gives a chance to the user to adjust weights of their personal priorities and see how the map changes when some indicators are excluded altogether. A timeline visualisation also gives the perspective of how Italian regions have developed over time.

All YourTopias can be saved and shared through social media.

So, join our efforts: go to italia.yourtopia.net and define the YourTopia that reflects your vision of social progress!

The application was created with a dataset assembled from istat, and the source code of the application is released under an open license. This project is a result of a team work effort and follows up on ideas initiated during the Open Economics Hackday in January this year.

by Velichka Dimitrova at May 10, 2012 01:00 PM

Tomorrow, I will travel to Peru for the third BSLA review meeting. I’m looking forward to hearing what they’ve been up to (a lot!) and sharing what’s already come out of Botswana and Cameroon. One of the best things about this project has been how enthusiastically everyone has embraced working with colleagues in their own region, and across the world when we meet. I’m sorry to say it, but far, far too often at home I encounter that all-too-familiar syndrome: Not Invented Here. Every country, organisation, and many individuals want to put their own stamp on the profession, which is great, but at the same time that can lead to a lot of reinventing of the wheel and ignorance of resources that already exist. I’ll list just one institutional obvious example: study guides. I’ve been guilty of this too. Research, statistics, standards and training materials are other resources that also tend to be rewritten frequently. There is a barrier of research to practice. One way we’ve tried to help with that is by distilling research into practical case studies.

In countries with few resources, there’s a tendency by many librarians to work smarter, across borders and regions to get the information, specialists, and advice that’s needed to develop the profession and library services. A librarian may travel to Cameroon from Senegal to train on digitisation. A librarian in Cameroon may go to Angola to advise on LIS curricula. Regional associations provide a common meeting place. Certainly, it would be preferable to have enough resources in the country itself, and replicating projects is never so simple as just running the same thing again in a different place.

The recognition of the necessity to work together, across borders and sectors, building on where you are and what you already have, is something that we could all gain from.

by Fiona at May 10, 2012 12:33 PM

The W3C has published the Open Annotation Core Data Model.

Annotating, the act of creating associations between distinct pieces of information, is a pervasive activity online in many guises but lacks a structured approach. Web citizens make comments about online resources using either tools built in to the hosting web site, external web services, or the functionality of an annotation client. Comments about photos on Flickr, videos on YouTube, people's posts on Facebook, or mentions of resources on Twitter could all be considered as annotations associated with the resource being discussed. In addition, there a plethora of closed and proprietary web-based "sticky note" systems, and stand-alone multimedia annotation systems. The primary complaint about all of these systems is that the user created annotations cannot be shared or reused, due to a deliberate "lock-in" strategy within the environments where they were created, or at the very least the lack of a common approach to expressing the annotations.

The Open Annotation data model provides an extensible, interoperable framework for expressing annotations such that they can easily be shared between platforms, with sufficient richness of expression to satisfy complex requirements while remaining simple enough to also allow for the most common use cases, such as attaching a piece of text to a single web resource.

by David (noreply@blogger.com) at May 10, 2012 10:51 AM

If you are at all familiar with the open textbook world, you’ve likely heard of the startup called Boundless Learning. Leveraging information in the public domain, as well as dipping into the enormous stockpile of learning that is Open Education Resources, Boundless Learning has a created a tool that hopes to eventually replace the traditional textbook model.

Just like “open” anything, however, Boundless Learning has not gone without its fair share of trouble from vested industry interests. Recently, the textbook publishing giant Pearson, along with MacMillan and Cengage, filed a complaint alleging copyright infringement. Even though Boundless Learning culls its information from material available to the public through Creative Commons licensing, the publishers allege that “Defendant [Boundless Learning] exploits and profits from Plaintiffs’ successful textbooks by making and distributing the free “Boundless Version” of those books in the hopes that it can later monetize the user base that it draws to its Boundless Web site. In short, to build its business on Plaintiff’s intellectual property rights.”

Boundless Learning, on the other hand, claims that the accusations are patently false. The startup states that it only uses information already in the public domain, and said in a Boston.com article, “you can’t copyright facts and ideas. When you look at educational information, it’s primarily facts and ideas.” Boundless Learning will soon send out a legal response, and has expressed disappointment that the textbook publishers didn’t communicate with Boundless Learning amicably before resorting to litigation.

So what does this mean for the open textbook movement? Can we expect more lawsuits of this nature against innovative businesses? For one, Boundless Learning has truly launched a paradigm-shifting product. Most open textbooks are presented to students in PDF format using e-readers and other devices. However, Boundless Learning has extended beyond just digitizing traditional books by offering more. Their content is distinctly interactive, and students may build upon Boundless Learning material in a way that closely resembles both Facebook and Wikipedia. You can study along with other students, help each other in the learning process, and do it all online. For free.

Lawsuits of this sort aren’t anything new, and it’s important for those of us who are believers in the open textbook movement that we understand what we’ll have to fight against to live in a more open society. While Boundless Learning may have been careless in copying the format of copyrighted textbooks, down to the pagination, it does offer a platform that is new, that goes beyond mere open versions of closed textbooks. It’s with this innovative spirit that we can effectively, legally, and affordably, make information available to all. The world is not yet open, but we can get it there.

This guest post is contributed by Katheryn Rivas, who writes on the topics of online university. She welcomes your comments at her email Id: katherynrivas87@gmail.com.

by Katheryn Rivas at May 10, 2012 09:30 AM

Last Friday (May 4), the Ministry of Planning in Brazil launched the final version of the Brazilian Open Data Portal. In line with the federal government policy to promote the use of free software in public administration, the portal was made using only free and open source tools. Among them is the Open Knowledge Foundation’s open-source data portal software CKAN. Moreover, the whole process of development of the portal was conducted with the participation of concerned citizens in an open way to promote open data.

Opening Data Openly

The development project of the Brazilian Open Data Portal takes the concept of social participation to the extreme. From the beginning, planning meetings and development forums were open to any interested citizen, announced in advance on open discussion lists and where possible relayed via live streaming video to the Internet (webcast) .

In each planning meeting, the development tasks were selected by a flexible development process, in which people present ideas of what they think is needed in small ticket records. At the end of the round, the tickets are grouped, categorized and prioritized. At the end of the meeting, the events were recorded in a publicly accessible wiki (INDA wiki), and a publicly visible task manager (Trac).

We engaged the participation of people from civil society and of civil servants, who collaborated in various ways. Some people were involved right through the process, while others made contributions along the way. We had contributions in the form of software development, design, and information architecture, among others. The latter began with an experimental “card sorting” conducted with the participants of the event Campus Party 2012 in Sao Paulo. This synergy between government and citizens working together for the common good is what we mean by open government.

The Portal has gone through several versions, but the most important are the first (a simple HTML page with a tagcloud of catalogue data), followed by its beta, a little more prepared and documented, and then the current version with a new set of features and extensive reference material and learning.

The dados.gov.br now has 78 data sets with 849 resources. These have mostly been catalogued based on a survey of data that public bodies already publish on the Internet, but that until then were scattered and lacked a central access point where the public could find them. They are, however, the tip of the iceberg compared to what there is to be opened around public data in Brazil.

Recognizing this and the urgency in meeting the new law on access to information, the Secretariat for Logistics and Information Technology is preparing a workshop to guide public bodies, on how to include their data in the catalogue. This will take place in early June.

The portal is part of a larger project called the National Infrastructure Open Data – INDA. The general idea of ​​INDA is to establish technical standards for open data, promote training and support public bodies in the task of publishing open data. This entire process is done through intra-government cooperation and cooperation between government and citizens, always aiming to achieve a real platform for open government.

by Rufus Pollock at May 10, 2012 08:59 AM

One thing I can say about Kindle: error reporting is easier.

You report problems in context, by selecting the offending text. No need to explain where - just what the problem is.

Feedback receipt is confirmed, along with the next steps for how it will be used.

By contrast, to report problems to academic publishers, you often must fill out an elaborate form (e.g. Springer or Elsevier). Digging up contact information often requires going to another page (e.g. ACM.). Some make you *both* go to another page to leave feedback and then fill out a form (e.g. EBSCO). Do any academic publishers keep the context of what journal article or book chapter you’re reporting a problem with? (If so, I’ve never noticed!)

by jodi at May 09, 2012 09:45 PM

by gaps96 at May 09, 2012 09:15 PM

The Rails Guides are actually really good overview documentation. The days of saying Rails documentation is terrible are over, with the good guides, and good api-level docs too.

I knew there was a Plugin Guide, but I only just noticed there’s an Engine Guide too.

For reasons I don’t know, the Engines guide is not listed on the Guides home table of contents page, even though it’s available at guides.rubyonrails.org.   It also doesn’t google very well.

So here’s my part to publisize it. Both the Engines and Plugin guides are pretty good. They’re also overlapping in coverage. As the Engines guide says:

Engines are also closely related to plugins where the two share a common lib directory structure and are both generated using the rails plugin new generator. The difference being that an engine is considered a “full plugin” by Rails as indicated by the --full option that’s passed to the generator command, but this guide will refer to them simply as “engines” throughout. An engine can be a plugin, and a plugin can be an engine.

Maybe ideally they’d be merged, but they’re both good guides; you’ll probably want to review both if you’re working on Rails plugins OR engines.

Actually, you won’t find those exact words above in the current stable release of the Engines Guide, you’ll find it on edgeguides.rubyonrails.org instead.  I believe the actual guides are versioned with Rails releases — after one is released to guides.rubyonrails.org with the most recent rails version, it’s never changed (but perhaps for very serious errors), rather any changes will be released with the next Rails release.But you can preview em on edgeguides (I’m not sure if new ‘stable’ guides at guides.rubyonrails are pushed with new patch-releases of Rails, or just new minor-releases).

So you might find edgeguides contains some content that doesn’t apply to the current Rails release; but it may also contain improved explication or better examples or more coverage, as a result of contributors improving things. It’s worth checking edgeguides for a complicated topic. In this case, the Engines guide is rather improved in the ‘edge’ version as of this writing, and I don’t believe it includes anything that won’t work in Rails 3.2, it’s worth checking out.

A while ago I learned about `rails plugin new plugin_name`  command to give you a gem plugin  skeleton. Including a very useful dummy app for testing.  Before that I had been doing it by hand!   But this generates a very lightweight plugin, I was going in by hand and adding the files and methods turning into a heavyweight engine. I only just now, from the Engines Guide, noticed you can do `rails plugin new plugin_name -full` to get a fully engine-ized plugin.

Note on plugins ‘vs’. gems

It’s not a “vs.”.  Since Rails 3.0, Rails plugins could be distributed as gems. Since Rails 3.2, distributing a plugin as anything but a gem is deprecated — vendor/plugins is probably going to go away in future releases.

Although the great architecture of Rails 3 is such it wouldn’t be that hard to put it back into a particular app yourself, if you really needed to for legacy purposes. But in general, you don’t want, plugins-as-gems are great, making dependency management a lot more sane.

The current guides.rubyonrails.org plugin guide still gives you the option of a /vendor/plugin or a plugin-as-gem — it ought not to, since the current Rails version deprecates /vendor/plugin.  So don’t do that with a new plugin.

The edgeguide is more clear.  (I made the commit myself! Did you know anyone can commit to docrails github repo?  The commits are reviewed by editors before being merged into actual rails, and in the case of guides, eventually deployed to guides.rubyonrails).

by jrochkind at May 09, 2012 06:46 PM

by vedmonds at May 09, 2012 05:04 PM

I’m expecting you already know about the important Oracle v. Google court case over Android’s use of Java APIs, including both copyright and patent claims. But it would be hard to find a more detailed and direct account than Groklaw’s series of notes from the courtroom like this one from the copyright phase, ultimately subtitled Partial Verdict; Oracle Wins Nothing That Matters.  For the entire ongoing catalog, try this rabbit hole.

Having read direct courtroom reporting and the Court’s own documents, the headlines in some mainstream news outlets declaring Oracle the “winner” and Google “guilty” will start to look awfully remote and more than a little bizarre.  Google has moved for a new case since this jury was unable to determine whether their code constitutes a “fair use” of the Java API, so we might get to see the whole thing play out again.

More likely, the Court itself could deliver definitive resolution to the question whether APIs are copyrightable, particularly if the Court’s opinion converges its EU counterpart in a very recent case (Ars Technica via Google Cache, since the original is 404-ing for some reason).  One can hope.  The EU ruling was particularly bold because it protects reimplementation to the extent of voiding any agreements (read: EULAs) inhibiting that right.  That is a smart extra step in order to make the rights not immediately click-through disposable.

For more, follow along during the trial’s current patent phase at Groklaw.

by Joe Atzberger at May 09, 2012 04:34 PM

The Open Knowledge Foundation’s Open Biblio group, and Working Group on Open Data in Cultural Heritage, along with DevCSI, present BiblioHack: an open Hackathon to kick-start the summer months. From Wednesday 13th – Thursday 14th June, we’ll be meeting at Queen Mary, University of London, East London, and any budding hackers are welcome, along with anyone interested in opening up metadata and the open cause – this free event aims to bring together software developers, project managers, librarians and experts in the area of Open Bibliographic Data. A workshop will run alongside the coding on the 13th, and a meet-up on the evening of the 12th is open to all whether you’re attending the Hackathon or not.

What is BiblioHack?

BiblioHack will be two days of hacking and sharing ideas about open bibliographic metadata.

There will be opportunities to hack on open bibliographic datasets and experiment with new prototypes and tools. The focus will be on building things and improving existing systems that enable people and institutions to get the most of bibliographic data.

If you’re a non-coder there are sessions for you too. We will be running a hands-on workshop addressing the technical aspects to opening up cultural heritage data looking at best of breed open source tools for doing that, preparing your data for a hackathon and the best standards for storing and exposing your data to make it more easily re-used.

When and where?

• The main hackathon will take place over two days between 13th and 14th June at Queen Mary University of London
• On the morning of the 13th June we’ll be running the workshop addressed at the technical challenges to opening up metadata. So for those unable to participate in the hack due to time constraints or lack of coding know how – this is for you!
• On the 12th June – Tuesday evening (details TBC but will be a pub in central / east London!) – we’ll also be hosting a meet-up for anyone attending the hack and open data more generally. Whether it’s open bibliographic data, spending or government data that floats your boat all tribes are welcome!

Who is organising the event?

• Open Biblio – emphasising the utilisation of tools we’ve developed. We will be focussing on our existing tools and software, the most recent updates of which are available here: front-end BibServer / BibSoup, BibJSON, back-end BibServer
• Open Knowledge Foundation’s Working Group on Open Data in Cultural Heritage and Open GLAM
• DM2E – a new EU funded project devoted to enabling more cultural institutions integrate their collections into Europeana
• DevCSI – encouraging diverse thinking around software development and technical innovation

Who else is involved?

We’ve already lined up a whole host of speakers and groups who’ll be attending both the hack and the workshop. The list so far includes UK Discovery, CKAN, Europeana, Total Impact, Neontribe, The British Library with many more to be added in the coming days…

You’re giving your time and expertise – what do you get if you attend the whole hack?

• Accommodation at QMUL overnight on the 13th
• Food and drink across the 3 days
• The chance to work with experts in their fields
• Admiration and respect from your peers
• We could expound at length, but… go on, you know you want to (it’s free!)

How can I sign up?

• Register here for the 2 day hack
• Register here for workshop only
• Register here for Meet-up only

Please note, if you wish to attend all 3 events you should sign up for each, and the Workshop will run in parallel with the hacking on the morning of the 13th.

More questions?

Contact Naomi Lillie on admin [@] okfn.org.

See you there!

by Naomi Lillie at May 09, 2012 12:04 PM

by ptsefton at May 09, 2012 12:09 AM

Just a warning that some of this gets fairly technical, especially with hardware setup, and without the related diagrams, it may be difficult to understand, but the basics are there.

Presenters

• Marc Lalond
• Andrew McAlorum
• Graham Stewart

Evolution of the UTL Website

• recognized need for CMS back in 2003
• 2005 – used Plone
• 2008 – had to move frontpage out of CMS, which meant more maintenance
• 2010 – took another look at new CMS since current CMS needed a lot of Python knowledge and decided on Drupal
• 2011 – launched new site in Drupal

Drupal

• little coding work
• modules (much like WordPress plugins) that are available
• steep learning curve, but coding not necessary

Drupal Related Additions

• Drupal Commons – community distribution, pre-configured package
• Islandora – digital asset management, Fedora database backend
• Solr – well integrated into Drupal, especially for faceted searching

Implementation

• multi-site drupal allows multiple instances
• especially useful for simpler sites with little custom code and modules that are updated
• built custom Drupal distribution for UTL with all modules, theme, settings
• theme built on LayoutStudio starter theme
• next: responsive version

Training

• regularly schedule training, about once a month
• covers setup and config, users, content, etc.

Performance

• 2000 visits per hour
• single page load = 279 MySQL queries
• initial loads for 2000 page loads = 558,000 MySQL queries
• Drupal on one box: User <-> Apache Web Services <-> PHP <-> MySQL
• problem occurs when there is a bottle neck with a single point of failure
• Solution: horizontal scaling with multiple servers with Drupal functions split into smaller boxes
  • very flexible
  • less expensive
  • more adaptive
  • fully redundant

Setup

• individual servers are virtual machines, buil using KVM virtualization with Ubuntu Linux
• High availability with Keepalived
• Load balancing with HAProxy
• Caching using Varnish and XCache
• Storage on shared high performance disk
• MySQL query caching with Memcached and Keepalived
• MySQL master/slave replication + Keepalived

Results

• 5 minute downtime (planned and unplanned) between Sept 2011 and April 2012
• load time = < 2s on campus, 4.1 on simulated DSL in Virginia
• 100% Open Source

As I posted on twitter, I’m quite glad we don’t take care of our own hardware, especially since we just don’t have the people and resources (including not having any server admin), but I was quite impressed with the setup of the UTL Drupal setup. Quite interesting to hear what they’re doing.

by Cynthia at May 08, 2012 11:33 PM

Once again I bring you a gCal of the LITA-sponsored events at ALA Annual conference.

Also, I encourage you to go and check out the awesome ALA Annual Conference Scheduler and build your own custom calendar for the conference.

Please note that the LITA Happy Hour will be on Sunday 6/24/2012 from 5:30pm – 8:00pm (a change from our usual Friday evening, so I felt it should be highlighted)

by AaronDobbs at May 08, 2012 10:41 PM

Angela Hamilton, U of T Scarborough, spoke about technologies that she has used particularly at a campus where many are commuter or distance education students.

Libguides: Customized tools

• branding yourself for students to recognize you as their librarian: picture, meebo, contact info
• info on what is an article, database, annotated bibliography, etc.
• custom course guide
• use the tools available to you
• helps to build relationship with users

Online Meeting Software

• e.g. Adobe Connect
• for more advanced reference questions
• share screen – the “show-er” needs to install a plugin, but viewer doesn’t need to
• one-to-one, but also for teaching sessions

Screencapture Videos

• check vendors for already made videos e.g. ISI for Web of Science
• Jing (sp?) – free 5 min videos
• answer longer questions
• can also do it at the reference desk and e-mail it to them
• esp useful for non-techsavvy and ESL students to review later
• can also work for one-on-one session if have software for longer videos

I think some of the ideas presented here are great ways to give students further reference on how to do their research, especially on-the-spot screencasts for customized tutorials for them to review later.

by Cynthia at May 08, 2012 08:32 PM

This presentation actually not only talks about digital signage itself, but also the work culture change that happened in the systems department at UTL.

Presenters

• Sian Meikle
• Bilal Khalid
• Graham Stewart

Good Signs Can Make a Difference

• brief
• consistent
• easily read

Writing the Message

• simple
  • reduce: punctuation, pictures, words
  • headline: 22 characters
  • body: 10-18 words
  • short URLs
• brief
  • 5 seconds per slide
  • 8-10 seconds total
  • usually less is more
• clear
  • call to action e.g. Chat with a librarian
  • photographs can be powerful
  • coherent design

I don’t know that I agree with all of these, but then it was clear that it depends on the size and distance of the sign as well as where it is.

Presenting the Message

• Chunking
  • group things together
• Coding
  • add headings
• position
  • prime spots on a list: first and last get noticed the most

What Makes Digital Signage Different?

• easy to update
• can differentiate content by
  • time of day
  • audience
  • viewing time

What Users Say

1. Help me make better decisions
   • chat with a librarian, workshops
2. Save me time
   • maps: library, stacks, workstations
   • directories: by floor, service, name, library
3. Show me something relevant to me
   • news, community content
4. Tell me something new and interesting
   • exhibitis, events, news
5. Give me ideas
   • collection highlights

This is not what their actual users were saying. These ideas were based on a talk done by someone outside of the library and the list here is how those ideas might be applied in a library setting.

Touchscreen Kiosks

• PHP – CodeIgniter
• jQuery
• MySQL
• Closed Environment – not open to the Internet
• Javascript Keyboard

Interaction

• Most Frequent Pageviews
  • since May 2011
  • Libraries & Hours
  • Robarts Directory
  • Workstations
• User Feedback
  • Let me find a book
  • Let me access this information from my phone

What’s Next

• catalogue search
  • entire catalogue available
• StackMap
  • map of physical item location, with directions
• Responsible Design
  • designed to be used on any device

This is interesting, because we’re working on something similar at our library and we were considering how responsive to make the site. Obviously, we need to seriously consider designing from desktop down to mobile.

Overhead Signage

• 4 vertical screens
• PHP + AJAX
• Media Commons
• JavaScript video player
• Fishers Rare Book
• screensaver

Features

• auto refresh
• detection of new content
• remote control
• ability to have different slideshows
• control to switch between slideshows
• control through phone

What’s Next

• Scala software across all overhead screens
  • content regions e.g. time at bottom of screen
  • RSS Feeds to Drupal based on another content type
  • Scheduling e.g. times of day

Building Directories

• one PHP + JavaScript page per vertical pylon (two vertical screens)
• alternating event feed display (from Drupal, via AJAX)

Development – Devops Movement

• focus on increase collaboration and cooperation
• agile methodology applied to system administration
• agile development and teams (self organizing, cross functional, quick daily meetings, open environments, face to face meetings, encourage input)

On System Administration

• timeframes all shrink
• web presence critical
• software is developed much faster and changes are more frequent
• massive growth in automation tools
• growth in OSS: sharing and collaboration

Devops Goals

1. Eliminate stereotypes
   • developers are careless, arrogant while sysadmins always say no and work all night
2. Increase communicatin between developers, operations, and management
3. Continuous systems improvement
4. Break down barriers and silos
5. Develop methods to encourage all team members to see the organization’s goals

Advantages

• all staff use all their skills
  • diversity
  • use knowledge outside defined roles
  • roles expand
  • cross pollination
  • creativity
  • “many minds”
• enhanced mutual respect and communication
  • greater trust
  • shared responsibility
  • everyone feels a sense of ownership over the end product
  • greater commitment to the product
  • everyone focused on the organization’s end goal
• happier, move productive staff

Implementing DevOps With Digital Signange

• operations and development involved jointly from the start
• weekly full meetings and as necessary (often daily) with quick interrupts/one-on-ones for specific issues
• fast code releases: several times/week
• “many minds”
• two screen display: one browser? 2 PCs?
• disabling right click
• URL shortening
• Planning and execution
  • browser choice
  • OS choice
  • development options
  • design decisions
• New and experimental project
  • innovative methods required

I thought it was interesting that they spoke a lot about the more technical aspect as well as development methodology. I think it’s a good lesson for a lot of library IT departments that agile development with integrated back and front end staff can be very beneficial, particularly because it makes more development faster and more flexible.

One of the things that came up during the code4lib conference too is that developers should have a small amount of time to work on whatever seems interesting to develop new tools or services.

by Cynthia at May 08, 2012 08:19 PM

What other sites share the same infrastructure with your site, or any other? Bing‘s IP search can answer. Do a search by IP number:

• ip:72.233.127.217
• ip:158.136.1.105
• ip:72.51.52.15

by Casey Bisson at May 08, 2012 07:39 PM

We now know much more about the Google Street View WiFi story, thanks to Google’s decision to release an unredacted version of the FCC report, to the New York Times’s identification of the Google employee involved as Marius Milner, and to further reporting from Ars Technica. The picture it paints is in some respects more flattering to Google, and in some respects worse.

Milner is the creator of NetStumbler, a tool for detecting and analyzing WiFi access points. It makes sense in hindsight that he ended up using his 20% time for the part of the Street View project that aimed to build a database of WiFi networks. And it turns out that he thought about the ethics and legality of recording payload data. He appears to have read some law-review scholarship on wardriving. He considered potential privacy issues, and concluded that the mobility of the Street View cars would minimize the risk of extensive data-gathering from any one user. Further, he emphasized that none of the data would be shared with Google users.

This is, I have to say, above the baseline of ethical cognition for programmers. Looking to legal scholarship at all is quite unusual. In fact, Milner’s thoughtfulness strikes me as roughly par for the course for front-line Google technologists. It’s a company that hires reasonably thoughtful people and encourages them to think about the implications of what they do for society, both good and bad.

But if Google is a company of smart, reflective, and well-intended individuals, collectively they make bad choices. Milner put his privacy concerns and the details of the WiFi payload recording in a design document. The document included a “to do”: “[D]iscuss privacy considerations with Product Counsel.” He talked to a member of the search quality team about the idea; he circulated the design document together with his code to Street View’s project leaders, who forwarded it to the entire Street View team. And he exchanged emails with other Street View programmers and managers that made clear Google was collecting payload data. But nothing happened. For fifteen months, Google Street View cars sucked up and recorded WiFi payload data.

As I said in an earlier post:

When it comes to privacy, this is a company out of control. Google’s management is literally not in control of the company.

Google’s Street View managers failed badly at their jobs. One of them “pre-approved” the design document before it was written, demonstrating complete failure to understand the purpose of managerial review. No one followed up to make sure the discussion with Product Counsel actually happened. Other engineers read the design document and Milner’s code, but either missed the fact that it was collecting payload data or didn’t realize that this could be a potential issue. Again, this is a failure of management: it’s an important part of their job to make programmers aware of the possible legal trouble zones in the areas they’re working on.

Milner has invoked the Fifth and isn’t talking to reporters. He made a mistake, but he’s not a legal expert and it’s a bit unfair to expect him to be. No, his managers let him—and the rest of us—down.

by James Grimmelmann (james@grimmelmann.net) at May 08, 2012 05:09 PM

So it turns out there’s a significant typo, that keeps the algorithm from working right, in the several previously blogged descriptions of reddit’s story-ranking algorithm.

update 6:28PM ET 8 May On reddit, someone with a flag suggesting they ought to know tells me I got it wrong and the original algorithm was correct and is used in production. All I can say is I can’t figure out how that could be, I could not get it to work in a non-reddit codebase, I could get it to work with my ammendment.

If I haven’t corrected a typo, then I guess I’ve derived my own variation which works a lot better for me (that is, works at all for me, but also seems to mimic reddit), in my own codebase. Good enough for me. If you are trying to reimplement this algorithm in a non-reddit codebase, I suspect you’ll find my investigation useful.

Now back to the blog post as originally written.

More oddly, this same significant typo is in the public version of reddit’s code released on github.

I’ve found myself finding joy in code-for-code’s-sake like I haven’t since past days of being an undergrad staying up all the night in the CS lab working on the most fun homework ever. And so I found myself staying up into the wee hours last night investigating reddit’s story ranking algorithm (the one used for stories/posts in the default ‘hot’ ranking,  that is time-of-posting sensitive. A different algorithm is used for comments).

The wrong algorithm

The (typo’d) algorithm is most nicely described by Amir Salihefendic. He even provides a python implementation. I figured I’d translate it to my preferred comfortable language ruby, and play around with it changing different parameters to get a feel for it, and get a feel of how it might be modified/tuned to behave somewhat differently if one wanted.

My assumption was that this algorithm outputs a number which can be stored in the database, and stories can be ordered purely by this number, to produce the on-page ranking.   This seems indeed to be true, although I was doubting it a bit in the middle. (Another nice thing about this particular algorithm, that everyone did catch on to even in the typo’d version, is that a ranking order calculation only needs to be done when a ‘vote’ happens, then it can be stored in the db unchanging forever (until another vote happens)).

So I translated Amir’s python to ruby and starting playing, and the results made no sense.  They didn’t match how things work on reddit, and they didn’t result in any kind of useful ranking algorithm.

Users of reddit know that the story listings are mostly chronological. The vote count will change a story’s position somewhat, but not put it dozens of weeks or pages ahead or behind of it’s strictly chronological order.  But that’s what this algorithm did.  It also gave any story with a net-negative vote a negative score. Which would put all the net-positive vote stories before any of the net-negative voted stories. Which is not how reddit works.

Looking at the math again now, I’m kind of embaressed I didn’t immediately see the problem, it’s not complicated math. But I didn’t, it just made my head swim. I’ll give you the relevant line from Amir’s python version here, maybe you can do better than I did, now that I’ve primed you:

return round(order + sign * seconds / 45000, 7)

Before I give you the answer, I’m going to tell you all the things I did first:

• Went back and from scratch re-ported Amir’s python to ruby, doing as literal a translation as possible. Same nonsensical result.
• Took the more mathematical description on Amir’s page (the one in a giant png? That he took from semoz?), and implemented that in ruby. Same nonsensical results (modulo some new bugs I introduced).
• Googled around for anyone elses description of the algorithm to see if they had a different version, or explained it better, or explained how it fit into the context of the software as a whole (maybe I was wrong that it was supposed to produce the actual bare ordering number?). No dice.
• Found the relevant source in the reddit github open repository (Amir’s link was broken as reddit moved their public source repo. Hooray github for being so easy to navigate on the web). Translated this to ruby. Same results.
• Okay, noted that the reddit source mentions an “equivalent function in
  postgres”  Okay, found the SQL implementation, translated THAT to ruby, same results.

(Yep, the implementation in github public reddit has the typo, and is wrong!).

At this point, I gave up on understanding the reddit algorithm, I figured there was something I was missing (wrong, only thing I was missing was the typo). But, okay, I dove back into the math, trying to understand it and convert it to something that would work for me.

Take a moment to note lesson learned

Like many programmers, I rather like working from fixed assumptions and constraints, and building on top. This is kind of the nature of abstraction, don’t question the lower levels, take em as assumptions, don’t question em,  build upon them.

This is the second time recently that’s led me astray into butting my head against a dead end wall repeatedly, assuming the problem was in my own implementation or understanding, instead of in the framework code I was using, or the published algorithm or explanation I was working from.

Sometimes you’ve got to start questioning the validity of the algorithm you’re working from or the correctness of the library/framework code you’re using sooner rather than later, to save yourself some time. However, do it privately, if you start questioning your dependencies in public without evidence, everyone’s probably (rightly) going to tell you “occam’s razor, the bug is probably in your code, not the dependency.”

The Fix

WRONG:  return round(order + sign * seconds / 45000, 7)
RIGHT:  return round(  (order * sign) + (seconds / 45000),   7)

Is it obvious now that you see it, that the first one makes no sense, but the second one does?  Maybe if you see it in context, here’s my ruby implementation of the corrected algorithm. 

I feel kind of stupid  for not noticing this right away; on the other hand, as far as I can find on google, nobody’s pointed out the typo bug before, and several have commented on the (wrong, typo buggy) algorithm.

The Explanation of the Algorithm

With the typo corrected, it’s much easier to explain the algorithm. The crucial line, from my ruby version, with variables named how I think is clearer:

return (displacement * sign.to_f) + ( epoch_seconds(date) / 45000 )

It plots each story on a fixed timeline by post, and then displaces a story on the timeline by it’s votes.  It uses only the vote difference between up and down for displacement, the total number of votes is irrelevant. First:

... + ( epoch_seconds(date) / 45000 )

This just plots each story on a fixed timeline, with distance between two stories always exactly proportional to difference between absolute posting time.

The `/45000` fixes the units of the timeline as “12 hour periods” (45000 seconds in 12 hours), rather than seconds. This reduces the order of magnitude of the units by 4.5ish, making them conveniently less likely to overflow wherever you’re keeping them. But more importantly, choosing the units matters for how much displacement the actual votes will cause, making sure they match appropriately. Then:

(displacement * sign.to_f) + ....

Here’s our displacement. `displacement` is the based on the vote difference (up – down), but on a logarithmic scale.  The way the logarithmic scale is calculated, it loses the sign, so it just has to be added back in to net-down-votes will displace the story to be older on the timeline, and net-up-votes will displace the story to be newer on the timeline.

Why is a logarithmic scale used? Other explainers have said something like “to weight the first votes higher than the rest.”  While it might have this effect because of reddit voter’s behavior, this is a misleading explanation.  The algorithm pays no attention to which votes were made first, either in absolute chronological time or in sequence. It’s just vote-difference.  ”10 up, 1 down” has exactly the same effect as “100 up, 91 down” or “1000 up, 991 down”.  And it doesn’t matter what order the ups and downs were placed.

The logarithmic scale is in fact used to prevent the displacement-from-voting from displacing the display order too much.  Reddit doesn’t want a very high or low voted story to be months ahead or behind, the reddit ‘hot’ order is mostly chronological, with just some displacement from votes.

I dont’ do this kind of mathematical analysis much, and don’t know how to get, say, R, to make you a pretty plot (it ought to be an actual function plot not a bar graph, for explanatory power). So I’ll just give you some samples of how much displacement a given vote-diff can get. Again vote-diff is just ups minus downs, doens’t matter total number of votes. I’ve converted from the “12-hour units” the displacement is actually expressed in to more comprehensible ‘in hours’ units.

As you see, even something with an absurdly high 10000 vote-diff only gets put 50 hours ahead of it’s usual place in the timeline. Likewise, if it had a -10000 vote-diff (10k more downvotes than upvotes), it would be only 50 hours behind it’s usual place in the timeline.

Keeps votes from changing the position of a story too much, keeping it at the top forever, or moving it so many pages in that nobody ever sees it.  That’s what the log scale does.

And that scale pretty well matches what we reddit users actually observe on reddit, I went and checked it against some popular reddits; reddit only displays approximate posting time of a story as far as I can tell (“1 day ago” could mean 28 hours or 32 hours or whatever), so can’t check completely, but the actual ordering could be explained by the corrected algorithm.

Wrong in the public source on github?

update 6:55pm ET 8 May 2012.  reddit assures me that this code is what reddit runs live, and I have made some really stupid mistake. Fair enough. Struck out this section.

Unless I’m making some really stupid mistake, this typo-bug is present in the reddit source publicly shared on github as of time of this writing. [1], [2]

This means that there’s pretty much no way actual reddit.com is using the code they’ve posted publicly on github. At the very least, they’ve fixed this bug in the implementation they’re actually running.

It probably means nobody else is using the reddit github source either, cause it wouldn’t work right with ‘hot’ ranking. (Or someone else is using it, and fixed the bug in their source but didn’t send it back).

How did this bug end up in the publicly shared reddit source? Not fixed yet? I’m kind of curious, and curious as to what relationship this publicly shared source has to what reddit actually runs.

Considering tweaking the algorithm

Now that we understand the basic “timeline + displacement” algorithm, we can consider tweaks/modifications/tuning of the algorithm to behave differently in different environments, which curiosity was my original motivation for looking into this in the first place.

You might want vote displacement to have more of an effect, or have the effect trail off faster or slower . You’d still want to use a log-scale (or a mathematical function with similar properties) to keep very high vote-diffs from displacing a story too much, you still want a trail-off effect.

You could change the log from base-10 to base-something else to effect the velocity of the trail-off effect.  You could also introduce a factor into the operand of the log, take `log( factor * vote-diff)` instead of just `log(vote-diff)`.   You potentially could change the units from 12-hour units to something else (the 45000 number), but that could get confusing quick, you might need to add another factor on the left hand of the sum to compensate. So actually, instead, you can just add a factor in the left-hand side, `factor * log(votediff)` instead of just `log (votediff)`

I’m not enough of a math guy to predict exactly what all those things would do, I’d want to actually plot the function in R (or something else) and see what it looks like when you change the various factors, and I don’t know enough R (or anything else) to do it. I think plotting vote-diff vs hours-of-displacement is the right thing to plot though to give you the right feedback.

You could also try to introduce something to the equation to take account of total number of votes, so “10 up, 1 down” and “100 up, 91 down” don’t have exactly the same effect. You’d want to base this on the Wilson score confidence interval used by reddit for default comment ranking somehow, that’s the right way to take account of total number of votes, but it’s not immediately clear to me where you’d introduce that into the equation how (Did I mention I’m not a math guy?).  That would make it a bit harder to see what it does by plotting it, since it’ll be a multi-variate function now, doh.

And you might not want to trust that the algorithm found in reddit’s public github source for Wilson score confidence interval is actually bug free. Last year someone said they found a bug in at least one published implementation; I think I saw someone say it had been fixed on reddit.com, but I don’t know if it’s been fixed in the github public source.

You might also want to make up votes worth more or less than downvotes, instead of equivalent. Not quite sure how you’d do that. You could make net-negative votes worth more or less than the same absolute value net-positive, just by using a factor in `factor * log(diff)` that depends on diff being positive or negative.

by jrochkind at May 08, 2012 04:18 PM