You are here

Feed aggregator

LITA: 2015 Election Slate

planet code4lib - Wed, 2015-02-25 16:10

The LITA Board is pleased to announce the following slate of candidates for the 2015 spring election as follows:

Candidates for Vice-President/President-elect

  • Nancy Coylar
  • Aimee Fifarek

Candidates for Directors at Large, 2 elected for 3 year terms

  • Frank Cervone
  • Martin Kalfatovic
  • Susan Sharpless Smith
  • Ken Varnum

See candidate bios and statements for more information; voting in the 2015 ALA election will begin at 9 a.m. Central Time on March 24, 2015. Ballots will close at 11:59 p.m. Central Time on May 1. Election results will be announced on May 8. Check here for information about the general ALA election

The slate was recommended by the Nominating Committee. Karen G. Schneider is chair of the committee and Pat Ensor, Adriene Lim, and Chris Evjy are the committee members. The Board thanks the Nominating Committee for all their work. Be sure to thank these candidates for agreeing to serve, and the Nominating Committee for developing the slate. Best wishes to all.

District Dispatch: Gearing up for South by Southwest

planet code4lib - Wed, 2015-02-25 15:42

“What the Future?” is not the usual interpretation of its acronym, but the question is at the heart of the upcoming South by Southwest (SXSW) Interactive and EDU technology conferences.

Participant enthusiasm is evident in “Libraries: The Ultimate Playground,”one of the Core Conversations that have become a hallmark of SXSW.

One answer to this question for public libraries, entrepreneurs and our communities is “Co-working, Collaboration and Creation @ your library.” This also is the title of our SXSW panel (Sunday, March 15, 9:30 a.m.) which brings together ALA’s national perspective and research, a dynamic example from the District of Columbia Public Library’s Dream Lab, and one of the “lab” partners, MapStory.

Public libraries have always been places for “makers” to connect and collaborate, but HOW this is happening in libraries continues to shift and expand. Hundreds of libraries now support co-work and mobile work spaces, as well as hosting maker programming and resources—together leveraging tech and social networks, specialized content and staff, and convenient locations, according to new data from the national Digital Inclusion Survey.

Starting in mid-2013, the Martin Luther King Jr. Memorial Library in Washington D.C. embarked on a journey to transform itself into a place for residents of diverse backgrounds and interests to connect, learn and “make” ideas into working realities. The space is called the “Dream Lab,” and the library already has attracted more than 56 local entrepreneurial projects. One of these projects is MapStory, an online social cartographic platform that empowers anyone to map historical change over time using open data.

In exchange for membership in the Dream Lab, each project commits to offering one hour of programming per month to engage the public in their efforts and ideas—extending the social network and empowering the community in a transformative way.

Through these collaborations, libraries serve as catalysts for learning and action, and help build a stronger knowledge economy more accessible to all people. I can’t wait to join Jon Marino and Nicholas Kerelchuk to talk about how we can build the future together with enterprising partners.

I’m also excited to again be part of a larger Libraries, Archives and Museums presence at SXSW. Together we believe that SXSW is a key forum for library professionals to position libraries and connect with other innovators and community connectors; pull ideas from other industries to feed our innovation at the intersection of science, technology, art, commerce and the public good; and to participate in emerging policy conversations directly relevant to our professional values. Check out the lib*interactive at SXSW guide to sessions here.

You can also catch school and public library folks presenting at EDU (March 9-12), including “Schools and Libraries: Rethinking Learning Together”, “Designing and Deploying for Adult Learners”, “Schools’ Vortex: Innovative Library Makerspaces”, and  “Connected Learning Networks in Austin.”

I hope I’ll see many of you there! If you can’t make it to SXSW, let me take your story with me. What is your library doing to support co-working, collaboration and/or creation? Comment here on the blog or email me directly at lclark@alawash.org.

The post Gearing up for South by Southwest appeared first on District Dispatch.

In the Library, With the Lead Pipe: Beyond the Threshold: Conformity, Resistance, and the ACRL Information Literacy Framework for Higher Education

planet code4lib - Wed, 2015-02-25 13:30

Photo by Flickr user laroyo (CC BY-NC 2.0)

In Brief: The recently adopted ACRL Framework for Information Literacy for Higher Education has generated much critique and discussion, including many important reflections on the nature of information literacy and librarianship itself. This article provides a brief consideration of some of these responses and as well a critique of the Framework from the perspective of critical information literacy. It argues that although the Framework demonstrably opens up possibilities for an information literacy instruction that encourages students to question the power structures and relations that shape information production and consumption, it nonetheless rests on a theoretical foundation at odds with that goal. It urges librarians to embrace the Framework yet also resist it, in the tradition of critical librarians who have practiced resistance to the instrumentalization of the library for neoliberal ends.

The ACRL Framework: Off to a Running Start

By design and in its implementation, the process of drafting the ACRL Information Literacy Framework for Higher Education invited intense scrutiny and critique. The Task Force charged with creating the Framework began its work in March 2013 and released the first draft for public scrutiny in February of the following year.1 Through three drafts to the final version, which was made public in January 2015 and officially “filed” (approved by the ACRL Board of Directors) shortly thereafter, the effort toward transparency included many opportunities for input that will help the Framework earn a strong measure of democratic consent and broad participation. But a successful launch and general adoption are by no means assured, as the resistance to plans to scrap the existing Standards has been and may continue to be strong.2 Less easy for the Task Force to control or to even keep track of, however, were reactions on blogs, Twitter, other social media, and in informal conversations, and these arenas continue to produce new and sometimes unexpected reactions to the proposed Framework.

It should have been expected that the lively debates among librarians have included searching, systematic, and thoroughgoing critiques of both the fundamental assumptions and the theory underlying the Framework and even its reason for existing at all.3 The draft process has provided an opportunity for many people to talk about the meaning and purpose of information literacy instruction, information literacy in general, and even librarianship itself. As Barbara Fister stated in one of her early commentaries on the Framework, “it is an opportunity for us to rethink how we do this and what kind of learning really matters.” (Fister 2014a) I consider the Framework already to be a success because the debate it has generated contributes to our thinking about and practice of librarianship in invigorating, productive, and necessary ways.

In this article I will review and compare some of the critiques of the Framework voiced thus far. I will also offer a critique of my own that attempts to read the Framework from the perspective of critical information literacy and critical librarianship. Librarians who identify with these labels, generally speaking, seek to anchor information literacy practice and librarianship as a whole to a commitment to both principles of social justice and a systematic critique of the power relations within which our field operates.4 But first it is important to cite some of the ample evidence that a wide range of reflective librarians are embracing aspects of the Framework and running with it, with exciting results and prospects. This has happened because of the Framework’s flexibility. Troy Swanson, a member of the Task Force, points out that the Framework “can enable us to get to real student learning because it can be adapted to align with your goals as a teacher.” (Swanson 2015) Megan Oakleaf has echoed this sentiment, stating: “Essentially, librarians can use the Framework as inspiration to focus on concepts, rather than exclusively on tools and techniques, and those concepts can be added or subtracted as student and faculty needs change.” (Oakleaf 2014)  Andy Burkhart has described a library lesson for an ethnography class for which he utilized the threshold concept “Research as Inquiry.” His conclusion is that “Using these threshold concepts may not work for everyone, but I can see them being exceedingly helpful to frame lessons and curricula. They help you focus on what is really important as opposed to getting stuck in what you think you are supposed to be teaching. Instead of just teaching a lesson about doing ethnographic research I taught a lesson about inquiry and asking increasingly sophisticated questions.” (Burkhart, 2014) Arguing along similar lines more recently, Lori Townsend, Silvia Lu, Amy Hofer, and Korey Brunetti have defended threshold concepts for being flexible and versatile while still rooted in common problem areas specific to academic disciplines, even while allowing that the specific threshold concepts in the Framework are necessarily only provisional and may not work for everyone: “If you consider your content with the threshold concepts criteria in mind, it helps identify some things that might prove problematic for students and stall their learning, yet that are needed in order to move forward in their understanding.” (Townsend, et al 2015)

Of more importance to my argument in this article is the evidence that librarians interested in critical information literacy and critical pedagogy are also inspired by the Framework. At at an upcoming LOEX15 session, for example, Eamon Tewell and Kate Angell will share how the Frame “Authority is Contextual and Constructed” “emboldened” them to construct “new ways to empower learners and discuss authority’s role in evaluating resources.” (“Elevating” 2015) Kevin Seeber has shown how the frame “Format as Process” allowed him to teach critical thinking skills in the context of web-scale discovery (Seeber 2015). Examples such as these (which seem to be multiplying daily) demonstrate the pedagogical value and potential of the Framework. Although several critical librarians have found threshold concepts and/or the Framework5 wanting in one or more areas, and although I am also critical of both, I don’t believe that these critiques invalidate the Framework. In fact, despite the reservations that I will outline below, the Framework does not contradict or undermine the possibility of a critical information literacy instruction or critical pedagogy, but may very well encourage it, which is a vital point that librarians should remember. Many librarians who are committed to critical librarianship seem to share this view since they see the Framework as more liberating pedagogically than it is constricting.6

A Variety of Responses

Before exploring the responses from the perspectives of critical information literacy and critical librarianship, I would like to provide an overview of some of the other critiques of the Framework’s drafts. These critiques have been diverse, ranging from stunned incomprehension to almost utopian celebration.

One subset of responses to the Framework has made suggestions for improvement or requested clarification. These criticisms generally accept the Framework on its own terms and are concerned with its practicality, implementation, adaptability, and accessibility. People ask how the Frames or the threshold concepts upon which they are based will work in practice, what challenges will be posed by adopting the Framework, how different it will be from the Standards in this respect, and how librarians, faculty, and administrators will be convinced to replace explicit Standards with a set of guidelines that are less prescriptive.7

Many librarians are also anxious about potentially forfeiting the gains in information literacy instruction that have been achieved in the years since 2000, and which were at least partially premised on implementation and ‘selling’ of the Standards. This has already proven to be one of the Framework’s major stumbling blocks; at least some librarians have voiced alarm that losing standards spells trouble for their IL programs and their libraries.8

Related to the concerns about the Framework’s accessibility and saleability to stakeholders have been complaints about the drafts’ alleged ‘jargon.’ Even though much of the language that earned this epithet has been removed as a result of these complaints, some voices continue to describe the Framework’s use of language borrowed from threshold concept theory as jargon.9 This concern about language is partly tied to the anxieties about strained relationships with stakeholders, but it also reflects a certain resistance among academic librarians to theory imported from other disciplines into library practice or even into LIS scholarship.10 Even someone sympathetic to the ideas behind the Framework and to threshold concepts themselves might object to their explicit inclusion in a document which is intended to be used as a guide for establishing cross-disciplinary and inter-administrative relationships.11

But I would caution that although it may be true that the Framework’s reliance on threshold concepts may put off some faculty members and administrators with whom we want to collaborate, we run the risk at the other extreme of offering something of only limited persuasive potential from the perspective of ideas, and this could jeopardize potential understanding and ultimately collaboration from non-library faculty members across the institution. Librarians, as members of the academic community, must be prepared to engage with the scholarship and research of our peers if we wish them to engage with ours. And the most serious evidence of such engagement is to find specifically library-related applications of theoretical approaches from such fields as education, psychology and anthropology. To embrace theory from other disciplines will inevitably require us to learn to adapt concepts and language from those fields. In other words, it will require the introduction of novel concepts and ideas, reflected in new vocabulary. But rather than be afraid of such importations, we should engage them to test their foundations as well as their usefulness.

Another set of critiques has dissected the theoretical approach of the Framework, and while not complaining so much about jargon, still finds it flawed, often fatally so. These critiques have been thorough. They tend to focus on the theory of threshold concepts and its application in the frames themselves and subject it to interrogation and detailed analysis. Lane Wilkinson (a former member of the Task Force) has provided perhaps the most exhaustive analysis and deconstruction of threshold concept theory and its application to information literacy in the Framework. He set out to demolish much of the theory and language of the Framework in several detailed blog posts over the summer of 2014. (Wilkinson 2014a-g) Wilkinson’s contentions are varied but his principal focus is mostly on the conceptual (in)coherence and contradictions of threshold concepts. There is less attention given to considerations of the ways that political, social, economic, and cultural power structures and relations are reflected by or are challenged by this approach to information literacy (although his discussion does not entirely exclude these concerns).12

A Critical Information Literacy Perspective on Threshold Concept Theory

It is a main tenet of critical information literacy that information literacy instruction should resist the tendency to reinforce and reproduce hegemonic knowledge, and instead nurture students’ understandings of how information and knowledge are formed by unequal power relations based on class, race, gender, and sexuality. Threshold concept theory, both as it was originally formulated and as it is applied in the Framework, can be seen as a reification of privileged knowledge that is historically and culturally contingent.13 Threshold concepts attempt to align information literacy goals with the way that knowledge functions in our existing information system. Threshold concepts were elaborated specifically to better enable students to master the difficult specialized fields of knowledge that define the various academic and professional disciplines. But they may end up functioning as the means to merely reinforce disciplinary boundaries and institutional hierarchies.

Morgan has noted how the Framework’s effort to present threshold concepts in this way has produced an elision of their origins and contexts: “threshold concepts are treated as immanent entities, unique to specific disciplines, and not as essentially contingent.” (Morgan 2012, 7) Fister also cautions, in referring to the first draft: “…we need to bear in mind how these thresholds we define are cultural constructs and avoid assuming upper-middle-class white American experiences that might seem hostile or exclusionary to those who don’t fit that assumed identity.” (Fister 2014a) If threshold concepts are cultural constructs, then a critical information literacy must move beyond them somehow. While threshold concepts may have an important place in the process of learning, information literacy must demand that the concepts themselves be questioned as part of the critique of the structure of knowledge that a critical pedagogy encourages.

It is possible to see threshold concepts as an efficient way of getting students to become expert practitioners of existing disciplines. They do this, in a sense, by learning the rules. Threshold concepts can be viewed as the habits of mind that one must have in order to make sense within a given intellectual community. Wilkinson has noted that threshold concept theory has oversimplified or even misrepresented the true nature of academic disciplines, whose competing discourses reveal the opposite of what the theory claims: “The entire theory of threshold concepts has a funny way of oversimplifying the very real distinctions and difficulties that are inherent in a body of knowledge.” (Wilkinson 2014a)  I would add to this point that teaching students how to function within an academic discourse can be perilously close to teaching students how to conform, how to get along, how to succeed. We want our students to succeed, but do we want the system that will enable their success to succeed as well? Some may, but many librarians committed to critical librarianship do not. For the latter group the question is, how can we encourage student success without supporting the underlying structure of the system within which that success will take place?

Much of the rhetoric of information literacy, including that of the Framework, represents the world of information (the Framework refers to it as the “information ecosystem”) as something that must be mastered by individual students making their own ways through an educational institution out into the world. Information literacy instruction is intended, positively and even progressively, to empower those individuals to succeed on their own terms to the greatest extent possible. It does this by inculcating habits of thinking and working that are most often described under the heading ‘critical thinking.’ But the problem with even some progressive information literacy rhetoric is that it does not question the fundamental units with which it is working: the individual information consumer/producer on the one hand, and the system of information on the other hand. The Framework, despite its (debatable) greater theoretical sophistication, its great flexibility as a tool for enabling dynamic and creative information literacy instruction, and its emphasis on collaborative learning, still posits as its goal an individual student who has become a master or expert of our system of information. And even though it seeks to empower that individual, who could potentially work to change the conditions of information production and dissemination that exist today, the Framework necessarily concentrates its efforts on the solitary mastery of the existing system.

Some critics have found the Framework too narrowly focused on library-centered activities and skills, and they have questioned whether the specific threshold concepts in the Frames are uniquely characteristic of the ‘field’ of librarianship or information literacy. Fister states that as a librarian she isn’t particularly “interested in helping students think like librarians, but rather as curious people who understand how information works so that they can be curious effectively and maybe change the world while they’re at it.” (Fister 2014b) Nicole Pagowsky has also expressed this sentiment in a blog post reacting to the first draft, referring specifically to the frame ‘Format as Process’ (renamed ‘Information Creation as a Process’ in the final draft). The frames pay insufficient attention to the factors beyond academia that shape students’ consumption and production of information: “I was hoping to see a discussion on marginalized groups and whose voices get to be heard in traditional publishing and media (and why). These are important conversations to have with students, and particularly so when we are encouraging them to be creators of information, joining the conversation themselves. What impact might avenues of publishing have on their ability to be vocal when considering their perspective and identity? How is privilege intertwined in format and volume?” (Pagowsky 2014) These observations indicate that instruction librarians interested in integrating an understanding of these larger issues into information literacy will need to supplement and/or alter the frames’ more restricted purview.

But even in its narrow focus, the Framework rests on questionable assumptions. The frame ‘Scholarship as Conversation’14 tends to idealize or even naturalize the process of knowledge production in disciplinary fields. It presents scholarly research as a largely honorable pursuit, viewed in isolation from the forces operating around (and within) it: “Research in scholarly and professional fields is a discursive practice in which ideas are formulated, debated, and weighed against one another over extended periods of time.” As described, it does not pay sufficient attention to the ways that some voices are suppressed, silenced, and marginalized because they do not fit the proscribed boundaries of that field – which are, in the end, determined by a consensus of practitioners whose professional reputations and livelihoods often depend on the preservation of these boundaries and conventions. In other words, threshold concepts describe knowledge creation in a decontextualized manner, even though the Framework tries to acknowledge the academic context of knowledge creation.

While one might not share Wilkinson’s denial that scholarship is in fact a ‘conversation,’ one can’t ignore how politics and power play a decisive role in the production of knowledge. (Wilkinson 2014b) It is a common complaint within many academic fields that conformity, uniformity, predictability and consensus are all-too common features of scholarship – are these the results of a ‘conversation’ or something else? Is it possible to build into the Framework an acknowledgement that scholarship and research themselves are always functioning within particular economic, social, and political systems that help determine the features and structure of the ‘scholarly conversation?’ Or must information literacy instruction move beyond threshold concepts altogether, even if it begins with them as a way of entering into and to some extent identifying with the existing structure of knowledge and expertise? These questions speak to one of the basic conundrums of critical librarianship and critical information literacy, namely: how does one teach students to understand and make the best use of existing systems of knowledge while at the same time prompt them to question the validity and structure of those systems? It’s a similar conundrum faced by all scholars and teachers in academia who see themselves as committed to radical social change: how can one be a part of the system of oppression yet claim to be fighting against it?15

To better appreciate the perils of relying on threshold concepts, it may help to consider the needs for which the theory was originally developed. They were proposed by educational theorists Erik Meyer and Ray Land with reference to teaching concepts in economics. That discipline, at least as it is practiced in the ‘western world’ today, functions largely as a closed field based on a broad consensus about the universal validity (at least in the abstract) of the so-called ‘free market’, in other words, the universality and inevitability of capitalism. Economics, as an academic field, tends to naturalize capitalism and works to maintain the belief that the rules/laws of that system are simply the rules/laws of economics as such (even economists like Thomas Piketty who dare to challenge some of the field’s pieties still share this core faith16). It is very difficult for an economist who questions the fundamental assumptions of capitalism or denies its “laws” to succeed in or even enter the field, and the refusal to accept the field’s central concepts prevents communication at a basic level with the vast majority of its practitioners. It is likely that a typical economist – someone who would be considered an ‘expert’ or ‘authority’  – would judge a person making such a challenge not only ineligible to participate in the field, but perhaps even a threat to it.

Meyer and Land do not pay attention to the limitations posed by established fields of knowledge, but rather the challenges that outsiders, or learners, face when trying to enter into productive learning, or ‘conversation’, within the field. Their insight was to identify certain seemingly universal characteristics of knowledge within disciplines that can be treated as concepts that one has to master in order to function successfully as a practitioner in that field. These ‘troublesome concepts’, once grasped, allow the learner to readily understand the assumptions and terms of debate in a field. But I would argue that at this point the learner has in some sense reached the starting point, not the end point, of learning on a deeper level. Now the task is to question what one has just learned – and this is where the question of information literacy’s ultimate goal returns.

What is the Purpose of Information Literacy Instruction?

From a critical information literacy perspective, the Framework’s larger assumptions pose perhaps the biggest problems. One has to do with the term ‘information literacy’ and its complicated history. Critical information literacy has sought to increase awareness of how much the information literacy agenda has been set and supported by broader structural forces in academia and the world at large that may in fact be at odds with the core values of librarianship, progressive learning and radical social change. Christine Pawley, in a trenchant and erudite critique of what she calls “Information Literacy Ideology”, states that information literacy “has contributed to the decontextualization of information, obscuring the specific conditions of its production.” (Pawley 2003, 425) This decontextualization allows people to forget, or not to learn, that “Information never stands alone – it is always produced and used in ways that represent social relationships,” and that those relationships “reflect the underlying patterns that structure society.” (Pawley 2003, 433) Pawley has informed us that “information literacy” was elaborated at the beginning of the digital age and was intended largely to recuperate forms and markers of the authority from the age of print that were feared to be slipping away from librarians’ control: “…institutional practices of information literacy have the effect of reestablishing relations of authority and authenticity that developed over three centuries for the print production of commodified information.” (Pawley 2003, 440)

Much of information literacy instruction, yesterday and today, is focused on preparing students to succeed in both academia and the world beyond it – which more often than not amounts to teaching them skills of research and thinking that will enable them to function as productive independent minds in a competitive, rapidly changing economic environment. In other words, information literacy is designed to improve students’ chances at getting jobs and succeeding in their chosen professions.17 No one who teaches and cares about students would object to that goal. But a critical information literacy expects more than this (and wants more for students); it pushes information literacy instruction, in various ways, not to be limited by this goal. Moreover, critical information literacy even looks beyond ‘lifelong learning’, since the question always should be asked, what actually is ‘lifelong learning’ and what is its purpose? As Cathy Eisenhower and Dolsey Smith have argued, lifelong learning and critical thinking fall within the realm of neoliberal rationality which push the learner “toward a perpetual anxiety of regulation, of adjustment, of optimization—and toward reason’s perpetual self-improvement.” (Eisenhower and Smith, 2009)

Information Literacy Instruction is also About Resistance

Chris Bourg, in a 2014 address at Duke University Libraries, insisted that despite the fact that “neoliberalism is toxic for higher education…research libraries can & should be sites of resistance.” (Bourg 2014). Critical librarianship is at pains always to show that the existing information system mirrors the larger social and political order, which is characterized by a radically asymmetrical distribution of power, and is shot through, systematically and structurally, by racism, sexism, homophobia, militarism, and class oppression. An advocacy of progressive literacy of any kind within this system and environment, requires resistance on the part of the librarian: resistance to existing regimes of knowledge, as institutionalized by academic disciplines and departments (and enforced by academic rules and administrative bureaucracies), resistance to the commodification of knowledge, and even resistance to the stated goals of higher education as they are commonly promoted, especially by administrators, politicians, bureaucrats and educational reformers. Failing to resist all too easily provides reinforcement to the existing system, and helps reproduce it.

Joy James and Edmund T. Gordon have described the problematic position that a radical or activist intellectual necessarily assumes within academia. Their observations are relevant to the aspirations of critical information literacy and the basic dilemma that questions around the Framework have called attention to. They claim that academic institutions “are at best liberal-reformist in their institutional policies and at worst complicit with the global military-industrial and consumer-commercial, complex that enforces and/or regulates the marginalization and impoverishment of the majority of the world.” They note that “Institutions of higher education have a vested interest in keeping scholarship ‘objective’ (mystifying), ‘nonpolitical’ (nonsubversive) and ‘academic’ (elitist) and in continuing to reserve the most advanced technical training for that small portion of the world’s population who will manage the rest, as well as consume or control its resources and political economies.” With such an important mandate, anyone who works within academia (i.e. who is engaged in the ‘scholarly conversation’) is subjected  to intense pressures to conform: “…incentives offered by the academy reward those whose knowledge production contributes to elite power…That same system diminishes the production of potentially transgressive political knowledge by questioning its ‘objective’ status or ‘scientific’ value.” The participation of radical intellectuals in academic institutions actually “strengthens [those institutions] by allowing them to make hegemonic claims to fostering ‘academic freedom,’ a ‘marketplace of ideas,’ and rational neutrality…” (James and Gordon 2008, 367-9). This perilous position, James and Gordon argue, can only be remedied by exiting the academy and establishing solidarities with oppressed peoples organizing against the system (even if one keeps one’s ‘day job’ as a teacher and researcher within the academy). Whether or not one accepts their conclusion, one can take their description of the situation of the activist scholar and apply it to critical information literacy, whose practitioners should always be aware of the reifying and recuperative functions of information literacy in the academy. Eisenhower and Smith have argued along similar lines, as I indicated above, but they believe that librarians may be in a position from which to exercise a greater freedom of action vis-a-vis the pressures to conform, by virtue of their marginal or liminal position within the academy. Although librarians’ status varies very widely across academia, and it is therefore difficult to make such generalizations, the opening they suggest is nonetheless one that all librarians should seek, whatever their situations may be. (Eisenhower and Smith 2009)

As long as we recognize the structural function of information and knowledge in our pedagogy, we can help bridge the gap between academia and the struggle for social justice. With respect to this goal, using the term ‘information ecosystem’, as the Framework does, is not helpful. I recognize that the term has entered our daily vocabulary, but whether one intends to or not, the term works to reify information, despite the first frame’s title, “Authority is Constructed and Contextual.” And even though the term stresses the rapidly changing nature of that system, it does not emphasize its artificiality and arbitrariness, that it is a reflection of a specific distribution of power. To describe natural processes requires a comprehension of complex and often rapid changes. But changes in knowledge are anything but natural. In the pages of this journal Joshua Beatty has pointed out the Framework’s “neoliberal underpinnings” (something it shares in common with the Standards). He usefully traces the history of using ecological language such as ‘information ecosystem’ to describe social forces to the business world of the 1990s, when today’s neoliberal order took shape. Beatty convincingly links this naturalized language to a revived social Darwinism in which only fittest survive in a cutthroat world of brutal competition. We are competing with others to acquire and produce the best information possible, and it is up to us (and our helpers, teachers and librarians), to acquire the necessary skills and smarts to do this. (Beatty 2014, 10-11) When we unwittingly adopt this language to describe the learning processes that we wish to encourage, we may be leaving fundamental neoliberal premises unquestioned.

While the framework does an admirable job of showing how threshold concepts can help shift information literacy toward a pedagogy that stresses the development of self-critical and self-conscious learning in the student, it does not state as its goal the formation of possible solidarities for the student to help change the information system itself, nor the hierarchies of knowledge and status within academia. Furthermore, by continuing to stress the individual learner, it obscures the fact that any real change would actually require collective understanding and action rather than individualized learning. In this way the Framework continues to do the work that the Standards were doing all along. But the vital difference between the two, perhaps, is the enhanced opportunities for critical interventions that the Framework provides and even encourages.18

From a critical information literacy perspective, then, it appears that the specific type of information literacy advocated by the Framework is one which accepts the existence of a particular regime of knowledge, and demands that we as librarians focus our energies on making students and faculty competent citizens of that regime, even if dynamic, critical, and progressive ones. Here again we are faced with the dilemma outlined above: students have immediate needs to be met – they are working on research papers, projects, reports, and theses. They not only need information and sources, and to learn how to conduct research, they also need to master the conceptual frameworks that will enable them to effectively and convincingly make persuasive arguments. All of this very sophisticated and complex instruction needs to be done in a short space of time. Librarians have to help so many of them, with insufficient resources, and not enough time. Where can an information literacy that raises an awareness of the contingent and arbitrary nature of the information system, be fit? When does it take place? Can something like ACRL’s Framework possibly incorporate such a vision without undermining itself?

The answers to these questions are varied and complex, and they are being explored by the many librarians who theorize and practice critical information literacy. They have taught us that we must assume a position of resistance rather than conformity to the existing information regime, if we wish to see it changed at all. Part of the solution has to do with the content of library instruction. For instance, in teaching specific research or searching skills, the examples that we use in the classroom and at the reference desk can provide opportunities to question information regimes in more systematic ways.19 Another part has to do with our everyday practice as librarians, inside and outside of the classroom. We can find a long tradition of resistance on the part of librarians, not only against the banning of books or spying by the government, but also against the very structure of information and knowledge that they are supposed to be the guides for unlocking.20 Resistance is shown by librarians who take proactive measures in pushing for open access, calling out or refusing rapacious vendor contracts, or finding ways to actually make our profession more diverse, just to name a few areas. But what does resistance in information literacy instruction look like? I think we will see more creative examples in the coming years, thanks, ironically perhaps, to the Framework, which, as I stated at the outset, has opened up the possibilities for action and maneuver on the part of instruction librarians, despite its ideological baggage. In this sense it is a progressive document, but it will require librarians to resist it in order for it to become a radical one.

My thanks go to Ellie Collier and Emily Drabinski for their many trenchant comments and suggestions and also for helpfully encouraging me to emphasize my own voice in the editing of this article. Thanks to Robert Farrell, Barbara Bonus-Smit, and Julia Furay for inviting me to give an earlier version of this paper at ACRL/NY-LILAC’s panel on the Framework in October, 2014 at Barnard College. Thanks also to Donna Witek for her brilliant reflections and inspiring efforts toward making the Framework understood for critical librarians as well as for her encouragement in my own efforts in this regard. Special thanks to Rory Litwin for pointing me to some of the rich history of librarian activism in the (near) past, and some of the written record of that history. Thanks finally to all my interlocutors over the last several months online and in person who have grappled with the Framework, especially to the #critlib community that generates a treasure trove of wisdom and practical insight every fortnight.

 

Works Cited

Accardi, Maria T., Emily Drabinski, and Alana Kumbier, eds. (2009) Critical Library Instruction: Theories and Methods. Library Juice Press.

ACRL Board of Directors. (2015) “Action Form: ACRL MW15 Doc 4.0.” January 16, 2015.

Adler, Kate. (2013) “Radical Purpose: The Critical Reference Dialogue at a Progressive Urban College.” Urban Library Journal, 19, 1.

Beatty, Joshua. (2014) “Locating Information Literacy within Institutional Oppression.” In the Library with the Lead Pipe. September 24, 2014.

Berg, Jacob. (2014a) “The Draft Framework for Information Literacy for Higher Education: Some Initial Thoughts.” BeerBrarian. 25 Feb. 2014.

—. (2014b) “The (Second) Draft for Information Literacy for Higher Education: My Thoughts.” BeerBrarian. 11 Jul. 2014.

Bourg, Chris. (2014) “The Neoliberal Library: Resistance is not Futile.” Feral Librarian. 16 Jan. 2014.

Burkhardt, Andy. (2014) “Threshold Concepts in Practice: An Example from the Classroom.” Information Tyrannosaur. 4 Mar. 2014.

Drabinski, Emily. (2014) “Toward a Kairos of Library Instruction.” The Journal of Academic Librarianship 40: 480-485.

Eisenhower, Cathy and Dolsy Smith. (2009) “The Library as ‘Stuck Place’: Critical Pedagogy in the Corporate University.” In Critical Library Instruction: Theories and Methods, edited by Maria T. Accardi, Emily Drabinski, and Alana Kumbier. Duluth, MN: Library Juice Press, 305-318.

“Elevating Source Evaluation: Teaching and Un-teaching Authority in the Critical Library Classroom.” (2015) LOEX 2015 – Sessions. Loex. Web. 18 February 2015.

Fister, Barbara. (2014a) “On the Draft Framework for Information Literacy.” Library Babel Fish. Inside Higher Ed, 27 Feb. 2014.

—. (2014b) “Crossing Thresholds and Learning in Libraries.” Library Babel Fish. Inside Higher Ed, 22 May 2014.

—. (2015) “The Information Literacy Standards/Framework Debate.” Library Babel Fish. Inside Higher Ed, 22 Jan. 2015.

Gregory, Lua, and Shana Higgins. (2013) Information Literacy and Social Justice: Radical Professional Praxis. Duluth, MN: Library Juice Press.

Hicks, Alison. (2013) “Cultural Shifts: Putting Critical Information Literacy into Practice.” Communications in Information Literacy, 7 Aug. 2013.

Hofer, Amy R, Lori Townsend, and Korey Brunetti. (2012) “Troublesome Concepts and Information Literacy: Investigating Threshold Concepts for IL Instruction.” portal: Libraries in the Academy 12(2): 387-405.

Hofer, Amy R, Lori Townsend, and Korey Brunetti. (2011) “Threshold Concepts and Information Literacy.” portal: Libraries in the Academy 11(3): 853-869.

James, Joy and Edmund T. Gordon. (2008) “Afterword: Activist Scholars or Radical Subjects?” In Charles Hale, ed., Engaging Contradictions: Theory, Politics, and Methods of Activist Scholarship. Berkeley, CA: University of California Press. 367-73.

Kagan, Alfred. (2015) Progressive Library Organizations. A Worldwide History. Jefferson, NC: McFarland & Company, Inc., Publishers.

Klipfel, Kevin Michael. (2014) “This I Overheard…Threshold Concepts Getting Laughed Out of the Room” Rule Number One: A Library Blog. 3 Nov. 2014.

Matthews, Brian. (2011) “What Can You Do to Help With Troublesome Knowledge?

Librarians and Threshold Concepts,” The Ubiquitous Librarian, Chronicle of Higher Education Blog Network, 3 Aug. 2011.

Morgan, Patrick. (2015) “Pausing at the Threshold.” portal: Libraries and the Academy. 15(1).

Oakleaf, Megan. (2014) “A Roadmap for Assessing Student Learning Using the New Framework for Information Literacy for Higher Education.” Journal of Academic Librarianship. Preprint.

Olson, Hope. (2001)  “The Power to Name: Representation in Library Catalogs.” Signs 26(3): 639-668.

Pagowsky, Nicole. (2014) “Thoughts on ACRL’s New Draft Framework for ILCSHE.” Nicole Pagowsky (pumpedlibrarian). 2 Mar. 2014.

Pawley, Christine. (2003) “Information Literacy: A Contradictory Coupling.” Library Quarterly 73(4): 422-452.

Samek, Toni. (2001) Intellectual Freedom and Social Responsibility in American Librarianship, 1967-1974. Jefferson, NC: McFarland & Company, Inc., Publishers.

Seeber, Kevin. (2015) “Teaching “Format as a Process” in an Era of Web-scale Discovery.” Reference Services Review 43:1: 19-30.

Swanson, Troy. (2015) “The IL Framework and IL Standards Cannot Coexist.” 12 Jan. 2015. Tame The Web. Web. 12 Jan. 2015.

Tewell, Eamon. (2014) “Tying Television Comedies to Information Literacy: Mixed-Methods Investigation.” Journal of Academic Librarianship 40: 134-141.

Townsend, Lori, Silvia Lu, Amy R. Hofer, and Korey Brunetti. (2015) “What’s Wrong with Threshold Concepts?” ACRlog. 30 Jan. 2015.

Wilkinson, Lane. (2013) “Information Literacy: Standards, Skills, and Virtues.” Sense and Reference. 5 Jun. 2013.

—. (2014a) “The Problem with Threshold Concepts.” Sense and Reference. 19 Jun. 2014.

—. (2014b) “Is Scholarship a Conversation?” Sense and Reference. 10 Jul. 2014.

—. (2014c) “Is Research Inquiry?” Sense and Reference. 15 Jul. 2014.

—. (2014d) “Is Authority Constructed and Contextual?” Sense and Reference. 22 Jul. 2014.

—. (2014e) “Is Format a Process?” Sense and Reference. 25 Jul. 2014.

—. (2014f) “Is Searching Exploration?” Sense and Reference. 29 Jul. 2014.

—. (2014g) “Does Information Have Value?” Sense and Reference. 5 Aug. 2014.

  1. See the ACRL Board of Directors Action Form from January 15, 2015 for background and details about the process of drafting the Framework.
  2. See for example the Open Letter of New Jersey Librarians criticizing the Framework and protesting the sunsetting of the ACRL Standards, which indicates that there will be opposition not only to the replacement of the Standards, but also the wholesale adoption of the Framework that is meant to replace it. The recent acceptance of the Framework by the ACRL Board of Directors was accompanied by a temporary cancellation of plans to sunset the Standards.
  3. For space reasons, I refer the reader to Lane Wilkinson’s helpful list of blog posts, which is now out of date, but provides a good starting point for reading a variety of responses to the Framework’s early drafts.
  4. The literature on critical information literacy and critical librarianship (which are closely related and overlap quite a bit, but should not be equated) is rapidly expanding. See especially the contributions in Accardi, Drabinski, and Kumbier 2009; and Gregory and Higgins 2013. Probably the most extensive (though not exhaustive) bibliography of critical information literacy we have is to be found in the recent dissertation by B. A. McDonough, Critical Information Literacy in Practice, 2014.
  5. As Townsend, et al, have indicated in their recent ACRLog post, the Framework contains only one possible collection and interpretation of threshold concepts among many: potentially there might be other valid threshold concepts applicable to information literacy instruction. (Townsend, et al 2015)
  6. Intriguingly, Fister has decided to embrace the Framework because it returns to the original promise of the ACRL Standards when they were established in 2000: they “were meant as a starting place, that each library should adapt them to fit their local cultures and needs. They weren’t set in stone.” (Fister 2015)
  7. The Task Force’s report attached to the final draft notes that in its tally of the 206 comment forms submitted following the release of the third draft, 67.4% of respondents supported the Framework. (ACRL MW15 DOC 4.0) Some librarians have already embarked on projects to help librarians map the Standards to the Framework in order to help them understand the continuities and make the transition easier in redesigning instruction programs. See, for example, Amada Hovious’ “Alignment Charts for ACRL Standards and Proposed Framework (work in progress)” and her related blog post.
  8. Troy Swanson, in a recent response to some of these concerns, convincingly defends the Framework on the following grounds: “if we believe that information literacy matters in the lives of our students and see information literacy as a form of empowerment for our students, the idea that we should write standards because that’s what everyone else is doing feels hollow.” And, even more boldly, he suggests that “our profession has the opportunity to take the lead in moving away from the mechanistic bureaucracy of standards-based education. I do not know many faculty members who honestly think that more standards and more standardization will improve teaching and learning.” (Swanson 2015) But Emily Drabinski urges us to take pause in this enthusiasm, alerting people how the Framework’s ACRL imprimatur insures that it will function in much the same way the Standards has, only now libraries will have to shoulder the burden of spelling out the specific learning outcomes to which these guidelines may point. She suggests as an alternative a kairos of library instruction that does not rely on a universal, fixed document of any kind and that instead always responds to the specific time and place in which the instruction takes place. (Drabinski 2014) Critical advocates of the Framework, myself included, would claim in response that it actually encourages kairos by not prescribing any specific learning outcomes or benchmarks.  
  9. Andy Burkhart, in a blog post, reported librarians at a discussion at 2014 ALA Midwinter referring to threshold concepts themselves as ‘jargon.’ (Burkhart, 2014)
  10. A distrust of theory from other fields of course is not limited to LIS or librarians. Academic history, for example, has maintained a stubborn resistance to theoretical approaches and many of its practitioners and gatekeepers are on the lookout for ‘jargon’ in an effort to maintain clarity and rigorous thinking. The writing and teaching of history is, like the practice of librarianship, self-consciously a ‘popular,’ or public-oriented undertaking, and the assumption is that it should always only utilize language immediately familiar to the widest possible audience. However well-intentioned and even democratic this impulse is, it is often used as an excuse for avoiding an engagement with developments in other fields, or for simply dismissing them. For a brief and spirited plea for the importance of theory for history, see Joan Wallach Scott, “Wishful Thinking,” Perspectives on History (December 2012)
  11. Kevin Michael Klipfel disapproves of threshold concept theory not so much because it has been borrowed from another field, but because it has not been ‘scientifically’ validated and is therefore not legitimate in the eyes of his colleagues in other disciplines. See his blog post, “This I Overheard…Threshold Concepts Getting Laughed Out of the Room,” Rule Number One: A Library Blog, 3 Nov. 2014. But even long before the Framework committee went to work revising the ACRL Standards, some librarians embraced threshold concepts precisely because there were positive signs of acceptance and enthusiasm from colleagues. See, for example, Brian Matthews, “What Can You Do to Help With Troublesome Knowledge? Librarians and Threshold Concepts,” The Ubiquitous Librarian, Chronicle of Higher Education Blog Network, 3 Aug. 2011.
  12. In a similar vein to Wilkinson, Patrick Morgan, in a recent article, has described the Framework as ‘inchoate.’ (Morgan 2014) And Jacob Berg, in his blog, has echoed and elaborated on some of Wilkinson’s critique, but his objections, too, I would classify more as linguistic-philosophical. (Berg 2014a-b)
  13. For an explanation of how threshold concept theory has been applied to information literacy instruction, see the foundational articles by Hofer, Townsend, and Brunetti 2011, 2012.
  14. This is the frame’s name in the final draft of the Framework.
  15. This is a question answered brilliantly by Joy James and Edmund T. Gordon who conclude that in some sense there is no such thing as ‘radical’ scholarship and teaching, since true activism actually cannot take place within the academy. (James and Gordon 2008) Cathy Eisenhower and Dolsey Smith offer a different view that sees more room for action for librarians in this regard because of their more liminal position in the academy. (Eisenhower and Smith 2009) See more on this in the section below.
  16. Slavoj Zizek and David Harvey each have noted that Piketty is best described as a utopian capitalist: for him there is no questioning the assumption that it is the only system that ‘works.’ See David Harvey, “Afterthoughts on Piketty’s Capital”; Slavoj Zizek, “Towards a Materialist Theory of Subjectivity,” lecture, The Birkbeck Institute for the Humanities, University of London, May 22, 2014.
  17. Drabinski shows how this justification became the “procustrean bed” into which IL instruction was placed by the Standards, and she argues convincingly that the Framework inevitably is driven by the same mandate, which has only intensified since 2000. (Drabinski 2014)
  18. Beatty, in fact, is another critical librarian who acknowledges that “there are many ways in which the Framework significantly improves on the Standards.” (Beatty 2014, 4)
  19. For examples of this see Hicks 2013; Adler 2013; Tewell 2014.
  20. I cite here as likely the most well-known example the legendary Sanford Berman. A general history of librarians and resistance, not only in the United States but globally, has yet to be written. Such a narrative would be useful for our present struggles since sometimes it seems that librarians believe that only recently has the library been challenged as a supposedly ‘neutral’ grounds for the discovery of knowledge. But librarians have been doing so since at least the 1930s. There are of course some excellent studies that are focused in scope, but very instructive for for this purpose: see especially, Samek 2001; Olson 2001;  Kagan 2015. Also helpful is the archive of the ALA Social Responsiblities Roundtable Newsletter and the archive of the blog Library Juice

Mark E. Phillips: DPLA Metadata Analysis: Part 2 – Beyond basic stats

planet code4lib - Wed, 2015-02-25 13:00
More stats for subjects

In my previous post I displayed some of the statistics that are readily available from Solr as part of its StatsComponent functionality (if you haven’t used this part of Solr yet you really should). There are a few other things that we could collect to get a more complete picture of a metadata field.

So far we have min, max, number of records, total number of subjects, sumofsquares, mean, and standard deviation. The other values I think we should take a look at are the following.

Records without Subjects – Number of records without subjects.

Percent of records without subjects – Percentage of the Hubs records that don’t have subjects

Mode – Number of subjects-per-record that is the most common for a specific Hub.

Unique Subjects – Unique subject strings are present for a specific Hub

Hub Unique Subjects – Number of subjects that are unique to that Hub.

Entropy of the field. – This calculation is a measure of the uncertainty in the metadata field, but for our purposes it is a good measure to understand how the distribution of subjects happens in the records.

Below is a table that contains the fields listed above,  plus some relevant fields from the previous post. Each Hub has a row in this table.

Hub Name Records Records Without Subjects % without Subjects Avg. Subjects per records Subject Count Mode Unique Subjects # of subjects unique to hub Entropy ARTstor 56,342 6,586 11.7 3.5 3 9,560 4,941 0.73 Biodiversity Heritage Library 138,288 10,326 7.5 3.3 2 22,004 9,136 0.65 David Rumsey 48,132 30,167 62.7 0.5 0 123 30 0.76 Digital Commonwealth 124,804 6,040 4.8 2.4 1 41,704 31,094 0.77 Digital Library of Georgia 259,640 3,216 1.2 4.4 2 132,160 114,689 0.67 Harvard Library 10,568 167 1.6 2.5 2 9,257 7,204 0.76 HathiTrust 1,915,159 525,874 27.5 1.4 1 685,733 570,292 0.88 Internet Archive 208,953 44,872 21.5 1.8 1 56,911 28,978 0.80 J. Paul Getty Trust 92,681 73,978 79.8 0.4 0 2,777 1,852 0.60 Kentucky Digital Library 127,755 117,790 92.2 0.2 0 1,972 1,337 0.62 Minnesota Digital Library 40,533 0 0 5 4 24,472 17,545 0.74 Missouri Hub 41,557 11,451 27.6 2.3 0 6,893 4,338 0.69 Mountain West Digital Library 867,538 49,473 5.7 3 1 227,755 192,501 0.68 National Archives and Records Administration 700,952 619,212 88.3 0.3 0 7,086 3,589 0.63 North Carolina Digital Heritage Center 260,709 41,323 15.9 3.3 2 99,258 84,203 0.66 Smithsonian Institution 897,196 29,452 3.3 6.4 7 348,302 325,878 0.62 South Carolina Digital Library 76,001 7,460 9.8 3 2 23,842 18,110 0.72 The New York Public Library 1,169,576 208,472 17.8 1.7 1 69,210 52,002 0.62 The Portal to Texas History 477,639 58 0 11 10 104,566 87,076 0.49 United States Government Printing Office (GPO) 148,715 1,794 1.2 3.1 2 174,067 105,389 0.92 University of Illinois at Urbana-Champaign 18,103 4,221 23.3 3.8 0 6,183 3,076 0.63 University of Southern California. Libraries 301,325 35,106 11.7 2.9 2 65,958 51,822 0.59 University of Virginia Library 30,188 229 0.8 3.2 1 3,736 2,425 0.60

In looking at the row for The Portal to Texas History we can see that of the 477,639 records in the dataset, 58 of them do not have any subjects,  which is a very small percentage (0.01214306202 to be exact).  From there we can go to the average of 11 subjects per record with a mode of 10,  nothing earth shaking here,  just more info.  There are 104,566 unique subjects in the Portal’s dataset with 87,076 of those being unique to only the Portal.  Finally the entropy for the Portal’s subject field is 0.49,  if compared to GPO’s which is 0.92 you can interpret this to mean that the subject values are more “clumpy” for the Portal,  (a smaller number of subjects are used across a larger number of records) than for GPO (a larger number of subjects are used across records).

The following two tables further illustrate the entropy values on the Portal’s and GPO’s subjects. The first table is the top ten subjects and the number of records with those subjects from the GPO’s dataset

National security–United States 1,138 United States. Congress. House–Rules and practice 748 Terrorism–United States–Prevention 718 United States. Department of Defense–Appropriations and expenditures 631 United States 536 Social security–United States–Periodicals 487 Emergency management–United States 485 Medicare 441 Consumer protection–United States 417 Wisconsin–Maps 406

Now take a look at the top ten subjects and their counts for the Portal.

Places 310,404 United States 306,597 Texas 305,551 Business, Economics and Finance 248,455 Communications 223,783 Newspapers 221,422 Advertising 218,527 Journalism 217,737 Landscape and Nature 76,308 Geography and Maps 70,742

So with the entropy value,  you can read a lower number to be more like the Portal’s subjects and the higher number to be more like GPO’s.  At the extreme,  a value of 1.0 would mean that every subject is used by one record, and value of 0 would mean that there is only one subject with all of the records using said subject.

Shared Subjects

In creating the table above I had to work out the number subjects that a hub has uniquely.  In doing so I went ahead and calculated this number for the whole dataset to find out how much subject overlap occurs.

The table below displays the breakdown of how subjects are distributed across Hub collection.  For example if two Hubs have the subject “Laws of Texas” then it is said to be shared by two Hubs.  The breakdown for the metadata in the DPLA is as follows.

# of Hubs with subject Count 1 1,717,512 2 114,047 3 21,126 4 8,013 5 3,905 6 2,187 7 1,330 8 970 9 689 10 494 11 405 12 302 13 245 14 199 15 152 16 117 17 63 18 62 19 32 20 20 21 7 22 7

Most of the subjects 1,717,512 to be exactly occur in only one Hub’s collection.

There are seven different subjects that are common across 22 of the 23 Hubs in the DPLA metadata dataset,  if you are curious,  theses subjects are the following:

There should be one final post in this series where I can hopefully suggest what we should do with this data.

Again, if you want to chat about this post,  hit me up on Twitter.

 

District Dispatch: House Committee holds hearing on network neutrality

planet code4lib - Wed, 2015-02-25 09:05

On the eve of the FCC’s expected vote in favor of strong net neutrality rules, the House committee provided a preview of the challenges ahead in defending the open Internet. Republicans on the House Energy and Commerce Committee today expressed concerns at its hearing, The Uncertain Future of the Internet, ahead of the FCC’s anticipated net neutrality vote. The FCC is expected tomorrow to approve an Order reclassifying Internet service providers as telecommunications services subject to a higher level of regulation.

Anyone who tuned in to today’s hearing at the House Energy and Commerce Committee hearing would have come away with two vastly divergent views of where the expected FCC ruling on net neutrality will be taking the internet. Congress, let alone the public, has yet to learn the details of what the FCC will be voting on forcing today’s hearing to focus on broad net neutrality principals.

ALA has urged the FCC to approve reclassification as the only means of ensuring a fair and open internet.

In his opening statement, Chairman Greg Walden (R-OR) noted that the FCC adoption “may not ultimately provide net neutrality protections for American consumers; that might lay the groundwork for future regulation of the Internet; that may raise rates for the American Internet users; and that could stymie Internet adoption, innovation, and investment.”

Several Republicans on the Committee echoed the Chairman’s concern calling on the FCC to delay its vote and allow Congress to enact legislation which, they argue, should have authority to make reclassification decisions over broadband providers. Republicans in the House and Senate are circulating draft legislation intended to supersede FCC action but have to date attracted any Democrats willing to co-sponsor legislation.

Democrats on the Committee, led by ranking member Anna Eshoo (D-CA), counter that the FCC should proceed with regulating the internet providers as the only means of ensuring network neutrality.

One thing is certain with network neutrality: Congress will be talking about this issue for some time. Republicans in the House and Senate have hinted that they intend to maintain pressure on the FCC through legislation, oversight and budget hearings, and possible investigations into White House actions appearing to pressure the FCC, an independent agency, towards adopting a particular course of action by approving reclassification.

The post House Committee holds hearing on network neutrality appeared first on District Dispatch.

DuraSpace News: Universidad de la Sabana: Colombia Evolves Its Institutional Repository

planet code4lib - Wed, 2015-02-25 00:00

Chía, Colombia  Universidad de la Sabana´s institutional repository provides services to the university’s researchers, students and the academic community of this prestigious Colombian University. Arvo Consultores has helped the institution upgrade their repository to Dspace 4.2.  Improvements include an enhanced interface, with a new adaptive and responsive interface –the first Mirage2 interface in a Colombian institution.

HangingTogether: New MARC Usage Data Available

planet code4lib - Tue, 2015-02-24 21:40

I just finished updating my “MARC Usage in WorldCat” web site that summarizes and reports on how MARC elements have been used in the 333,518,928 MARC records in WorldCat as of 1 Jan 2015.

Not surprisingly, the totals for new fields such as 336, 337, and 338 have shot up, from some 9-10 million occurrences in January 2014 to 40-50 million occurrences in January 2015.

Also, it appears that well over 33 million records have come in through the Digital Collection Gateway.

As always, if you wish to see the summarized contents of any subfield just let me know. And don’t forget about the visualizations (pictured).

About Roy Tennant

Roy Tennant works on projects related to improving the technological infrastructure of libraries, museums, and archives.

Mail | Web | Twitter | Facebook | LinkedIn | Flickr | YouTube | More Posts (86)

FOSS4Lib Upcoming Events: CollectionSpace Walkthrough March 2015

planet code4lib - Tue, 2015-02-24 20:53
Date: Friday, March 27, 2015 - 12:00 to 13:00Supports: CollectionSpace

Last updated February 24, 2015. Created by Peter Murray on February 24, 2015.
Log in to edit this page.

From the announcement:

Curious about CollectionSpace?

Join Megan Forbes, Community Outreach Manager, for the first in a series of bi-monthly walkthroughs. More than just a demo of features and functionality, the walkthrough will:

FOSS4Lib Upcoming Events: CollectionSpace Open House Feb 2015

planet code4lib - Tue, 2015-02-24 20:49
Date: Friday, February 27, 2015 - 12:00 to 13:00Supports: CollectionSpace

Last updated February 24, 2015. Created by Peter Murray on February 24, 2015.
Log in to edit this page.

From the announcement:

Curious about CollectionSpace?
Join the program staff, leadership, functionality, and technical working group members, implementers, and special guests for a bi-monthly open house. Bring your questions (functional, technical, operational), your ideas, and your projects to this free-ranging conversation.

Mark E. Phillips: DPLA Metadata Analysis: Part 1 – Basic stats on subjects

planet code4lib - Tue, 2015-02-24 18:01

One a recent long flight (from Dubai back to Dallas) I spent some time working with the metadata dataset that the Digital Public Library of American’s (DPLA) provides on its site.

I was interested in finding out the following pieces of information.

  1. What is the average number and standard deviation of subjects-per-record in the DPLA
  2. How does this number compare across the partners?
  3. Is there any different that we can notice between Service-Hubs and Content-Hubs in the DPLA in relation to the subject field usage.
Building the Dataset

The DPLA makes the full dataset of their metadata available for download as single file, and I grabbed a copy before I left the US because I knew it was going to be a long flight.

With a little work I was able to parse all of the metadata records and extract some information I was interested in working with, specifically the subjects for records.

So after parsing through the records to get a list of subjects per record and the Service-Hub or Content-Hub that the record belongs to I loaded this information into Solr to use for analysis. We are using Solr for another research project related to metadata analysis at the UNT Libraries (in addition to our normal use of Solr for a variety of search tasks) so I wanted to work on some code that I could use for a few different projects.

Loading the records into the Solr index took quite a while (loading ~1,000 documents per second into Solr).

So after a few hours of processing I had my dataset and I was able to answer my first question pretty easily using Solr’s built-in statsComponent functionality.  For a description of this view the documentation on Solr’s documentation site.

Answering the questions

The average number of subjects per record in the DPLA = 2.99 with a standard deviation of 3.90. There are records with 0 subjects (1,827,276) and records with as many as 1,476 subjects (this record btw).

Answering question number two involved a small script to create a table for us, you will find that table below.

Hub Name min max count sum sumOfSquares mean stddev ARTstor 0 71 56,342 194,948 1351826 3.460083064 3.467168662 Biodiversity Heritage Library 0 118 138,288 454,624 3100134 3.287515909 3.407385646 David Rumsey 0 4 48,132 22,976 33822 0.477353943 0.689083212 Digital Commonwealth 0 199 124,804 295,778 1767426 2.369940066 2.923194479 Digital Library of Georgia 0 161 259,640 1,151,369 8621935 4.43448236 3.680038874 Harvard Library 0 17 10,568 26,641 88155 2.520912188 1.409567895 HathiTrust 0 92 1,915,159 2,614,199 6951217 1.365003637 1.329038361 Internet Archive 0 68 208,953 385,732 1520200 1.84602279 1.966605872 J. Paul Getty Trust 0 36 92,681 32,999 146491 0.356049244 1.20575216 Kentucky Digital Library 0 13 127,755 26,009 82269 0.203584987 0.776219692 Minnesota Digital Library 1 78 40,533 202,484 1298712 4.995534503 2.661891328 Missouri Hub 0 139 41,557 97,115 606761 2.336910749 3.023203782 Mountain West Digital Library 0 129 867,538 2,641,065 17734515 3.044321978 3.34282307 National Archives and Records Administration 0 103 700,952 231,513 1143343 0.330283671 1.233711342 North Carolina Digital Heritage Center 0 1,476 260,709 869,203 8394791 3.333996908 4.591774892 Smithsonian Institution 0 548 897,196 5,763,459 56446687 6.423857217 4.652809633 South Carolina Digital Library 0 40 76,001 231,270 1125030 3.042986277 2.354387181 The New York Public Library 0 31 1,169,576 1,996,483 6585169 1.707014337 1.648179106 The Portal to Texas History 0 1,035 477,639 5,257,702 69662410 11.00768991 4.96771802 United States Government Printing Office (GPO) 0 30 148,715 457,097 1860297 3.073644219 1.749820977 University of Illinois at Urbana-Champaign 0 22 18,103 67,955 404383 3.753797713 2.871821391 University of Southern California. Libraries 0 119 301,325 863,535 4626989 2.865792749 2.672589058 University of Virginia Library 0 15 30,188 95,328 465286 3.157811051 2.332671249

The columns are min which is the minimum number of subjects per record for a given Hub,  Minnesota Digital Library stands out here as the only Hub that has at least one subject for each of their 40,533 items.  The column max shows the highest number of subjects per record.  Two groups, The Portal to Texas History and North Carolina Digital Heritage Center have at least one record with over 1,000 subject headings. The column count is the number of records that each Hub had when the analysis was performed. The column sum is the total number of subject values for a given Hub,  note this is not the number of unique subject, that information is not present in this dataset. The column mean shows the average number of subjects per Hub and stddev is the standard deviation from this number.  The Portal to Texas History is at the top end of the average with 11.01 subjects per record and the Kentucky Digital Library is on the low end with 0.20 subjects per record.

The final question was if there were differences between the Service-Hubs and the Content-Hubs, that breakdown is in the table below.

Hub Type                                  min max count sum sumOfSquares mean stddev Content-Hub 0 548 5,736,178 13,207,489 84723999 2.302489393 3.077118385 Service-Hub 0 1,476 2,276,176 10,771,995 109293849 4.73249652 5.061612337

It appears that there is a higher number of subjects per record for the Service-Hubs over the Content-Hubs,  over 2x with 4.73 for Service-Hubs and 2.30 for Content-Hubs.

Another interesting number is that there are 1,590,456 records contributed by Content-Hubs, 28% of that collection that do not have subjects compared to 236,811 records contributed by Service-Hubs or 10% that do not have subjects.

I think individually we can come up with reasons that these numbers differ the ways they do. There are reasons to all of this, where did the records come from, were they generated as digital resource metadata records initially or using an existing set of practices such as AACR2 in the MARC format? How does that change the numbers? Are there things that the DPLA is doing to the subjects when they normalize them that change the way they are represented and calculated? I know that for The Portal to Texas History some of our subject strings are being split into multiple headings in order to improve retrieval within the DPLA and are thus inflating our numbers a bit in the tables above. I’d be interested to chat with anyone interested in this topic who has some “here’s why” explanations to the numbers above.

Hit me up on Twitter if you want to chat about this.

DPLA: Announcing our third class of DPLA Community Reps

planet code4lib - Tue, 2015-02-24 17:45

We’re extremely excited to introduce our third class of DPLA Community Reps–-volunteers who engage their local communities by leading DPLA outreach activities. We were thrilled with the response to our third call for applicants, and we’re pleased to now add another roster of nearly 100 new Community Reps to our outstanding first and second classes, bringing the total number of Reps to just over 200.

Our third class continues our success at completing the U.S. map, bringing local DPLA advocacy to all 50 states, DC, and a handful of international locations. Our Reps work in K-12 education, public libraries, state libraries, municipal archives, public history and museums, technology, publishing, media, genealogy, and many areas of higher education. This third class in particular solidifies our presence in states where previously we had only one rep, including Alabama, Arkansas, Iowa, Montana, Oklahoma. We also received a number of excellent applications from people working in museums, technology, genealogy, and K-12, so we’re excited to further these avenues for DPLA involvement and outreach. We’re eager to support this new classes’ creative outreach and engagement work, and we thank them for helping us grow the DPLA community!

For more detailed information about our Reps and their plans, including the members of the third class, please visit our Meet the Reps page.

The next call for our fourth class of Reps will take place in one year (January 2016).  To learn more about this program and follow our future calls for applicants, check out our Community Reps page.

LITA: 2015 LITA Forum – Call for Proposals, Deadline Extended

planet code4lib - Tue, 2015-02-24 17:18

The LITA Forum is a highly regarded annual event for those involved in new and leading edge technologies in the library and information technology field. Please send your proposal submissions here by March 13, 2015, and join your colleagues in Minneapolis .

The 2015 LITA Forum Committee seeks proposals for excellent pre-conferences, concurrent sessions, and poster sessions for the 18th annual Forum of the Library Information and Technology Association, to be held in Minneapolis Minnesota, November 12-15, 2015 at the Hyatt Regency Minneapolis. This year will feature additional programming in collaboration with LLAMA, the Library Leadership & Management Association.

The Forum Committee welcomes creative program proposals related to all types of libraries: public, school, academic, government, special, and corporate.

Proposals could relate to any of the following topics:

• Cooperation & collaboration
• Scalability and sustainability of library services and tools
• Researcher information networks
• Practical applications of linked data
• Large- and small-scale resource sharing
• User experience & users
• Library spaces (virtual or physical)
• “Big Data” — work in discovery, preservation, or documentation
• Data driven libraries or related assessment projects
• Management of technology in libraries
• Anything else that relates to library information technology

Proposals may cover projects, plans, ideas, or recent discoveries. We accept proposals on any aspect of library and information technology, even if not covered by the above list. The committee particularly invites submissions from first time presenters, library school students, and individuals from diverse backgrounds. Submit your proposal through http://bit.ly/lita-2015-proposal by March 13, 2015.

Presentations must have a technological focus and pertain to libraries. Presentations that incorporate audience participation are encouraged. The format of the presentations may include single- or multi-speaker formats, panel discussions, moderated discussions, case studies and/or demonstrations of projects.

Vendors wishing to submit a proposal should partner with a library representative who is testing/using the product.

Presenters will submit draft presentation slides and/or handouts on ALA Connect in advance of the Forum and will submit final presentation slides or electronic content (video, audio, etc.) to be made available on the web site following the event. Presenters are expected to register and participate in the Forum as attendees; discounted registration will be offered.

Please submit your proposal through http://bit.ly/lita-2015-proposal, by the deadline of March 13, 2015

More information about LITA is available from the LITA website, Facebook and Twitter. Or contact Mark Beatty, LITA Programs and Marketing Specialist at mbeatty@ala.org

David Rosenthal: Using the official Linux overlayfs

planet code4lib - Tue, 2015-02-24 16:00
I realize that it may not be obvious exactly how to use the newly-official Linux overlayfs implementation. Below the fold, some example shell scripts that may help clarify things.

An overlayfs mount involves four directories:
  • below, a directory that, accessed via the mount, will be read-only.
  • above, a directory that, accessed via the mount, will be read-write.
  • work, a read-write directory in the same file system as the above directory.
  • mount, the directory at which the union of the entries in below and above will appear.
Lets create an example of each of these directories:
% mkdir below above work mount
%
Now we populate below and above with some files:
% for A in one two three
> do
> echo "Content of below/${A}" >below/${A}
> done
% chmod a-w below/* below
% for A in four five six
> do
> echo "Content of above/${A}" >above/${A}
> done
% ls -la below above work mount
above:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 8
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..

work:
total 8
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
%
Now we create the overlay mount and see what happens:
% OPTS="-o lowerdir=below,upperdir=above,workdir=work"
% sudo mount -t overlay ${OPTS} overlay mount
% ls -la below above work mount
above:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 32
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%
The union of the files in the below and above directories has appeared in the mount directory. Now we add a file to the mount directory:
% echo "Content of mount/seven" >mount/seven
% ls -la below above work mount
above:
total 24
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 36
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%
The file seven has appeared in the mount directory, and also in the above directory. They have the same content:
% cat mount/seven
Content of mount/seven
% cat above/seven
Content of mount/seven
%
Now we write to a file that is in the below directory:
% cat mount/two
Content of below/two
% echo "New content of mount/two" >mount/two
% cat mount/two
New content of mount/two
% ls -la below above work mount
above:
total 28
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
-rw-r--r-- 1 pi pi 25 Feb 18 10:42 two

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 36
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 25 Feb 18 10:42 two

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%
A file two has appeared in the above directory which, when viewed through the overlay mount, obscures the file two in the below directory, which is still there with its original content:
% cat above/two
New content of mount/two
% cat below/two
Content of below/two
%
Now we remove a file from overlay mount directory that is in the below directory:
% rm mount/three
% ls -la below above work mount
above:
total 28
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
c--------- 1 pi pi 0, 0 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 25 Feb 18 10:42 two

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 32
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
-rw-r--r-- 1 pi pi 25 Feb 18 10:42 two

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%
The file three is still in the below directory, but a character special file three has appeared in the above directory that makes the file in the below directory inaccessible, a "whiteout". Now we remove the file in the below directory that we wrote to earlier:
% rm mount/two
% ls -la below above work mount
above:
total 24
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
c--------- 1 pi pi 0, 0 Feb 18 10:42 three
c--------- 1 pi pi 0, 0 Feb 18 10:42 two

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 28
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%

The file two in the above directory has been replaced with a whiteout. Now we undo the overlayfs mount:
% sudo umount mount
% ls -la below above work mount
above:
total 24
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
c--------- 1 pi pi 0, 0 Feb 18 10:42 three
c--------- 1 pi pi 0, 0 Feb 18 10:42 two

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 8
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
% cat below/two
Content of below/two
% cat above/seven
Content of mount/seven
%
The content of the read-only below directory is, as expected, unchanged. The data that was written to the overlay mount and not removed remains in the above directory. The data that was written to the overlay mount and subsequently removed is gone. The whiteouts remain.

Now we re-assemble the mount:
% sudo mount -t overlay ${OPTS} overlay mount
% ls -la below above work mount
above:
total 24
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six
c--------- 1 pi pi 0, 0 Feb 18 10:42 three
c--------- 1 pi pi 0, 0 Feb 18 10:42 two

below:
total 20
drwxr-xr-x 2 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 three
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 two

mount:
total 28
drwxr-xr-x 1 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 five
-rw-r--r-- 1 pi pi 22 Feb 18 10:42 four
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 one
-rw-r--r-- 1 pi pi 23 Feb 18 10:42 seven
-rw-r--r-- 1 pi pi 21 Feb 18 10:42 six

work:
total 12
drwxr-xr-x 3 pi pi 4096 Feb 18 10:42 .
drwxr-xr-x 6 pi pi 4096 Feb 18 10:42 ..
d--------- 2 root root 4096 Feb 18 10:42 work
%
The state of the mount returns.

Thom Hickey: Testing date parsing by fuzzing

planet code4lib - Tue, 2015-02-24 15:45


 Fuzz testing, or fuzzing, is a way of stress testing services by sending them potentially unexpected input data. I remember being very impressed by one of the early descriptions of testing software this way (Miller, Barton P., Louis Fredriksen, and Bryan So. 1990. "An empirical study of the reliability of UNIX utilities". Communications of the ACM. 33 (12): 32-44), but had never tried the technique.

Recently, however, Jenny Toves spent some time extending VIAF's date parsing software to handle dates associated with people in WorldCat.  As you might imagine, passing through a hundred million new date strings found some holes in the software.  While we can't guarantee that the parsing always gives the right answer, we would like to be as sure as we can that it won't blow up and cause an exception.

So, I looked into fuzzing.  Rather than sending random strings to the software, the normal techniques now used tend to generate them based on a specification or by fuzzing existing test cases.  Although we do have something close to a specification based on the regular expressions the code uses, I decided to try making changes to the date strings we have that are derived from VIAF dates.

Most frameworks for fuzzing are quite loosely coupled, typically they pass the fuzzed strings to a separate process that is being tested.  Rather than do that, I read in each of the strings, did some simple transformations on it and called the date parsing routine to see if it would cause an exception. Here's what I did for each test string, typically for as many times as the string was long.  At each step the parsing is called

  • Shuffle the string ('abc' might get replaced by 'acb')
  • Change the integer value of character up or down (e.g. 'b' would get replaced by 'a' and then by 'c')
  • Change each character to a random Unicode character

For our 384K test strings this resulted in 1.9M fuzzed strings. This took about an hour to run on my desktop machine.

While the testing didn't find all the bugs we knew about in the code, it did manage to tickle a couple of holes in it, so I think the rather minimal time taken (less than a day) was worth it, given the confidence it gives us that the code won't blow up on strange input.

The date parsing code in GitHub will be updated soon.  Jenny is adding support for Thai dates (different calendar) and generally improving things.

Possibly the reason I thought of trying fuzzing was an amazing post on lcamtuf's blog Pulling JPEGs out of thin air.  That post is really amazing.  By instrumenting some JPEG software so that his fuzzing software could follow code paths at the assembly level, he was able to create byte strings representing valid JPEG images by sending in fuzzed strings, a truly remarkable achievement. My feeling on reading it was very similar to my reaction reading the original UNIX testing article cited earlier.

--Th

 

Harvard Library Innovation Lab: Link roundup February 24, 2015

planet code4lib - Tue, 2015-02-24 14:21

This is the good stuff.

Sit Down. Shut Up. Write. Don’t Stop.

Hard work and working hard consistently. That’s the thing. Not romantic sparks of inspiration.

What makes us human? Videos from the BBC.

Fun, beautifully produced, short videos on what makes us human.

The Future of the Web Is 100 Years Old

Our current version of the Web (HTTP/HTML) is just one (far and away the most successful one) in a series of webs.

“Sea Rambler” Customized Bike by Geoff McFetridge

“I can learn small things that get me to points”

Boston Button Factory – Making Buttons Since 1872

17 pound, beautiful buttons. Want.

Open Knowledge Foundation: Open Data Camp UK: Bursting out of the Open Data Bubble

planet code4lib - Tue, 2015-02-24 13:23

“But nobody cares about Open Data”

This thought was voiced in many guises during last weekend’s Open Data Camp. Obviously not entirely true, as demonstrated by the 100+ people who had travelled to deepest Hampshire for the first UK camp of its kind, or the many more people involving themselves in Open Data Day activities around the world. However the sentiment that, while many of us are getting extremely excited about the potential of Open Data in areas including government, crime and health, the rest of the planet are ‘just not interested’ was very clear.

As a non-technical person I’m keen to see ways that this gap can be bridged.

Open Data Camp was a 2-day unconference that aimed to let the technical and making sit alongside the story-telling and networking. There was also lots of cake!

Open Data Camp t-shirts

Open Data Board Game

After a pitch from session leaders we were left with that tricky choice about what to go for. I attended a great session led by Ellen Broad from the Open Data Institute on creating an Open Data board game. Creating a board game is no easy task but has huge potential as a way to reach out to people. Those behind the Open Data Board Game Project are keen to create something informative and collaborative which still retains elements of individual competition.

In the session we spent some time thinking about what data could underpin the game: Should it use data sets that affect most members of the general public (transport, health, crime, education – almost a replication of the national information infrastructure)? Or could there be data set bundles (think environmental related datasets that help you create your own climate co-op or food app)? Or what about sets for different levels of the game (a newbie version, a government data version)?

What became clear quite early on was there was two ways to go with the board game idea: one was creating something that could share the merits of Open Data with new communities, the other was something (complex) that those already interested in Open Data could play. Setting out to create a game that is ‘all things to all people’ is unfortunately likely to fail.

Discussion moved away from the practicalities of board game design to engaging with ‘other people’. The observation was made that while the general public don’t care about Open Data per se they do care about the result it brings. One concrete example given was Uber which connects riders to drivers through apps, now with mainstream use.

One project taking an innovative approach is Numbers that Matter. They are looking to bypass the dominant demographic (white, male, middle class, young) of technology users and focus on communities and explore with them how Open Data will affect their well-being. They’ve set out to make Open Data personal and relevant (serving the individual rather than civic-level participant). Researchers in the project began by visiting members of the general public in their own environment (so taxi drivers, hairdressers,…) and spoke to them about what problems or issues they were facing and what solutions could be delivered. The team also spent time working with neighbourhood watch schemes – these are not only organised but have a ‘way in’ with the community. Another project highlighted that is looking at making Open Data and apps meaningful for people is Citadel on the Move which aims to make it easier for citizens and application developers from across Europe to use Open Data to create the type of innovative mobile applications they want and need.

The discussion about engagement exposed some issues around trust and exploitation; ultimately people want to know where the benefits are for them. These benefits needs to be much clearer and articulated better. Tools like Open Food Facts, a database of food products from the entire world, do this well: “we can help you identify products that contain the ingredient you are allergic to“.

Saturday’s unconference board

“Data is interesting in opposition to power”

Keeping with the theme of community engagement I attended a session led by RnR Organisation who support grassroots and minority cultural groups to change, develop and enhance their skills in governance, strategic development, operational and project management, and funding. They used the recent Release of Data fund, which targets the release of specific datasets prioritised by the Open Data User Group, to support the development of a Birmingham Data and Skills Hub. However their training sessions (on areas including data visualization, use of Tablau and Google Fusion tables) have not instilled much interest and on reflection they now realise that they have pitched too high.

Open Data understanding and recognition is clearly part of a broader portfolio of data literacy needs that begins with tools like Excel and Wikipedia. RnR work has identified 3 key needs of 3rd sector orgs: data and analysis skills; data to learn and improve activities; and measurement of impacts.

Among the group some observations were made on the use of data by community groups including the need for timely data (“you need to show people today“) and relevant information driven by community needs (“nobody cares about Open Data but they do care about stopping bad things from happening in their area“). An example cited was of a project to stop the go ahead of a bypass in Hereford, they specifically needed GIS data. One person remarked that “data is interesting in opposition to power“, and we have a role to support here. Other questions raised related to the different needs of communities of geography and communities of interest. Issues like the longevity of data also come in to play: Armchair Auditor is a way to quickly find out where the Isle of Wight council has been spending money, unfortunately a change in formats by the council has resulted in the site being comprimised.

Sessions were illustrated by Drawnalism

What is data literacy?

Nicely following on from these discussions a session later in the day looked at data literacy. The idea was inspired by an Open Data 4 Development research project led by Mark Frank and Johanna Walker (University of Southampton) in which they discovered that even technically literate individuals still found Open Data challenging to understand. The session ended up resulting a series of questions: So ‘what exactly is data literacy’? Is it a homogeneous set of expertise (e.g. finding data), or is the context everything? Are there many approaches (such as suggested in the Open Data cook book or is there a definitive guide such as the Open Data Handbook or a step by step way to learn such as through School of Data. Is the main issue asking the right questions? Is there a difference between data literacy and data fluency? Are there two types of specialism: domain specialism and computer expertise? And can you offset a lack of data expertise with better designed data?

The few answers seemed to emerge through analogies. Maybe data literacy is like traditional literacy – it is essential to all, it is everyone’s job to make it happen (a collaboration between parents and teachers). Or maybe it is more like plumbing – having some understanding can help you understand situations but then you often end up bringing in an expert. Then again it could be more like politics or PHSE – it enables you to interact with the world and understand the bigger picture. The main conclusion from the session was that it is the responsibility of everyone in the room to be an advocate and explainer of Open Data!

“Backbone of information for the UK”

The final session I attended was an informative introduction to the National Information infrastructure an iterative framework that lists strategically important data and documents the services that provide access to the data and connect it to other data. It intended as the “backbone of information” for the UK, rather like the rail and road networks cater for transport. The NII team began work by carrying out a data inventory followed by analysis of the quality of the data available. Much decision making has used the concept of “data that is of strategic value to country” – a type of ‘core reference data’. Future work will involve thinking around what plan the country needs to put into play to support this core data. Does being part of the NII protect data? Does the requirement for a particular data set compel release? More recently there has been engagement with the Open Data user group / transparency board / ODI / Open Knowledge and beyond to understand what people are using and why, this may prioritise release.

It seems that at this moment the NII is too insular, it may need to break free from consideration of just publicly owned data and begin to consider privately owned data not owned by the government (e.g. Ordnance Survey data). Also how can best practices be shared? The Local Government Association are creating some templates for use here but there is scope for more activity.

With event organiser Mark Braggins

Unfortunately I could only attend one day of Open Data Camp and there was way too much for one person to take in anyway! For more highlights read the Open Data Camp blog posts or see summaries of the event on Conferieze and Eventifier. The good news is that with the right funding and good will the Open Data Camp will become an annual roving event.

Where did people come from?

Terry Reese: MarcEdit 6 Update

planet code4lib - Tue, 2015-02-24 05:38

A new version of MarcEdit has been made available.  The update includes the following changes:

  • Bug Fix: Export Tab Delimited Records: When working with control data, if a position is requested that doesn’t exist, the process crashes.  This behavior has been changed so that a missing position results in a blank delimited field (as is the case if a field or field/subfield isn’t present.
  • Bug Fix: Task List — Corrected a couple reported issues related to display and editing of tasks.
  • Enhancement: RDA Helper — Abbreviations have been updated so that users can select the fields that abbreviation expansion occurs.
  • Enhancement: Linked Data Tool — I’ve vastly improved the process by which items are linked. 
  • Enhancement: Improved VIAF Linking — thanks to Ralp LeVan for pointing me in the right direction to get more precise matching.
  • Enhancement: Linked Data Tool — I’ve added the ability to select the index from VIAF to link to.  By default, LC (NACO) is selected.
  • Enhancement: Task Lists — Added the Linked Data Tool to the Task Lists
  • Enhancement: MarcEditor — Added the Linked Data Tool as a new function.
  • mprovements: Validate ISBNs — Added some performance enhancements and finished working on some code that should make it easier to begin checking remote services to see if an ISBN is not just valid (structurally) but actually assigned.
  • Enhancement: Linked Data Component — I’ve separated out the linked data logic into a new MarcEdit component.  This is being done so that I can work on exposing the API for anyone interested in using it.
  • Informational: Current version of MarcEdit has been tested against MONO 3.12.0 for Linux and Mac.

Linked Data Tool Improvements:

A couple specific notes of interest around the linked data tool.  First, over the past few weeks, I’ve been collecting instances where id.loc.gov and viaf have been providing back results that were not optimal.  On the VIAF side, some of that was related to the indexes being queried, some of it relates to how queries are made and executed.  I’ve done a fair bit of work added some additional data checks to ensure that links occur correctly.  At the same time, there is one known issue that I wasn’t able to correct while working with id.loc.gov, and that is around deprecated headings.  id.loc.gov currently provides no information within any metadata provided through the service that relates a deprecated item to the current preferred heading.  This is something I’m waiting for LC to correct.

To improve the Linked Data Tool, I’ve added the ability to query by specific index.  By default, the tool will default to LC (NACO), but users can select from a wide range of vocabularies (including, querying all the vocabularies at once).  The new screen for the Linked Data tool looks like the following:

In addition to the changes to the Linked Data Tool – I’ve also integrated the Linked Data Tool with the MarcEditor:

And within the Task Manager:

The idea behind these improvements is to allow users the ability to integrate data linking into normal cataloging workflows – or at least start testing how these changes might impact local workflows.

Downloads:

You can download the current version buy utilizing MarcEdit’s automatic update within the Help menu, or by going to: http://marcedit.reeset.net/downloads.html and downloading the current version.

–tr

SearchHub: Parsing and Indexing Multi-Value Fields in Lucidworks Fusion

planet code4lib - Mon, 2015-02-23 21:14
Recently, I was helping a client move from a pure Apache Solr implementation to Lucidworks Fusion.  Part of this effort entailed the recreation of indexing processes (implemented using Solr’s REST APIs) in the Fusion environment, taking advantage of Indexing Pipelines to decouple the required ETL from Solr and provide reusable components for future processing. One particular feature that was heavily used in the previous environment was the definition of “field separator” in the REST API calls to the Solr UpdateCSV request handler. For example: curl "http://localhost:8888/solr/collection1/update/csv/?commit=true&f.street_names.split=true&f.street_names.separator=%0D" --data-binary @input.csv -H 'Content-type:text/plain; charset=utf-8' The curl command above posts a CSV file to the /update/csv request handler, with the request parameters "f.aliases.split=true" and "f.aliases.separator=%0D" identifying the field in column “aliases” as a multi-value field, with the character “\r” separating the values (%0D is a mechanism for escaping the carriage return by providing the hexadecimal ASCII code to represent “\r”.)  This provided a convenient way to parse and index multi-value fields that had been stored as delimited strings.  Further information about this parameter can be found here. After investigating possible approaches to this in Fusion, it was determined that the most straightforward way to accomplish this (and provide a measure of flexibility and reusability) was to create an index pipeline with a JavaScript stage. The Index Pipeline Index pipelines are a framework for plugging together a series of atomic steps, called “stages,” that can dynamically manipulate documents flowing in during indexing.  Pipelines can be created through the admin console by clicking “Pipelines” in the left menu, then entering a unique and arbitrary ID and clicking “Add Pipeline” – see below. After creating your pipeline, you’ll need to add stages.  In our case, we have a fairly simple pipeline with only two stages: a JavaScript stage and a Solr Indexer stage.  Each stage has its own context and properties and is executed in the configured order by Fusion.  Since this post is about document manipulation, I won’t go into detail regarding the Solr Indexer stage; you can find more information about it here.  Below is our pipeline with its two new stages, configured so that the JavaScript stage executes before the Solr Indexer stage. The JavaScript Index Stage We chose a JavaScript stage for our approach, which gave us the ability to directly manipulate every document indexed via standard JavaScript – an extremely powerful and convenient approach.  The JavaScript stage has four properties:
  • “Skip This Stage” – a flag indicating whether this stage should be executed
  • “Label” – an optional property that allows you to assign a friendly name to the stage
  • “Conditional Script” – JavaScript that executes before any other code and must return true or false.  Provides a mechanism for filtering documents processed by this stage; if false is returned, the stage is skipped
  • “Script Body” – required; the JavaScript that executes for each document indexed (where the script in “Conditional Script,” if present, returned true)
Below, our JavaScript stage with “Skip This Stage” set to false and labeled “JavaScript_Split_Fields.” Our goal is to split a field (called “aliases”) in each document on a carriage return (CTRL-M).  To do this, we’ll define a “Script Body” containing JavaScript that checks each document for the presence of a particular field; if present, splits it and assigns the resulting values to that field. The function defined in “Script Body” can take one (doc, the pipeline document) or two (doc and _context, the pipeline context maintained by Fusion) arguments.  Since this stage is the first to be executed in the pipeline, and there are no custom variables to be passed to the next stage, the function only requires the doc argument.  The function will then be obligated to return doc (if the document is to be indexed) or null (if it should not be indexed); in actuality we’ll never return null as the purpose of this stage is to manipulate documents, not determine if they should be indexed. function (doc) { return doc; } Now that we have a reference to the document, we can check for the field “aliases” and split it accordingly.  Once its split, we need to remove the previous value and add the new values to “aliases” (which is defined as a multi-value field in our Solr schema.)  Here’s the final code: function (doc) { var f_aliases = doc.getFirstField("aliases"); if (f_aliases != null) { var v_aliases = f_aliases.value; } else { var v_aliases = null; } if (v_aliases != null) { doc.removeFields("aliases"); aliases = v_aliases.split("\r"); for (var i = 0; i < aliases.length; i++) { doc.addField('aliases',aliases[i]); } } return doc; } Click “Save Changes” at the bottom of this stage’s properties. Bringing It All Together Now we have an index pipeline – let’s see how it works!  Fusion does not tie an index pipeline to a specific collection or datasource, allowing for easy reusability.  We’ll need to associate this pipeline to the datasource from which we’re retrieving “aliases,” which is accomplished by setting the pipeline ID in that datasource to point to our newly-created pipeline. Save your changes and the next time you start indexing that datasource, your index pipeline will be executed. You can debug your JavaScript stage by taking advantage of the “Pipeline Result Preview” pane, which allows you to test your code against static data right in the browser. Additionally, you can add log statements to the JavaScript by calling a method on the logger object, to which your stage already has a handle.  For example: logger.debug("This is a debug message"); will write a debug-level log message to <fusion home>/logs/connector/connector.log.  By combining these two approaches, you should be able to quickly determine the root cause of any issues encountered. A Final Note You have probably already recognized the potential afforded by the JavaScript stage; Lucidworks calls it a “Swiss Army knife.”  In addition to allowing you to execute any ECMA-compliant JavaScript, you can import Java libraries – allowing you to utilize custom Java code within the stage and opening up a myriad of possible solutions.  The JavaScript stage is a powerful tool for any pipeline! About the Author Sean Mare is a technologist with over 18 years of experience in enterprise application design and development.  As Solution Architect with Knowledgent Group Inc., a leading Big Data and Analytics consulting organization and partner with Lucidworks, he leverages the power of enterprise search to enable people and organizations to explore their data in exciting and interesting ways.  He resides in the greater New York City area.

The post Parsing and Indexing Multi-Value Fields in Lucidworks Fusion appeared first on Lucidworks.

District Dispatch: Registration opens for the 16th annual Natl. Freedom of Information Day Conference

planet code4lib - Mon, 2015-02-23 20:38

Registration is now open for the 16th annual National Freedom of Information (FOI) Day Conference, which will be held on Friday, March 13, 2015, at the Newseum in Washington, D.C. The annual FOI Day Conference is hosted by the Newseum Institute’s First Amendment Center in cooperation with OpenTheGovernment.org and the American Library Association (ALA). The event brings together access advocates, government officials, judges, lawyers, librarians, journalists, educators and others to discuss timely issues related to transparency in government and freedom of information laws and practices.

Madison Award Awardee Patrice McDermott

This year’s program will feature a discussion of the first ten years of the “Sunshine Week” national open records initiative, presented by the Reporters Committee for Freedom of the Press and the American Society of News Editors. Additionally, the event will include a preview of a major reporting package from The Associated Press, McClatchy and Gannett/USA Today on a decade of open government activity. Miriam Nisbet, former director of the National Archives’ Office of Government Information Services, will address attendees at the FOI Day Conference.

During the event, ALA will announce this year’s recipient of the James Madison Award, which is presented annually to individuals or groups that have championed, protected and promoted public access to government information and the public’s right to know. Previous Madison Award recipients include Internet activist Aaron Swartz, Representative Zoe Lofgren (D-CA) and the Government Printing Office. ALA Incoming President Sari Feldman will present Madison Award this year.

The program is open to the public, but seating is limited. To reserve a seat, please contact Ashlie Hampton, at 202-292-6288, or ahampton[at]newseum[dot]org. The program will be streamed “live” at www.newseum.org.

The post Registration opens for the 16th annual Natl. Freedom of Information Day Conference appeared first on District Dispatch.

Nicole Engard: Bookmarks for February 23, 2015

planet code4lib - Mon, 2015-02-23 20:30

Today I found the following resources and bookmarked them on <a href=

  • codebender Online development & collaboration platform for Arduino users, makers and engineers

Digest powered by RSS Digest

The post Bookmarks for February 23, 2015 appeared first on What I Learned Today....

Related posts:

  1. Home Automation with Arduino/RaspberryPi
  2. NFAIS: Embracing New Measures of Value
  3. Another way to use Zoho

Pages

Subscribe to code4lib aggregator