AI in Library Research Platforms - Findings from EBSCO's Recent Beta Release

As technology reshapes information access, libraries are adapting to the digital age with AI at the forefront. AI-driven tools are revolutionizing library research platforms, enhancing search capabilities, and connecting users with the right information quickly. However, it's crucial to balance innovative AI features with thoughtful approaches.

EBSCO is embracing the power of generative AI in library research platforms to improve the effectiveness of research and transform research. We have also established guiding principles to drive our approach to AI in a responsible, ethical, and safe manner.

We have started to test and develop AI opportunities with the user interface of our library research platforms, EBSCO Discovery Service (EDS) and EBSCOhost. One such project is around a feature called ‘AI Insights.’

EBSCO AI Insights is a Generative AI feature that summarizes 3-5 key points of an article, helping users quickly assess its relevance. Accessible via a button on EBSCO’s interface for EBSCO Discovery Service and the EBSCOhost research platform, it complements abstracts and subject headings. Insights are marked as AI-generated, with a disclaimer urging users to verify their accuracy before use.

Between April and May of 2024, EBSCO released a beta version of AI Insights to a group of 50 libraries. A user experience research study with 20 end-users was also conducted, and in June focus group discussions were held with small groups of participating beta customers.

When asked about their overall impressions of the AI Insights tool:

• 20% of survey respondents were Very Positive
• 41% were Somewhat Positive
• 15% were Neutral
• 20% were Somewhat Negative
• 4% were Very Negative

For end-users, the user experience study found that:

• 96% of those identifying as efficiently seekers found AI Insights very helpful
• 85% of those identifying as research enthusiasts found AI Insights helpful
• 20% of those identifying as power researchers found AI Insights neutrally helpful

Additionally, an average of 85% of end users felt that AI Insights would have a significantly positive impact on user workflow.

Common themes in the positive feedback were that AI Insights complement abstracts, increase research efficiency, are easier to read than most abstracts which is a great way to help novice researchers begin to understand complex scholarly works. Some beta testers appreciated the ability to read AI Insights in English for articles written in other languages, and others saw an opportunity to use it as an information literacy skill builder for users in these library research platforms.

“I see an application for various early stages of the research process such as determining keywords, subtopics, narrowing and broadening topics, creating outlines, etc.” - Librarian Beta Tester

Survey respondents highlighted that AI Insights helps speed up the research process by offering a quick evaluation tool before reading full articles. This benefit was especially valued by both undergraduate students and advanced researchers, who appreciated the time saved by filtering out irrelevant articles.

“I would perform more research. I can quickly find which texts I need; I'm not wasting time reading texts I can't use ... it's making my research more productive and efficient. Instead of taking hours to find the perfect source, here it takes me minutes.” – Advanced Researcher

Some novice researchers were more inclined to apply insights directly to their research without first consulting the source, while some faculty were unsure of the benefits of Insights compared to abstracts.

It's a nice overview, I just don't know that it's more helpful than the summary that's typically included with the article. I'd need to click into the article to get more information on the size of the study, how many students it included, was it secondary or undergraduate, etc. – Faculty Member

This is supported in scholarship where findings suggest more information literacy for AI is needed. Librarians participating in the beta test across K-12 and academic libraries anticipated this risk, however this points back to a benefit often mentioned in survey feedback from multiple respondents: AI Insights provides a “teachable moment” in the early days of AI, for information literacy instruction.

There was a wide spectrum of tolerance for GenAI inaccuracies among the beta customers. Focus group participants from some research libraries said that it would not meet the needs of their users until AI is more consistently accurate, while others stated they could not turn AI Insights on until it was citable or reproducible. We observed the main AI Insights that had accuracy issues were those that cited other references as their main body of text, like book and literature reviews. The majority of survey respondents and focus group participants were not as concerned because they felt the risk of harm was low for their library users. Many focus group participants pointed to the previously mentioned benefit, that AI Insights provides information literacy instructors the opportunity to incorporate AI literacy in their teaching outcomes by teaching the occasional fallibility of GenAI technologies - a critical point that many students need to be taught.

Outside of improved accuracy for referential materials like reviews, some beta participants suggested that AI Insights should consistently generate the same number of insights for consistency, while others appreciated that the number varied according to the source text. Some wanted reproducible insights for easier verification in academic work, while others appreciated the flexibility to regenerate insights for better phrasing. Transparency in how AI Insights is implemented was also important, with librarians wanting clarity on why the feature is available for some articles but not others. This understanding would help them teach and use AI Insights more effectively.

EBSCO Product Managers gained valuable insights from beta feedback, revealing varying levels of comfort with AI tools in libraries. When AI Insights launches for EDS and EBSCOhost research platforms later this year, customers will choose whether to enable it – the default setting will be off. A key takeaway is the need to empower librarians to teach AI Literacy, which will be supported by a Connect article with FAQs, and materials at AI at EBSCO. EBSCO will continuously refine AI Insights, incorporating feedback and enhancement requests over time. AI Insights is expected to be available by the end of 2024, with more details to follow in the release notes.

EBSCO launched the EBSCO AI Beta Program in June 2024. We are publishing executive summaries like this one for each, and publishing an academic research article, focusing on how our work and findings fits into the larger scholarship of AI in Academia, in early 2025. All materials and more can be found by visiting AI at EBSCO.