Fixing the Broken Search Process

Jane Reed

According to the market research firm IDC, knowledge workers spend almost nine hours each week searching for information needed for their work. This includes scientists searching for knowledge to advance the development of new drugs, or help identify life-preserving gene therapies, or ensure compliance with the latest regulatory rules. While some scientists may have strategies to manage wading through mountains of documents to find relevant insights, imagine how much more productive they could be if their searches were less time-consuming, more efficient and cost-effective.

Overwhelming amounts of data

Over the past few decades, the amount of available life science and healthcare information has grown dramatically, paralleling the vast proliferation of high-throughput biology, digital technologies, and online journals. Both life sciences and healthcare sectors encourage the sharing of scientific knowledge and collaboration among researchers. Every year, millions of new documents are published in the form of academic research, patent applications, clinical trial findings, and more. The sheer volume of available data can be overwhelming, even for scientists with very narrow fields of study.

Scientists often struggle to find the answers they need, despite the vast amount of available data. Part of the challenge is the fact that as much as 80 percent of information is stored in unstructured formats that are difficult to search and analyze using traditional manual methods. By leveraging artificial intelligence-based (AI) technologies such as natural language processing (NLP), users can more easily search through unstructured text sources and find high value, relevant results.

Technical experts vs. self-service queries

Typically, NLP searches require the expertise of technical users to build the queries and extract data insights. While this method can yield excellent results, the technical experts within many organizations are stretched very thin and unable to quickly address the needs of end users. This means end-users have to resort to their own searches using standard search engines.

Basic search engines, however, are not well-suited for scientific searches. Many lack comprehensive ontologies for key domain concepts, and don’t offer NLP-based pattern matching capabilities to effectively surface key relationships between scientific concepts. Users often end up with results that fail to provide direct answers to specific questions. Instead they receive long lists of hyperlinks to full documents with few details about what each document provides in terms of relevant content. Scientists must spend additional hours reading through pages and pages of documents that may or may not provide the answers they’re seeking.

Fixing the broken search process

The inefficiencies of today’s search processes limit the ability of life science organizations to increase productivity, speed product time to market or find hidden nuggets of valuable data. These companies could hire or train new information experts, though it’s an expensive and time-consuming option. Alternatively, organizations could empower end-users and equip them with more effective self-service AI-powered search tools that deliver quick and reliable answers to their searches.  An intuitive interface that provides access to powerful NLP algorithms and results would allow end-users to use ontologies and find, for example, the key relationships in text that are critical to differentiating between a search for drugs that treat hypertension versus drugs that cause hypertension. The technical experts would then have more time to concentrate on developing the more complex, time-consuming searches.

To make the most positive impact, self-service AI-based search applications must be intuitive and easy-to-use and have the ability to query unstructured text from a broad set of knowledge resources. The tools must also deliver answers - and not just documents - and provide deep insights from a single search.

The use of AI technologies in end-user applications is growing in retail, banking, travel and other sectors. But until now, life sciences organizations have lacked adequate self-service AI-tools to facilitate effective searches more broadly across their research teams. By addressing this gap, life science companies have the opportunity to spend less time searching and more time advancing their organizational goals.  

Jane Reed is Head of Life Science Strategy at Linguamatics.