In my list of the Five (5) Pros & Cons of OvidSP, one caught the eye of rising health librarian,
Jeff Mason, from Saskatchewan:
Jeff: “I’m wondering why you think basic search results won’t be reproducible. …if a search is completely documented (exactly what you typed in, what options you had checked off etc.) and OVID’s algorithms stay the same over time (questionable) the search could easily be reproduced.”
Questionable is right, Jeff. OVID’s basic search in OvidSP is not the same as it was in the OVID Gateway. However, irreproducibility is not a significant problem generally, unless we are asked as health librarians to perform (and document) searches for grants, clinical trials or other types of systematic reviews.
There are a number of tools that wreak havoc on reproducibility: all search engines, Google scholar, OvidSP Basic Search (and, indeed, any tool or algorithm that interprets a search based on links or term relevance). Read the section on how OvidSP’s feature How Relevancy is Calculated works. (Librarians have disliked features like this since they were introduced for books@OVID several years ago.)
To me, this feature is heuristic (and iterative) but not reproducible over time.
2 replies on “Irreproducibility? – OvidSP Basic search”
Alright, I see what you’re getting at here. If basic search works in the nebulous way Google does, and my understanding now is that it in fact does, I will agree that results from the search cannot be reproduced. The search can be repeated but, over time, it is possible the original set or results will change (I realise results will change over time because there will be new content).
The more I think about and work with the basic search the more I think OVID got something right. From a clinical point of view, I know physicians want answers and they want them quickly (this is why many of them will choose Google or UpToDate first). The ability of the basic search to take a clinical question and translate it into results that SEEM to answer the question, is a definite plus. This scenario has to be better than searching Google – at least you know what the content of the database is – and might draw users to our resources in a more obvious way (unlike when they find an NEJM article in Google and don’t understand that the only reason they have access to the full text is because WE subscribe to it for them).
I’m not sure how I want to approach basic search with my users, but it definitely needs to be addressed as a useful tool.
Hi Jeff,
I agree with you, and your research and presentations at CHLA/ABSC with Google have paid off clearly.
Basic search in OvidSP will confound our users because search results will be stable for some topics over time, and unstable for fast-moving topics. Since the OVID algorithms are proprietary and patented (like Google’s), we don’t really know much more than what they are telling us about how they weight certain search terms.
As the knowledge-based in medicine changes, so will our results in OvidSP. Just like Google
Dean