Home » Blog

Why and where we search…

9 October 2007 17 Comments

This quote is grabbed from a comment by Jean-Claude Bradley at bbgm in reply to my comment on Deepak’s post on my post on…. anyway my original comment was that our Wiki review would not be indexed on Google Scholar which is where people might go for literature searches

Jean-Claude:

Getting on Google Scholar is something on my list to look into – if anyone knows how to do it please let us know. But from our Sitemeter tracking on UsefulChem it is clear that scientists are using Google to search for (and find) actionable scientific information.

Now this is an interesting point and it mirrors what I do. Jean-Claude has established that a lot of the ‘new’ traffic coming to UsefulChem comes from Google searches for specific information. Specific molecules in many cases but also spectra and other experimental data. If I’m looking for information, or a resource, scientific or otherwise, I will do a generic Google search for the most part (the most successful recent one was for ‘sticky apple pudding‘ – the result was very good indeed – see the Waitrose.com link).

But if I’m looking for scientific literature I will go to PubMed or sometimes to Google Scholar if I’m getting frustrated. I only ever use WOK for citation based searching (i.e. who cited a paper) or on the rare occasions when I’m looking for material that is not indexed in PubMed. Partly this is because I really like the ‘related items’ tab in PubMed. But what strikes me is that in my mind I have obviously divided these classes of searches up into two different things: ‘information/resources’ and ‘literature’. I bet this correlates quite strongly with both age and with scientific field. Do others out there think of these things as different or as all part of a continuum of information?

I recently saw a talk on a ‘Research Information Centre’ being developed by Microsoft, a sort of portal for handling research projects and all the associated information. This is at an early stage of development but one of the features they were working on was an integrated search where you could add and subtract various items (PubMed, WOK, Google, GoogleScholar, and various toll access info sources as well). GoogleScholar could do this well. So as Jean-Claude says above. Anybody got any contacts with the developers? We could just talk to them…


  • Because Google links to a lot of the “traditional peer-reviewed literature”, you can often get what you are looking for without even going to Google Scholar. Some searches just won’t pull up too much else except for scientifically relevant results. For example, take a Google search that someone used this morning to find a post on UsefulChem looking for pKa of sulfuric acid. Who else but a chemist would bother to put that on a website?

    http://www.google.nl/search?hl=en&client=firefox-a&rls=org.mozilla:en-US:official&hs=oN9&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=pka%20sulfuric%20acid&spell=1

  • Because Google links to a lot of the “traditional peer-reviewed literature”, you can often get what you are looking for without even going to Google Scholar. Some searches just won’t pull up too much else except for scientifically relevant results. For example, take a Google search that someone used this morning to find a post on UsefulChem looking for pKa of sulfuric acid. Who else but a chemist would bother to put that on a website?

    http://www.google.nl/search?hl=en&client=firefox-a&rls=org.mozilla:en-US:official&hs=oN9&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=pka%20sulfuric%20acid&spell=1

  • Timo Hannay gave me the contact information with the Scholar team. If I get a response, will let you know. Used your projects as examples of what needs to be supported. I am sure they’re watching

  • Timo Hannay gave me the contact information with the Scholar team. If I get a response, will let you know. Used your projects as examples of what needs to be supported. I am sure they’re watching

  • For a general literature search, it’s often better to use a range of sources (say one of the traditional databases plus Google Scholar) as their different approaches can find different things. A meta-search of several databases might be the answer, but if they all hold the same data, will it really find more articles? As you say, it’s a particular feature of the interface of PubMed that you like, not the size of its index.

    Could Google Scholar really be expanded to search the likes of WoK, given that they are restricted to academic users with Athens logins? I’m not familiar with the “Research Information Centre”, but I do know that any project that Microsoft is involved in doesn’t have a place in making science more open – it might improve users’ ability to search, but only while lining Bill Gates’s pockets at the same time.

  • For a general literature search, it’s often better to use a range of sources (say one of the traditional databases plus Google Scholar) as their different approaches can find different things. A meta-search of several databases might be the answer, but if they all hold the same data, will it really find more articles? As you say, it’s a particular feature of the interface of PubMed that you like, not the size of its index.

    Could Google Scholar really be expanded to search the likes of WoK, given that they are restricted to academic users with Athens logins? I’m not familiar with the “Research Information Centre”, but I do know that any project that Microsoft is involved in doesn’t have a place in making science more open – it might improve users’ ability to search, but only while lining Bill Gates’s pockets at the same time.

  • Hi Jonathon,

    To be fair to the microsoft guys I think I am right in saying that they intend this to be an open source product. Obviously they intend to make money on it in some way, primarily I think through ‘value added’ services.

    I think we are less interested in having Google Scholar search WOK which as you say wouldn’t add anything and more in having it access a wider range of scientific info; primary data online (we are talking largely about online lab books here) but also other useful resources as well. So for instance the search Jean-Claude mentions above doesn’t give such useful results on Scholar, whereas Google finds several helpful sites.

  • Hi Jonathon,

    To be fair to the microsoft guys I think I am right in saying that they intend this to be an open source product. Obviously they intend to make money on it in some way, primarily I think through ‘value added’ services.

    I think we are less interested in having Google Scholar search WOK which as you say wouldn’t add anything and more in having it access a wider range of scientific info; primary data online (we are talking largely about online lab books here) but also other useful resources as well. So for instance the search Jean-Claude mentions above doesn’t give such useful results on Scholar, whereas Google finds several helpful sites.

  • In my experience, custom search engines engines are the way to go at this point (essentially a meta search)

    While I am a Google-phile, I believe Cameron is right. Microsoft has been open sourcing its escience solutions and this would fit into that trend

  • In my experience, custom search engines engines are the way to go at this point (essentially a meta search)

    While I am a Google-phile, I believe Cameron is right. Microsoft has been open sourcing its escience solutions and this would fit into that trend

  • I’m all for using a plain Google search for anything, as I once said on a slightly different but not unrelated matter: http://jonathan.rawle.org/2006/07/07/web-based-services-should-be-free/

    If your lab book was a simple blog, and it pinged Technorati and Google Blog Search, anything you wrote would immediately be searchable by everyone. No need for a special index. One might need to use three tabs on Google to find everything (primary, refereed, and misc pages on the web). I realise that’s a simplistic view: from your previous postings, there are many issues surrounding publishing lab books for everyone to see. But from a technological point of view, using Google is a good solution.

  • I’m all for using a plain Google search for anything, as I once said on a slightly different but not unrelated matter: http://jonathan.rawle.org/2006/07/07/web-based-services-should-be-free/

    If your lab book was a simple blog, and it pinged Technorati and Google Blog Search, anything you wrote would immediately be searchable by everyone. No need for a special index. One might need to use three tabs on Google to find everything (primary, refereed, and misc pages on the web). I realise that’s a simplistic view: from your previous postings, there are many issues surrounding publishing lab books for everyone to see. But from a technological point of view, using Google is a good solution.

  • Jonathan,
    Our lab notebook is on Wikispaces, with summaries on Blogger and both of these get indexed efficiently by Google. It is up to us to make sure that our experiments are tagged appropriately (with InChIs and InChIKeys for molecules we use for example).

    Because Google is the default search for most users, I think that the “search problem” for us is essentially solved. If all scientists used a Google-indexed location for their work it would be even better. But there is some perception that the quality of results is better on Google Scholar so I was curious about getting UsefulChem on there to reach another audience. I’m hoping Deepak will know more about that soon.

  • Jonathan,
    Our lab notebook is on Wikispaces, with summaries on Blogger and both of these get indexed efficiently by Google. It is up to us to make sure that our experiments are tagged appropriately (with InChIs and InChIKeys for molecules we use for example).

    Because Google is the default search for most users, I think that the “search problem” for us is essentially solved. If all scientists used a Google-indexed location for their work it would be even better. But there is some perception that the quality of results is better on Google Scholar so I was curious about getting UsefulChem on there to reach another audience. I’m hoping Deepak will know more about that soon.

  • It is probably too early but if eventually enough people start having open notebooks then we could make a case with pubmed that alongside peer reviewed papers there could be a tab or an option to also present results from these notebooks.

  • It is probably too early but if eventually enough people start having open notebooks then we could make a case with pubmed that alongside peer reviewed papers there could be a tab or an option to also present results from these notebooks.

  • Pingback: Science in the open » The Soton Lab Blog Book US Tour()