Search analysis component
I am fascinated by the number of intranet components we have been developing recently for our clients.
Strangely no client has ever specifically asked for an analysis of user searching behavior – which puzzles me because if we are all concerned with user uptake then surely we should be monitoring, above all else – when they do and do not find what they are looking for.
This goes back to the ongoing discussion we are having at Claromentis about tagging, quality measurements and our continual challenge to deliver a very accessible information layer through a permission engine.
I am fascinated by the requirement here, and look forward to a debate about how to deliver this – even though no client has asked for it – but I still feel it should be fundamentally important to our client intranet project teams and sponsors.
I would like to propose that we develop a special searching monitor tool that does, as a minimum, the following :
• Logs into new tables all searches – additional to the current audit log – for special analysis – primarily so that they are not overwritten in the same way as general audit information must be.
• Delivers reports – probably via tag clouds /semantic analysis / stemming or some other mechanism that can allow for variations in the actual entered search string – on what users are searching for
• Saves the top 10 search results for that term, at the time the search was made
Now here is what I would like debate about – we need to track whether the user found what they were looking for. That of course, is far from easy.
My thoughts are the we should:
• Track whether they click on a returned result – they may not have been pleased with the results of the click but at least they saw something in the search return that was sufficiently positive they clicked on it.
• Compare this to users who did NOT click on any of the returned results.
• Over time compare this for different groups and roles of users looking for the same search term – in case we are not making the correct information available to the users who need it.
• Develop a concept of the “refined search” – so assume that if a user enters a new search within a few minutes – it is assumed to be another attempt to find the same information. Track this refined search term, compare that to the first – and a again report on the users that thought they had found what they are looking for after they had refined their search.
• Compare that with any tagging or metadata for the resulting file they clicked on.
I would be interested in ideas from the community as to whether this makes sense and of course from our team if we could deliver an engine that provides our clients with an accurate measure over time of what users are looking for, and whether they found it.
The added dimension over time of whether the success rate in specific searches actually improves would of course by a fundamental goal here.
