Based on the first iteration, the structure of the primary interface was designed. We conducted a small-scale user study in order to see information searchers' evaluation of the interface. Particularly we are interested in answering the following questions:
(1)
Is
the "cross-language retrieval" function explicit to users?
(2)
Is the "translation disambiguation"
function explicit to users?
(3)
Does the setting of all functional units in
the same interface create information overloading and/or disorientation? If so,
how to improve it?
Two
library school graduate students were recruited to participate the study. Right
after being briefly introduced to the basic functions of the interface, the
subjects were exposed to the interface. From a brief interview following users’
introduction to the interface, we found that:
(1)
Users
easily understand it is a cross-language document retrieval interface based on
the query input label and the search button. However, the search button should
be put on the same level as the query input area.
(2)
The
rest of the interface is confusing to users. Both subjects indicated they would
go ahead clicking the "search Chinese documents" button without
reviewing the disambiguation area and the results area;
(3)
Upon
the investigator's request, both subjects examined the disambiguation and
results areas. Neither of them could easily figure out functions of these
components;
(4)
After
the investigator explained the functions of each component, both subjects
thought that putting two "search" buttons and one
"retrieve" button in the same interface was confusing - they didn't
know which one should be used first;
(5)
In
addition, both subjects thought that instead of checkboxes, one text area
should be used to display multiple terms for disambiguation (together with
their translations) and another for the summary of retrieved documents.
Based
on these findings and follow-up team discussion, we agreed upon the following
modifications to the primary interface:
(1)
"Search
Chinese Documents" button is changed to "Translate/Refine Query"
button, and it is put on the same level as the query input area;
(2)
Translation
disambiguation area is split into two parts: the "Terms with multiple
translations" and "Terms that have no translation." We believe
that informing searchers of translation-missing query terms is very important.
However, we simply display these terms without further processing because we
want to focus on translation disambiguation;
(3)
A
"Final query" text area is added following the previous two parts. It
will be used to display the senses of query terms being sent to the IR engine.
The surface form of these query terms is still in English, but the deep form is
actually in Chinese. Next to this area is a "Search These Terms"
button. Clicking on it will send query terms in Chinese to IR system;
(4)
The
list of retrieved articles will be displayed in a text area instead of as a
checkbox group. A "Display Summary" button will be added so that
summaries of selected documents will be displayed. Since text areas are used for translation selection and document
selection, we decided to use "highlighting" lines as the method of
selection. Weight of importance for query terms, as it is stated in
Specification 1, will not be displayed. However, query terms will be displayed
in decreasing order of importance.