From: "Saved by Windows Internet Explorer 7" Subject: Multilingual Information Discovery and AccesS (MIDAS): A Joint ACM DL'99 / ACM SIGIR'99 Workshop Date: Wed, 8 Apr 2009 20:40:54 -0400 MIME-Version: 1.0 Content-Type: multipart/related; type="text/html"; boundary="----=_NextPart_000_0000_01C9B88A.4FF71400" X-MimeOLE: Produced By Microsoft MimeOLE V6.0.6001.18049 This is a multi-part message in MIME format. ------=_NextPart_000_0000_01C9B88A.4FF71400 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Location: http://www.dlib.org/dlib/october99/10oard.html Multilingual Information Discovery and AccesS = (MIDAS): A Joint ACM DL'99 / ACM SIGIR'99 Workshop
3DStories=20

D-Lib Magazine
October 1999

Volume 5 Number 10

ISSN 1082-9873

Multilingual Information Discovery and AccesS = (MIDAS)

A Joint ACM DL'99 / ACM SIGIR'99 Workshop

3D"blue

Douglas Oard
University of=20 Maryland
oard@glue.umd.edu=20 		Carol Peters
Consiglio = Nazionale delle=20 Ricerche
carol@iei.pi.cnr.it= =20
 =20
Miguel Ruiz
University of = Iowa
mruiz@cs.uiowa.edu=20 		Robert Frederking
Carnegie = Mellon=20 University
ref@cs.cmu.edu =
 =20
Judith Klavans
Columbia=20 University
klavans@cs.columbia.edu=20 		P=E1raic Sheridan
TextWise = LLC
paraic@textwise.com= =20

=20

Introduction

The development of technologies that enable access to information = regardless of geographic or language barriers is a key factor for = truly=20 global sharing of knowledge. Users of internationally distributed=20 information networks need tools that allow them to find, retrieve = and=20 understand relevant information, in whatever language and form it = may have=20 been stored. This pressure has created a convergence of interests = from=20 diverse research communities around a topic that we refer to as = Multilingual=20 Information Discovery and AccesS (MIDAS). This multidisciplinary = enterprise=20 has drawn the attention of researchers with backgrounds in fields = such as=20 information retrieval, natural language processing, machine = translation,=20 summarization, speech processing, document image understanding, and=20 human-computer interaction. Sharing ideas, approaches and = methodologies=20 between those communities should accelerate research progress, but = crafting=20 effective cross-community venues is a challenging task. The = conjunction of=20 the ACM conferences on digital libraries and information retrieval = provided=20 a perfect opportunity to assemble a group with a broad range of=20 perspectives. The aim of the MIDAS Workshop, held in Berkeley on = Saturday=20 August 14, 1999, was thus not only to provide a forum in which = researchers=20 could describe ongoing work and present new results, but also to see = the=20 issues from other perspectives and to discuss unresolved questions = that=20 might be best addressed with approaches that draw from more than one = research tradition. The workshop was attended by about 50 = participants from=20 academic, government and industrial institutions in North America, = Europe=20 and Asia. It was co-chaired by Douglas Oard (College of Library and=20 Information Services, University of Maryland, USA) and Carol Peters=20 (Istituto di Elaborazione della Informazione, National Research = Council,=20 Italy).

This was not the first meeting to seek to bring together = researchers from=20 different communities around this topic. A first workshop on = cross-language=20 information retrieval -- at SIGIR =9296 -- brought together people = with=20 experience in information retrieval, machine translation, = computational=20 linguistics and digital libraries (Gref= enstette,=20 1998). Later that year, the U.S. Defense Advanced Research Projects = Agency=20 organized a similar meeting between people with experience in = machine=20 translation and speech processing. Participants from each of those = workshops=20 came together at Stanford for an AAAI symposium in 1997 (Hull = and=20 Oard, 1997). Two working groups have been jointly funded by the = U.S.=20 National Science Foundation and the European Commission to examine=20 multilingual information access and management from the perspectives = of=20 digital libraries and computational linguistics (Klavans = and=20 Sch=E4uble, 1998; Hovy et = al.,=20 1999), and presentations on aspects of the problem now routinely = appear at=20 major conferences on digital libraries, information retrieval, = machine=20 translation and computational linguistics.

In opening the workshop, Carol Peters explained that the MIDAS = workshop=20 aimed to take the discussion a step further by:

  • exploring the issues introduced by multilingual information = access in=20 media other than character-coded text,

  • identifying the intersection between cross-language = information=20 retrieval and multilingual metadata, and

  • examining the broad range of evaluation issues raised by = multilingual=20 systems.

Addressing such issues requires a mixture of theory, experience = and=20 insight, so the workshop schedule was planned to provide room for = all three.=20 Each of the first four sessions was built around one of the three = questions=20 identified above (the first point being the focus of two sessions). = Each=20 session included a keynote to help set the stage, and three sessions = included a research presentation illustrating some aspect of the = state of=20 the art. The goal was to prime the discussion with one or two talks = on each=20 subject, but to preserve as much time as possible for an exchange of = opinions between the panel and the audience. The final session = consisted of=20 a panel discussion in which representatives from governmental and=20 nongovernmental organizations talked about the directions that the = global=20 research agenda should take.

In this report, we summarize the key issues raised in each = session.=20 Miguel Ruiz (University of Iowa, USA) and Anne Diekema (TextWise = LLC, USA)=20 served as rapporteurs. The workshop program, position papers, = research=20 papers, and slides from most of the presentations can be found at = <http://www.clis.u= md.edu/conferences/midas.html>.

Multilingual Access to Electronic Texts

The first session, led by session chair Judith Klavans (Center = for=20 Research on Information Access, Columbia University, USA), addressed = topics=20 ranging from historical perspectives on developments in = cross-language=20 information retrieval (CLIR) to user needs. The discussion was = provocative,=20 and included questions on the role of different tasks in CLIR, the = range of=20 user abilities, and the need for cross-fertilization between fields. =

The keynote speaker in this session, Eduard Hovy (Information = Sciences=20 Institute, University of Southern California, USA), focused on the = need for=20 integration between the machine translation (MT) and information = retrieval=20 (IR) communities in order to tackle the multilingual retrieval = problem. He=20 identified three generations of multilingual systems. In generation = 0,=20 multilingual systems were built by performing MT and IR in a sort of = pipelined process: CLIR =3D MT | IR. Generation 1, which includes = systems that=20 are being built today, interleaves the functionality of both = communities=20 more tightly to overcome their respective weaknesses, thus building = more=20 efficient and robust systems. The generation 2 systems that Hovy = envisions=20 will build new modules that mix MT and IR functions in such a way = that the=20 border between them cannot be easily recognized. Hovy identified = four=20 sources of inaccuracy in CLIR: query underspecification, query = translation,=20 unfocused full document retrieval, and full document translation. He = outlined two potential solutions: (1) have the user help when = translating=20 the query and selecting relevant documents, and (2) have IR and MT=20 compensate for each other=92s weaknesses. In Hovy=92s opinion, IR is = unfocused=20 because it seeks to provide documents rather than answers. In this = respect,=20 Natural Language Processing (NLP) techniques can help by analyzing = sentence=20 structure and context. Hovy dedicated the major part of his talk to=20 discussing some of the important steps in CLIR and showing how = component=20 technologies could be mixed and matched.

The keynote was followed by a research paper presented by William = Ogden=20 (Computing Research Lab, New Mexico State University, USA). Ogden = described=20 experiences at NMSU with the development and evaluation of systems = for=20 cross-language retrieval, summarization and translation. He posed = two=20 fundamental questions concerning user/system interaction: (1) who = are the=20 users? and (2) what are their goals? Ogden suggested two = illustrative=20 scenarios to provide a framework for considering these questions: = (1) an=20 advanced foreign language reader, who may simply need help with=20 cross-language query formulation, and (2) a monolingual user who = wishes to=20 pose a query in a single language and then understand the answers = contained=20 in any retrieved document regardless of language. He used this = discussion of=20 user needs in terms of the tasks involved to highlight the need for=20 task-based evaluation. Ogden then described results from several = experiments=20 that evaluated the usefulness of different kinds of visual = presentations for=20 two tasks: (1) helping users judge the relevance of documents they = cannot=20 read, and (2) helping users reformulate a query. He showed a = document=20 thumbnail view of retrieval results in which a user can see the = spatial=20 distribution of key terms in each document at a glance. The goal of = new=20 presentation technologies of this type is to reduce the time spent = in=20 repeating and refining the viewing of results. Results of evaluation = showed=20 that users were able to recognize relevant documents using = thumbnails alone,=20 rather than titles. They could retrieve useful information just by = knowing=20 where the key terms in a document occurred. Ogden concluded with a=20 demonstration of a summarization system in which named entities are = used to=20 help select the best sentences.

Mun-Kew Leong (Kent Ridge Digital Labs, Singapore) made the first = of the=20 two short panel presentations in this session, drawing an analogy = between=20 multilingual information and database normalization. He suggested = that=20 several of the steps involved in CLIR could be seen as = normalization, for=20 example: (1) normalize the queries -- by translating the keywords = (2)=20 normalize the data -- by translating the collection, or (3) = normalize the=20 index space -- through techniques such as Cross-Language Latent = Semantic=20 Indexing. The overarching message was that the IR community might = learn from=20 the database community, in which the normalization process is a key = to less=20 ambiguous semantics.

W. Bruce Croft (Center for Intelligent Information Retrieval, = University=20 of Massachusetts, USA), summarized some research results from his = group. He=20 reported that dictionary-based query translation, combined with some = low=20 level but robust disambiguation, can be a successful CLIR approach,=20 achieving results close to monolingual IR (90% of monolingual = retrieval=20 effectiveness by commonly used measures). A comment was made = regarding the=20 difficulty of obtaining useful machine-readable dictionaries. Croft = observed=20 that rapid incorporation of new languages for which limited language = resources exist will be a key issue in this regard. He suggested = other=20 priorities for CLIR research as well, including summarization and=20 visualization of results, claiming that such topics are even more = important=20 for CLIR than in monolingual IR since there can be wide variation in = user=20 abilities and user needs.

Much of the discussion in the first session focused on the = differing=20 perspectives of the IR community, which has found statistical = methods to be=20 particularly useful, and the NLP community, which has embraced a = mixture of=20 both statistical and rule-based symbolic techniques. It was claimed = that the=20 integration of NLP and IR does not merely add complexity, but rather = multiplies it because present techniques in each field are quite = "fragile,"=20 requiring extensive tuning to perform well in specific applications. = It was=20 thus generally agreed that continued communication between the two=20 communities will be needed if we are to make progress in = multilingual=20 information discovery and access. There was, however, less agreement = regarding Hovy=92s claim that substantial unrealized potential = exists for=20 exploiting NLP in IR systems -- the crucial question that was posed = is=20 "where is the evidence that NLP techniques beyond those presently = used=20 (e.g., stemming) would actually improve retrieval?" The tenability = of manual=20 query refinement was also challenged by an observation that systems = that=20 rely on such approaches have not always been well received in = monolingual=20 applications.

Handling Multilingual Speech, Document Images, and Video = OCR

The second session focused on access to multilingual information = in=20 formats other than character-coded text. An underlying theme, posed = by=20 session chair Robert Frederking (Language Technologies Institute, = Carnegie=20 Mellon University, USA), was "does information access using = modalities other=20 than electronic text involve anything beyond retrieval from degraded = text?"=20 This led to a discussion of issues including the cost of adding new=20 languages to a multilingual system, automatic topic labeling for = foreign=20 language video, the state of the art in document image processing=20 technology, and CLIR from recorded speech.

The keynote speaker was Peter Sch=E4uble (Eurospider Information=20 Technology, Switzerland). Sch=E4uble presented an overview of the = processing=20 stages and resources that are needed to produce a state-of-the-art=20 multilingual information retrieval system. His conclusion was that = system=20 development using the present methodology would require about ten=20 person-years of effort for each additional language, allocated as = follows:=20 64 person-months for resource acquisition and processing, 32 = person-months=20 for software development, and 18 person-months for system tuning. = Covering=20 the thirty languages proposed for the U.S. Translingual Information=20 Detection, Extraction and Summarization (TIDES) program would thus = require a=20 total of 300 person/years if we were to keep working in the same way = that we=20 are now. Sch=E4uble=92s presentation stimulated considerable = discussion, both=20 about cost factors that might grow less rapidly as new languages are = added=20 (evaluation, for example) and about which components would be most = amenable=20 to lower-cost techniques.

The research paper by Alexander Hauptmann (School of Computer = Science,=20 Carnegie Mellon University, USA) explained how automatic topic = labeling for=20 multilingual broadcast news has been implemented in the Informedia = project.=20 Informedia has long included topic-labeling for English video, and = Hauptmann=20 explained how this capability has been extended to foreign-language = stories.=20 English topic labels were thought to be more useful than title = translation=20 for identifying relevant stories in a foreign language because title = translations are error-prone due to their telegraphic style and the=20 resulting lack of adequate context to constrain word choice. = Hauptmann=20 explained that the probabilities of successful speech recognition, = machine=20 translation and information retrieval combine in a multiplicative = way, but=20 that exploitation of redundant information in different modalities=20 (including metadata) can help to overcome this limitation. He gave a = brief=20 demonstration of the system, showing results obtained using Croatian = television news stories. As a final note, Hauptmann observed that = the=20 k-nearest-neighbors approach that is presently employed may not be = well=20 suited for use with topics that change significantly over time = because the=20 training data would need to be updated periodically.

The first panelist to speak was Henry Baird (Information Sciences = and=20 Technologies Lab, Xerox Palo Alto Research Center, USA). Baird = described the=20 state-of-the-art in document image processing technology, including = language=20 recognition, two-dimensional layout analysis, recognition of = document=20 structure, and Optical Character Recognition (OCR). He pointed out = that=20 while OCR can perform well, it is subject to unpredictable, = catastrophic=20 failure, and illustrated the point by describing the disappointing = results=20 of early experience with the integration of off-the-shelf OCR and = machine=20 translation systems in the US Army Research Laboratory=92s FALCON = system. He=20 observed that combining OCR with information retrieval seems to work = better=20 because the "bag of words" approach used in IR is quite robust. He = reported=20 that monolingual OCR systems are presently available for a = relatively small=20 number of languages and that, with the exception of a system for = English and=20 Japanese, there is almost no ability to handle mixed-language = documents.=20 Rapidly retargeting an OCR system to a new language is presently = quite=20 challenging, largely due to the monolithic structure of current OCR=20 technology in which language-specific constraints are deeply = enmeshed with=20 other code. On the brighter side, language identification in = document images=20 turns out to be a quite tractable problem, much easier and more = effective=20 than might initially have been expected.

George Doddington (National Institute for Standards and = Technology, USA)=20 described the U.S. Topic Detection and Tracking (TDT) initiative. = TDT has=20 adopted an evaluation-driven research paradigm in which participants = work=20 with broadcast news (newswire stories and automatic transcriptions = of radio=20 and television broadcasts), attempting to answer the question = "What=92s New?"=20 (detection) and to track topics over time. This year, TDT-3 is using = English=20 and Mandarin Chinese data for five tasks:

  • Segmentation: deciding where a news story starts or ends in an = audio=20 stream.

  • Detection: given a set of stories, cluster those that describe = the=20 same event.

  • Tracking: given a few on-topic stories and a stream of future = stories,=20 find all those that are relevant to the topic.

  • First story detection: find the first story that reports each = event.

  • Link detection: determine whether a pair of stories describe = the same=20 event.

A recurring observation throughout the session was that there was = a=20 pressing need for the integration of technologies in order to = achieve=20 progress in multilingual multimodal system development.

Multilingual Metadata

The aim of the third session, chaired by Shigeo Sugimoto = (University of=20 Library and Information Science, Japan), was to explore issues = regarding=20 multilingual metadata in order to understand their relevance to = multilingual=20 information discovery and access. The Dublin Core effort (OCLC, = 1999)=20 has spawned a community interested in metadata standards in a = multilingual=20 environment, and that community has in turn built a working = relationship=20 with people interested in multilingual metadata content. This area = is still=20 very much in an initial exploratory stage, and thus no research = paper was=20 presented.

The keynote speaker was John Kunze (National Library of Medicine, = USA),=20 who focused on metadata standards. Kunze described the current state = of the=20 Dublin Core standard, reporting that it is presently described in = sixteen=20 languages, with about twelve more on the way (Baker, = 1998). He observed that the fifteen core elements of the "standard" = have=20 been adapted and tailored by each organization that has adopted it, = and=20 offered a worse-is-better hypothesis in which implementation = simplicity=20 takes priority over completeness and consistency as a way of = explaining=20 this. Kunze stressed the importance of document surrogates (such as = those=20 provided by metadata) as a basis for information discovery, but = noted that=20 whether the Dublin Core in particular is useful for this purpose = still=20 remains to be established.

The panel session was opened by Jos=E9 Luis Borbinha (Biblioteca = Nacional,=20 Portugal), who offered a perspective on metadata as a device for = mediating=20 between an information provider and an information seeker. This led = him to=20 focus on metadata content, in contrast to the standards issues = addressed by=20 Kunze. Borbinha believes that there should be movement towards = normalization=20 of metadata content, with a more consistent use of controlled = vocabularies=20 for names, institutions, subject fields, etc. He also introduced the = question of how we can evaluate the utility of metadata, a topic = that=20 received some attention in the next session.

Clifford Lynch (Coalition for Networked Information, USA) = observed that=20 with multilingual metadata we are in essence talking about = cross-cultural=20 retrieval. This is far more than mere cross-language searching, he = claimed,=20 and he offered as an example the fact that places and institutions = in=20 another country might be organized in ways that are not amenable to = simple=20 translation. Lynch views metadata as an encoding system that uses a=20 quasi-natural language structure to create document surrogates that = are=20 critical for resource discovery. One particular problem noted by = Lynch is=20 that, in some cases, validity checks are needed to preclude abuse = when the=20 metadata is provided by the same source as the information it = describes.=20

Evaluation of Multilingual Systems

Information retrieval and machine translation have developed = quite=20 different evaluation methodologies, and multilingual information = discovery=20 and access lies squarely at the intersection of the two. The goals = for this=20 session, chaired by P=E1raic Sheridan (TextWise LLC, USA), were to = identify=20 important issues for the evaluation of multilingual systems by = considering=20 the way in which IR systems are presently evaluated and then = exploring the=20 extent to which evaluation methodologies from related fields can be = usefully=20 incorporated.

Donna Harman (National Institute of Standards and Technology = (NIST), USA)=20 presented the keynote talk in which she described the evaluation of=20 multilingual systems within the context of the Text Retrieval = Conferences=20 (TREC) (NIST, = 1999).=20 Spanish documents and queries were introduced in TREC-3, followed by = Chinese=20 in TREC-4. Retrieval systems that had been designed for English = transferred=20 remarkably well to monolingual applications in these languages, = requiring=20 only relatively minor modifications such as the replacement of = English word=20 stems with Chinese character bigrams. The Chinese evaluation task = turned out=20 to be "too easy," however, with most systems performing so well that = it was=20 almost impossible to distinguish between the results of systems that = used=20 widely different techniques. That test collection is therefore of = relatively=20 little value as a resource for further experimentation. In TREC-6 a=20 Cross-Language Retrieval task was introduced with documents and = queries in=20 English, French, and German, and Italian was added in TREC-7. = Through this=20 experience, NIST has learned valuable lessons about designing an = evaluation=20 that includes multiple languages. Query formulation and relevance = judgments=20 are now performed by groups in Europe in order to involve native = speakers.=20 And as issues related to the cultural aspects of cross-language = topics have=20 arisen, it has become clear that topic translation must be more = conceptual=20 than literal. This approach has implications for the comparative = evaluation=20 of systems, but it ensures that the topic statements are more = natural in=20 each of the languages.

A research paper was presented by Aitao Chen (School of = Information=20 Management and Systems, University of California Berkeley, USA). In = his=20 talk, he described the automatic construction of a bilingual lexicon = using a=20 new Japanese/English CLIR test collection (NACSIS= ,=20 1999). Chen=92s approach involved using aligned sentences from = parallel texts=20 of scientific abstracts and automatically creating a = Japanese-English=20 lexicon through statistical identification of word-word translation = pairs in=20 the aligned sentences. The lexicon was then used to translate = queries from=20 Japanese into English, and the queries were used to retrieve = abstracts from=20 an English portion of the test collection. Chen explored both the=20 performance of individual system components and their contribution = to=20 overall task performance.

The first contribution from the panel was by John White (Litton = PRC,=20 USA). He observed that translation engines are now merely one = component of=20 complex multilingual systems, so machine translation evaluation = should be=20 viewed from the perspective of the contribution of translation to=20 "downstream" applications. White stated that evaluation of = multilingual=20 systems is a difficult task because the evaluation of translation = results is=20 necessarily subjective, observing that the evaluation of translation = and=20 summarization are similar in this regard because there is no one = "right"=20 answer in either case.

Noriko Kando (National Center for Science Information Systems = (NACSIS),=20 Japan) briefly described her experiences as the organizer of the = first=20 NACSIS Test Collection Information Retrieval (NTCIR) evaluation (NACSIS= ,=20 1999). NTCIR was designed to evaluate retrieval in a Japanese = environment,=20 with a focus on abstracts of scientific papers. The evaluation = included=20 three tasks: ad hoc retrieval of mixed language (Japanese and = English)=20 documents using Japanese queries, cross-language retrieval of = English=20 documents using Japanese queries, and an information extraction task = known=20 as term recognition and role analysis. This is the first year for = the NTCIR=20 evaluation, and 23 groups from four countries participated.

The discussion centered on the costs of evaluation, the need to = establish=20 satisfactory criteria that depend on the component to be assessed, = and the=20 fact that some tasks cannot be evaluated on a "right or wrong" = basis. Where=20 possible, the creation of "ground truth" data such as TREC relevance = assessments can be an important factor in keeping costs down. By = contrast,=20 it was claimed that the subjective nature of machine translation = evaluation=20 makes it impossible to create useful ground truth for translations = that are=20 meant to be read by people: factors of style, quality, utility, and = target=20 audience are always involved. Metadata evaluation is an issue about = which=20 not much is presently known, at least by that name, but it was = observed that=20 the question is reminiscent of the early debates between advocates = of=20 controlled vocabularies and free text searching. Early studies in = the IR=20 literature showed that retrieval using controlled vocabularies was = highly=20 variable, often either a great success or a total failure. Baird = reported=20 that monolingual OCR evaluation resources are well established for = some=20 languages and that clearly defined metrics exist, but that little = evaluation=20 has been done in the context of multilingual document image = processing=20 applications. There was general agreement with the observation that=20 evaluation -- whatever the component type being evaluated -- should = be=20 primarily task-driven.

The Way Ahead

The final session, chaired by Douglas Oard, bought together a = panel with=20 a variety of perspectives on the management of research. The first = speaker=20 was Noriko Kando, who identified five layers of technology that = should be=20 considered:

  • the pragmatic layer: cultural and social aspects, convention

  • the semantic layer: concept mapping

  • the lexical layer: language identification, indexing, CLIR

  • the symbolic layer: encoding issues

  • the physical layer: the network

Each layer raises specific problems and must be addressed = independently=20 before integration is possible.

Hans-Georg Stork (DG XIII/E5, European Commission, Belgium) = outlined the=20 funding policies of the European Commission (EC) in the digital = library and=20 multilingual information access areas. He described the shift in = focus from=20 the Fourth Framework to the Fifth Framework, and the key lines of = action=20 within the Fifth Framework. Multilinguality is clearly a priority = area for=20 European research -- the Commission gives equal status to each = official=20 language and devotes resources to European minority languages as = well. In=20 the Fourth Framework, considerable funding was allocated to the = development=20 of linguistic resources under the Language Engineering programme. In = the=20 Fifth Framework, language resources are expected to be integrated = with=20 language technologies in the Information Societies Technology = programme,=20 with the greatest concentration of work likely being in two "action = lines:"=20 Information Access, Filtering, Analysis and Handling, and Human = Language=20 Technologies (EC, = 1999). The=20 EC and the US National Science Foundation recently released a joint = call for=20 proposals to support international collaboration on Multilingual = Information=20 Access and Management that generated considerable interest in the = research=20 community (NSF, = 1999). At=20 the time of the workshop, the proposals received for that call were = under=20 review, and a similar joint EC/NSF program for international digital = library=20 research is now being considered.

Ronald Larsen (Maryland Applied Information Technology = Initiative, until=20 recently with the Defense Advanced Research Projects Agency (DARPA), = USA)=20 described the new DARPA program for Translingual Information = Detection,=20 Extraction and Summarization (TIDES) (DARPA, = 1999). The main goal of TIDES is to dramatically reduce the time it = takes to=20 develop integrated systems that apply these technologies to = languages for=20 which the required resources do not presently exist. Larsen noted = that=20 multilingual information discovery and access will likely require = fusion of=20 technologies that have not previously been used together. He also = suggested=20 some key points that are considered by funding agencies when = evaluating=20 proposals:

  • What=92s the problem?

  • How is it addressed at the present?

  • Why is it significant?

  • What=92s new in this proposal?

  • How can progress be measured?

  • Who is the end user?

Clifford Lynch was the final speaker in the session. He observed = that=20 much of our discussion had focused on technologies, and that more = discussion=20 about applications -- particularly digital library applications -- = would be=20 useful. Lynch also raised issues of modularity, interoperability, = and=20 scalability that must be addressed if system performance is to be=20 guaranteed.

The discussion following the presentations focused on = collaborative=20 activities, both internationally and across present disciplinary = boundaries.=20 The idea of creating some sort of registry for multilingual = resources as a=20 way of accelerating research in this field and avoiding duplication = of=20 effort was broadly supported. Stork suggested that one or more of = the=20 funding agencies would find a proposal to create such a registry = attractive=20 if it went significantly beyond what is presently available.

At the end of the session, Oard asked the participants in the = workshop to=20 identify problems that they would advise a doctoral student just = starting to=20 work in this area to consider. The range of answers revealed the = diverse=20 perspectives brought by the participants, and included:

  • moving beyond bilingual access to truly multilingual systems

  • moving beyond text to multimedia retrieval

  • rapidly extending systems to cover new languages

  • developing evaluation collections

  • building multilingual classification systems

Conclusion

Multilingual Information Discovery and Access is a broad topic, = and it is=20 certainly not possible to treat every aspect of it with equal depth = in a=20 single day. At the outset we had sought to explore three key issues, = and we=20 made some progress with each:

  • The first and second sessions initiated a good interchange = between=20 researchers working on multilingual access and researchers working = with=20 linguistic modalities such as document images and digitized = speech.

  • The third session began the process of exploring the = relationship=20 between free-text retrieval and metadata-based techniques for = information=20 discovery and access in multilingual environments.

  • The fourth session developed a broad perspective on = evaluation,=20 highlighting fundamental differences between those based on ground = truth=20 (e.g., OCR accuracy or retrieval effectiveness), those that seek = to=20 measure understanding (e.g., the accuracy of translation for end = users),=20 and those that seek to measure how well users employ a tool (e.g., = a=20 multilingual classification system encoded as metadata).

The workshop attracted an extremely broad cross-section of = researchers,=20 and important contributions were made both by the speakers and by = members of=20 the audience on a number of other points. Among these were:

  • The need for a greater focus on users. Relatively little is = known=20 within the research communities that we assembled for this = workshop about=20 the abilities, expectations and goals of potential users. As we = expand our=20 focus beyond search technologies to encompass components that = perform=20 visualization, summarization and translation, we will need a = greater=20 understanding of user requirements and cross-cultural issues. = Social=20 issues, such as the importance of applying this technology to = minority=20 languages, will also arise in this context.

  • The potential for substantial contributions by developers with = experience building practical digital library systems. In addition = to what=20 could be learned about user needs from this perspective, important = issues=20 regarding interoperability, modularity, scalability and = reusability are=20 likely to emerge.

  • The importance of effective cross-disciplinary communication. = Each of=20 the communities that contributes to multilingual information = discovery and=20 access can benefit from an improved understanding of what the = others are=20 doing. The creation of a resource repository was suggested as an = initial=20 step in this direction, and the ultimate result of such = collaboration=20 could be the development of a common theoretical framework.

This MIDAS workshop has been one link in a chain of events that = have=20 brought together researchers from around the world and across a = broad range=20 of disciplines to consider different aspects of multilingual = information=20 discovery and access. The strength of this approach is the ability = to=20 include new communities while achieving a sufficient degree of focus = to make=20 progress. Living as it does at the intersection of several = disciplines, it=20 seems that a sequence of what are essentially "one of a kind" = meetings=20 serves the needs of researchers in this area well. Leadership in = this field=20 has come from a wide range of sources, and we are eager to see that = trend=20 continue. Accordingly, we are interested in discussing the potential = for=20 future MIDAS workshops that address some of the opportunities = identified=20 above, or perhaps issues that we have not yet foreseen, with people = who=20 would be interested in building on our work. We profited from a = great deal=20 of valuable advice from those who have gone before, and will gladly = share=20 our perspective on what works with the next organizers.

References:

Baker, T., Languages for Dublin Core, D-Lib, December 1998 = (available at http://www.dlib= .org/dlib/december98/12baker.html).

DARPA Translingual Information Detection, Extraction and = Summarization=20 (TIDES) program, 1999 (available at http://www.darpa.mil= /ito/ResearchAreas.html

EC Fifth Framework Program, 1999 (available at htt= p://europa.eu.int/comm/dg13/fifth-framework-programme.htm.

Grefenstette, G. (ed.), Cross-Language Information Retrieval,=20 Kluwer Academic Publishers, Boston, 1998 (some papers available = at http://www.rxrc.xerox.com/research/mltt/DMHead/CLIR/SIGIR96CLIR.html= ).

Hovy, E., Ide, N., Frederking, R., Mariani, J., Zampolli, A. = (eds.),=20 Multilingual Information Management: Current Levels and Future = Abilities,=20 1999 (available at http://www.cs.cmu.edu/Peo= ple/ref/mlim/).

Hull, D. and Oard, D. (eds.), "Cross-Language Text and Speech = Retrieval"=20 Papers from the 1997 AAAI Spring Symposium, Technical Report = SS-97-05, AAAI=20 Press (some papers available at http://www.clis.= umd.edu/dlrg/filter/sss/papers/).

Klavans, J. and Sch=E4uble, P., Summary Review of the Working = Group on=20 Multilingual Information Access, in Report of the Joint US National = Science=20 Foundation-European Union Working Groups on Future Developments for = Digital=20 Library Research, ERCIM Technical Report, No.98/W004, 1998 = (available at http://www.iei.p= i.cnr.it/DELOS/NSF/Brussrep.htm).

NACSIS Test Collection Information Retrieval Evaluation, 1999 = (available=20 at http://www.rd.n= acsis.ac.jp/~ntcadm/index-en.html).

NIST Text Retrieval Conferences, 1999 (available at http://trec.nist.gov/).

NSF Multilingual Information Access and Management: Call for=20 International Research Co-operation, 1999 (available at http://www.interact.nsf.gov/cise/html.nsf/html/jointannounce?Op= enDocument).

OCLC Dublin Core Metadata Initiative, 1999 (available at http://purl.org/metadata/du= blin_core).

Copyright =A9 1999 Douglas Oard, Carol Peters, Miguel Ruiz, = Robert=20 Frederking, Judith Klavans, and P=E1raic Sheridan

=20

Top = | Contents<= BR>Search = | Author Index = | Title=20 Index | Monthly Issues
Previous=20 Story | Next=20 story
Home | E-mail the=20 Editor

=20

D-Lib=20 Magazine Access Terms and Conditions

DOI:=20 10.1045/october99-oard=20

------=_NextPart_000_0000_01C9B88A.4FF71400 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: http://www.dlib.org/dlib/october99/images/story_bar1.gif R0lGODlh1QEOAPcAAP////j4+ff4+ff4+Pj3+ff3+ff3+PHy9PDx9PDy8/Dw8/Dx8u7x9O7x8u7w 8unr7ujp7Ojp6+bo7Obo6+Lk6ODk6ODi6OLi5uDi5uLg5uLi5ODi5OLg5ODg5N7i6N7i5t7g5t7i 5N7g5Nvd49vd4NjY4Nbd4Nbb4NbY3dHU3dTX2tTR2tHU183U2s3R2szQ18XQ1MnM0MXJ1MXM0MHJ 0sXJzcHJzcHFzcHBzcHFycHByb3J0r3Jzb3BzbrDzL3Bybq+zL6+x7q+x7q6x7S6x7K3x7K3wbKy way8x6y8way3waq1wayywbeyvKy3vKqwu6Swtaqqu6Kuu6Kou6iutKiotKKutKKotKiitKGhtqKi tKiorqKorqiirqKirpyuu5yutJyotJyitJuhtpOhtpOhsIuhsJyorpOorpuhrpuhqJOhrpOhqIuh qKGbrouasJubrpqaqJOaqJqTqJOTqpOTqJOToouaqouaqIOaqouTqouTqIOaooyUpYuToouToYSU pZOLooyMpYSMpYOLooyEpYSEpYyUnIuTmoSUnIyUlISUlIyMnISMnIyEnISEnIyMlISMlIyElISE lIyEjHuUpXuMpXuEpXuUnHuUlHuMnHuEnHuMlHuElIR7pYR7nIx7lIR7lIx7jIR7jIxzjIx0hIRy hIxrhHt6pXJ1pW1rpWZjpXt7nHN7nHtznHNznGtznHNrnGtrnGNrnGtjnGNjnGhrlGRjlFpjnGNa nFpanFJanFpSnFJSnFpblFJalFpSlFBSlFJKnEpNnE5KlI1se4Rre41je4xrc45jc49ja41Zc5BT Y49aa49Ra5RCY49KY5FKWpFBWpFCUpE5UpQxUpI5SpQxSpQxQowxSZMpSpQpQpwmQpQhQowpQpUp OZYhOZUhMZUYOZwYMZQYMZcQMZgYKZwQKZQQKZgIKZkLIQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwAAAAA1QEOAAAI/gDVoRN4 7lw6dOPQlSPHcBy4cOK4hftGcVvFbRixacPGMdu1bNaohZxGbZq0aSalRYsGrWVLZ8yYNWvmrFky ZcqW6UxWrGfPU6dKBTVlqpQoUqREiZIUShIlSZAkNdoUqVGkTZwacer0iOqkSJEeTdI0aWynqVwf aWqkadMmTZE0cSL7tRHXTm2pxqX6tqpWtm+xdu2E1y2nuGcnTd3UKa7frG3bchpcGCviRordwm30 N7JgxpUPa0q8+OzUzZ0DTwadVzTex4AlU259OTNjq5y2qh3dN9JkTmUfxX3EdS3ZvITlau002a5w t12rbiK+dqtlTWTxtn00F+tawljt/raNNIlr1keGCjkalH4Qqk+PPHky9OkTK/uoULF6xYpV/vyp vCIgLK/IImAqsMgSSyyyNEjLLLOsQgstDd5SSy233IJLLbl0qMuHIH4ITDC78KLLLr0A4wsvvbRI jDAuBvPiMOkcdFBB55iTkI7kQBTOQ+AECc433HSDUUYcXfPRNdVUE1JJ0pxkUjQqsWRlTM5kOZNO zSyjTDLLHJPMMcf8VEopxRRVSlKimDKKU0uBIokkXzmyRyWEYLIHIYlUIogle1jCSCOCEAIIno0Q MkifjVSCiCWANJLIIIg0AogliBDCCJ6YSpooIIlIAsijkiKSiB+JIBIpJoBEqkki/6MCcqcfmCDC aiJ7WiLpIYn0kcghq7bKFqzA9lFJH5gccquuv/b6a7CuEguIscgqC2ofiVrCLK++AtsIq9ECUuyx 2zrrLbjDijvtscne6qslqDICqaOAoKrqIJg0YskggDACLrXJTjuIIJUAy0i/gBCciCaCYLIwr4I0 AmyxiSSSbyODHFtxJYBoMoilrWpyyCDWYjKIx5pcysh6hgxSSCOPfLLJJYbUjMoln+hnXyqutOLK zwG+4vMrAcbySiywLNigKrA82OAsF8oC9YQX4pKLLrXoYjUuWPuii9cgBvNLL8Hw8kswLabdyzAv EjOMOurcqM4545SToznljCOOQ/7iADlkN9xsYyQ223iTjTZKWvPRkyhRQ02UVEa+EktY1jRTl83k tEwyySDDk0+lGHPmmUcpJcooobw5p1SOtAVJJ448IkkkjDS2yeuSwLWJc4c9ghkknHHCyO1ghdUJ JJMgXxbMkAj/SL6TSMIJJJrMHpf0rzuCGcxUCQ9XWJr4Lvv01YNV/fSdzCGEDjdEQYhvjHxfRxha dHHGFmF0wQUi52ffiBc3wAER1OA9sKjFd1UIAxfstwU8WK9/ncDCFbzAPfhpgoH54wIX1oA92GFG DkPoQQ6gYIfvHRAzjhAeJ1onifFJZRK+a0TrGPEISDwifrl7xCMcQbvbaQJ4k/7Y3Q2JBzzqOWIT kyigYrISFuE1woZHHFTrkMcITkhCiK+bSvyGR0P1sKcrNPPEJWi2if/4pz/7YUUqWNGKAAWtQUaL hSpkoQqlyeJBt5iFhfR4IQtZKBcXyoXXwAaiFQUjGB/iBYp2cTaytSgYw2AbJOd2DkqeAx10WwhD etS3cEikIt840jauMcokOYka1xCJ40yCEipFySUugQlMLtclMNmScz7pyZnUxCZRhCIUopATKHYI CUgQohGGqASeHIYJRV3qUSkDFZ/0ZQlNEMISeEIEnvB0qVElgk93Uua3QBUqTKjlYITQxJ5a1c1G 7AETlWgE/6LJqIZpAhOaOP/YyfrAr1ZZggYACGhAA4AHelbiBgJNKACmwE5I+eEBCm1Bq+qJT4UG 9AP8bCggBBrPQ2CnVRYFQAX8Gak+8EChAigDRTXBv0NZii2D4tOkADGwQUBKEIsSFKimZTJNGKtP h7BEIiyBU01Y4hCVWNhQK3HUiuULm4P4mMUOxiqPzjRjIqtEVZm6qIr1a1A1K0QhdNgVQ9wsZp5g RRn7ox+hoaIVbWxjHI2mCqIl6I6ruOODoFYLWtyCaoG8kC+4dourCXJFwOhFiVbEC1/sYhdic+Ta hIE2tKHjIHNDB0LKUQ5zmKMh4BCHJ80Bjm5QpBtG2oZGSom4xDGJJI6TUpT+quSSmECDGZaz3OZy wrky+RYoQDkTUkyhFFCYLnpzcoQjmle74lVxM7X7IV4U47tIHHF6WUFi/DahPe69bjLVm0rrUgYJ S2jPNLtTYSdmt7vpCOeK3G2EVMJSifNyJr1V/ENAC3ADCQSUBfDVXhMwsAECBBQCGABBGtQriQ0E FAQ4UEBAZRDf+T5CoBnAgECZoJUqrleg7o0EfAWKgQRjYAYent0e9ouDCATUBdqz8A3hosNINIaH NMwKYXCMHcU0Jixx2W7sMNOJ6AKPTlypCiPMd5auSKLIfcGOI2AInK4Qjzy+W4tdOFM8jAnCEYag oQ7dW7OcoZEVruhPz1D+4Yr8CKgVRxOQHFOhoKUxaBZPm5AtLkSLPuYCF7jIEKBDtKJEiqiRZzvk MHoBI2K4bW3pgJuN0tFZz6Jjk+MIR4+4wQ0iYSSURzrcRpZkDcaZBEqQkwY0VgJLWWppGTbZ3OaO gYyfBGV0pzBFUowyimCGghJl+cokhGqpS0yrEdhKGaP8xLFJsQoTB8MUpPogqEp8jGMRWyq1MYUJ nEaiEpEARCf64FOO8YtSg+AnpCxGsKjmM1/8e4S4yW0smtKUCgFFABnuYAUwZMHaJ/OXxCoQ0DDU +9yHGENAGQCqNgg04PnyqEDF/YKA7sDeJBvExNkNcIGWG+Mj68MUApr/ADTQ4QtTGMO7JRZNeX2T M4fqUyIahi9rW9ujQWWXyTBB7Y/Bc2Ef29eh+Nmngmn1UtROmKWQaqxjaZXcC5sWqEJeiVexRViJ MIQg0INTHZYZPdP5xCX0w0Y0tgIV/HkF2l8BC1ggiO13fdocZWGLvUJNFhfCEIcC+TWweW0XuuCF 4Hexol6wyJGLlpHihxGMStYoHZfsbEPIkelxZPohE9kGOIq0jcBhIxvYUNLiqNGkVUrjcSqZBquj ERPcypImXYL1l44x6zGVqSdqEm6bem1c4yZvuWD51iYWsburtE44V5kdl3n4lfb+UC1sMd/wjCMW 74UPO4mABCbKy93o/nElfuE7XiemY2MdSi8rnADEt7k/5SfPWBMS3m8JuJCW82sFEBoIKBa8/34c BPQH5CcCAdUFuXF/kSBQGuABAtUF34cdFxZQ5Fd/GNYBHCACRwB+xAEJDiBQBlACWmB/6VcVXSEW NaRjZIEVYqYWkBAX1JM7mhA7ViRduxM+SdR8S1RFQBRE3GU+4kNduOFhwhE/IqYWV7RkUjYXxCc+ gxA7NWMIOiQzYnRG+QFXadYKaMYfcDYgbJcKSBN3dNcgdNRXdddnELJ3GKIhWJM1uAAMX/MhkOVY KDIij3RIkEQ2MkIQmnVZ43BJ5LBZe7M3nuRJoGYRgaNaoVdq1/Ak/09yeihRJSuhaszQErjlejVR E7C2OZwzJrmEa6MjCkYRTHLyFJKgXOtRTfIyMDuHbB+DUxbDc60iCJGiVVHFTI2SLEA1CPGDbLBy MiPDCIMgKshmKVqFVDM3U0ZXCYzwLCSzFo6QCI5QUsKYLEklCGyQAgoVBMroUTJEcABwBYcijTNn AxaHVJVwAQElBcsoQ4lgURCwBMM4jesYUOSYjMASUjUAj/xiBiqgUD6Qjs2oVelUMEQFCA7TbMh4 KfrCMZZAdBgTMNtGTcbSMK2iVQRDU/lyKDTVKCKTCBjjU9gRVD6FMCYDK7CoLCKDVIRyT42wMjDj hE4YH/PRFWhEdv9p1mZZmAqtUFdBM0dcCIZzhGcSMjV9lHdm2CGFpQt/5gtK6TWGJ3i6oFgtwiLE YIeVFUmQd0mXVQ6YhBB10zcMEQ7hgA5/8zedpw2dtw3aAHqKU2ojwYgloXpWIonQkCV0CXuxh4m+ VQy3VwrIcGuk02vFtRRNIQmMAAkcuQmZADMraRybkDuc8UPkMSiKIUPhMzyTOV6zo2XN80Nw0TyP kAi+uJId6RtrEXzOFz/H85iTcAiTsITxNCgnQ5pWIR1+IAYnEFAGADxssZqtqWELBRdzMZtCMGH5 pAkTEFBpQBasuYQC9QQBAAACEAayGXweh5q6KVBToEBngAezORX/p2EHYYACASUAytmajnBFyaND s6kY2lM+yBc76XUVt3MWNMQIk1kWwFM7VhE+U2RjX7FCWZFErTM9O6iY+QQZK4kXLagVVbEwkTkI jwChX+QJYvcJnuAWm8AKhjCTApJmb3YgRGM0CdJ2dPQ0YIhnd4cheFcLgdYhLPoht7ALXMOGH/IL uvALbKhIaoM2jLdoxBAMcCMQkIdJmTQO5MBZmhaIQUIR35BaoyR6SuIkT/KWqDdbrLZqWOJ6uLUl d4lLtAY6uZZ7SdFrlMB7ZTEnkDBGlWAIyGYJmFAIJEkovNgnkeJR4sJU3/JTl3IsCeMxgzIvQSUo 0AZugwAZfRAJ/6wiVExlLR2zKQwzVJFiCc/TCYXKFoeaqImABAFlAoBwBx4nCJCqL49gAQV3qZfS J28wnnegCVUwUKEqqZjwcE4QUAeAqKfKVBOXT1UHqh5nqop6VEkQUCfACHkgUJAiqpjgpkgFqiLD L3+CLDS1KB3TdBj3bn4CKzsVKQTTkEiVLNqyrZpgbazSMNRmdazSkf60kPEkVL7iMYUwCJcgCIYA COnxMuKDHjJTHzqkH27hH2/1Ch7qCgISZyJ6VwlyZw5SC3dkC3zVV4B0NVyTCxrColfThl7zCxjb C7qQWGizo4yXNgIxECEbeQthpH3TSX0zJEQyJBmBOIeYiCDhOP+xdXoqUSXPEIkvoaWWaBO1JCZj Umu6hCajQzqnY1yqM4pSkaCNsAidsAicgAlHFHw2JmJrEYSdkDxJlD2+0zqNSRbRlxV2UUVZsX34 BAicUAmTAAjwOQk0VBXM8To2JkOA8AjJJhUpc7Zpu7ZxIFAT0AD/dRbPSLds8QEB9QV52xVs+wgx EFADkH8BZQSBW7ceBwghEFA+sLY0JFDoE7eOEFIAoF3CIQd8uwABtQJzW7ftmUKxw39B1hWSyR2x IzssCBdXi7izIxacuYNDFj/q6RsACha2gRjCYRUypBatAzO+MxdrkQickAibwAjqoR5gVjNlJR8y MzNn9h8gOrD/A8IgBguGTkNHeDYhFpJ3GRJIuAA2XPMhStl3v8BIHduxdrho9CsQQloQdWM3CdGV 5CBaElFaoVSIhJMRooeIjhMSJzFbqrdqWHpbMLGlsBZrmVgmn3MKxRAUojMKpTAKupYUwCYnk7Bc g7IJ1fSuyJRPNKUwH4MsrNJsNDWMoGJT2AJt5DQt1YRUyRhP6ne6neCMzDsI43Yoi4JP3UJTyLIJ h9AJ5oS2fdDDzcgJQNwHh1IEApBQJHAH6YYJSKzEj0AB+vfDQWxtiQADCSUAQHDESbzEAtXEVsC4 ehDGGhdQRZzFnivFYowJSvCcAjUCfLDFS2wyltJP+MJzktJN/8yiU67oU5OCTUi1kEIFT8CykJYS rlLMMJKSLxajU39idFkFcwtDVBwTceyRdfSqMuzRkmVmH5ewCWklhRwqsKmQCms3ILFsIAzCICX6 NFAzC3tGC3tmhoFVsRSrC8EANijSIoWXNj8qDD+6NjJSI5dFSZSGDuZwaQ3BDeggWt+gsk76pKqF iKmESjLbiI/oiJTjwM0gE8tgiTdRe51zwRccFH7Za4C5FEuBpl/hL5ggCYsgKbeTmPEDPLILFs+b tWKBGTZWZLnhO/bZHLeDRDATCTqFxPqStrSTZCH2gsSrw1DcCf2ixYdQ0epXO8/xCHDABWEQCBtt KR0NCBRtCf8WTdJXUWNhIEFVwdFA3C8c2RUhDdMjjdE1ptE3PRU9HdNALRya4AZXwAV+gNMeTUMO aF3v9TuiSRVWMWWOgBdc+xWvc31rMZm0MxZXVH7Nw0TD4zsQbUWccDtdoZtTMRhZfUUrmQiTkAmT gAmToAjC45Jahx4b+oQVqh/5wUaDLbBqlIUg2iBs55N0RyEr+ssNyyEx2iGBJqNXU2iCJ3hk4wtj U1ktwsxoIwzCEKSYpVkFgRCflaRCsqREYlqqtQ0eARKKk4jh3IgkoXpVomqtlltcOmtiQiZkYsFA UQylUBQa/Im+BgrItT1r8TGPUE0QGrhBNZGNspKREimNIlTa2FQxGtmKxDgp1JZU+3zdabsJ5ObS 1TZz9pZPbppC2Ye2ieDRgFDe503Cm6Le/MLeUNu8kADf8+3S9Z3esJjf8rLf7z0J8Z1CrELfKWPf 07jeBZ5C/23eDS7gEN7e/A3fSizFuiJO/ULJ4DIwSRWp3aJMIY2M8XR0ljowjRJUmIxTR4VPfwKL y4IxUqzed4oxfpKszeisHGkJYfauY9UyYzYzZeQJqJBWgm2FUyg03Hs0YMh2CyJHDoJnT1MLu9xn epd37XsLSOmGJgIMLFI2ivSU8jtZbLNoAQEAOw== ------=_NextPart_000_0000_01C9B88A.4FF71400 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: http://www.dlib.org/dlib/october99/images/blue-dot.gif R0lGODlhAQABAPcAAAAAAAAAMwAAZgAAmQAAzAAA/wAzAAAzMwAzZgAzmQAzzAAz/wBmAABmMwBm ZgBmmQBmzABm/wCZAACZMwCZZgCZmQCZzACZ/wDMAADMMwDMZgDMmQDMzADM/wD/AAD/MwD/ZgD/ mQD/zAD//zMAADMAMzMAZjMAmTMAzDMA/zMzADMzMzMzZjMzmTMzzDMz/zNmADNmMzNmZjNmmTNm zDNm/zOZADOZMzOZZjOZmTOZzDOZ/zPMADPMMzPMZjPMmTPMzDPM/zP/ADP/MzP/ZjP/mTP/zDP/ /2YAAGYAM2YAZmYAmWYAzGYA/2YzAGYzM2YzZmYzmWYzzGYz/2ZmAGZmM2ZmZmZmmWZmzGZm/2aZ AGaZM2aZZmaZmWaZzGaZ/2bMAGbMM2bMZmbMmWbMzGbM/2b/AGb/M2b/Zmb/mWb/zGb//5kAAJkA M5kAZpkAmZkAzJkA/5kzAJkzM5kzZpkzmZkzzJkz/5lmAJlmM5lmZplmmZlmzJlm/5mZAJmZM5mZ ZpmZmZmZzJmZ/5nMAJnMM5nMZpnMmZnMzJnM/5n/AJn/M5n/Zpn/mZn/zJn//8wAAMwAM8wAZswA mcwAzMwA/8wzAMwzM8wzZswzmcwzzMwz/8xmAMxmM8xmZsxmmcxmzMxm/8yZAMyZM8yZZsyZmcyZ zMyZ/8zMAMzMM8zMZszMmczMzMzM/8z/AMz/M8z/Zsz/mcz/zMz///8AAP8AM/8AZv8Amf8AzP8A //8zAP8zM/8zZv8zmf8zzP8z//9mAP9mM/9mZv9mmf9mzP9m//+ZAP+ZM/+ZZv+Zmf+ZzP+Z///M AP/MM//MZv/Mmf/MzP/M////AP//M///Zv//mf//zP///wAAAA0NDRoaGigoKDU1NUNDQ1BQUF1d XWtra3h4eIaGhpOTk6Ghoa6urru7u8nJydbW1uTk5PHx8f///wAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwAAAAAAQABAAAIBAAFBAQA Ow== ------=_NextPart_000_0000_01C9B88A.4FF71400 Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Location: http://www.dlib.org/dlib/october99/images/d-line2.gif R0lGODdh1QEEAOYAAP///0pKhEpKjEpChEpCjEpCe0o5e0o5c0o5a0oxa1Ixa1IxY1IpY1IpWlIp UlIhUlIhSlIYQlIhQloYQlohQloQOVoYOVoQMVoIKVoQKVoIIWMIIWMAGFoIGGMIGGMAEAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwA AAAA1QEEAEAH/4AfgoOEhYaEHImKi4yMHhwekZKRG5WWGpWYGpucnZ6eGKGioxgZpqWmqaqrpheu r7CxsrMXFbYWFRa6u7y9vrwTFhPDxMXGxBTJERQSzcwSENHS09TV1tEP2drb2g7e3+Dh3g3k5ebn 6OnlDOwL7u/w8fLzCvX29gn5+vv5CAkIAAMCPECwoMGDCBMaMHBgocOHBgpELECxosWLBQZo3Mix o8ePGwkIGEmypEkBh1Kq/NCopctJkzZ4sLQBU81POHNqwLCTVKhTP1kJbUWrqNFatXL9WspUV7Bj UItFGLbsWbNm17JqxQaB2zYH2cSJDaeurFly7BowmMe2rTt8CtLyxZ3Lr+4/gQIT6t17EKJfhxgD WwRJuLBGAgFEnlw8cqXjQi4jK4o0WZJMmppu6tzc0+eooaAzHB09y8KFpqh9BRMWFepUCq8jSLAK bavtag+6eu02treDs8DTuR3u9t69BPjk9hN4Ny/f5wr/PqQ4UXBgw9g/jlTM2OTj75IlP3oEaVIH D5tuauas0/NnVKFVkZ4P65bS1KmftpY6YSoxCRPQhtVtBGKzG2++jRXcgmipRdyD7xgHF3J2LYeX c9BlSJB0gFnnYXYgckQAd92RFAgAOw== ------=_NextPart_000_0000_01C9B88A.4FF71400--