研 究 生 毕 业 论 文
（申请硕士学位）
论文题目 对几种常用英汉机器翻译软
件译文质量的评测
作者姓名 颜薇薇
学科、专业名称 英语语言文学
研究方向 翻译研究
指导教师 柯 平 教授
2004 年 5 月 25 日
学 号： MG0109046
论文答辩日期： 2004年5月25日
指导教师： _______________（签字）
An Evaluation of the Output Quality of Some Prevalent English-Chinese Machine Translation Programs
by
Yan Weiwei
Under the Supervision of
Professor Ke Ping
Submitted in Partial Fulfillment of the Requirements
for the Degree of Master of Arts
Department of English
School of Foreign Studies
Nanjing University
May 2004
I hereby declare that this submission is my own work and that, to the best of my knowledge and belief, it contains no material previously published or written by another person or material which has to a substantial extent been accepted for the award of any other degree or diploma at any university or other institute of higher learning, except where due acknowledgment has been made in the text.
Signature:
Name: Yan Weiwei
Date: May 25, 2004
Table of Contents
1.1.2 Rationale for the Present Research
1.1.3.2 Previous Approaches to the Evaluation of Output Quality
1.3 Organization of This Thesis
2.2.2 Criteria for Measuring the Quality of MT Output
CHAPTER THREE RESULTS AND DISCUSSION
3.2 Analysis of the Output Quality in Terms of Separate Subjects
3.2.6 Syntactic Ambiguity and Semantic Analysis
3.2.8 Special Difficulties in English-Chinese Machine Translation
4.2 Implications of the Findings
4.3 Limitations of the Study and Suggestions for Further Research
APPENDIX: MACHINE TRANSLATION EVALUATION TEST SUITE
LIST OF TABLES
Table 2.2.2-1 Scoring criteria
Table 5.1-1 Means of total scores & rates of correctness
Table 5.1-2 Ranking of the programs in terms of means of total scores
Table 5.1-3 Ranking of the programs in terms of rate of correctness
Table 5.1-4 Ranking of the programs in terms of overall output quality
Table 5.2.1-1 The means of the total scores on the subject of “lexical coverage”
Table 5.2.1-2 The rates of correctness on the subject of “lexical coverage”
Table 5.2.2-1 The means of the total scores on the subject of “phrases”
Table 5.2.2-2 The rates of correctness on the subject of “phrases”
Table 5.2.3-1 The means of the total scores on the subject of “morphology”
Table 5.2.3-2 The rates of correctness on the subject of “morphology”
Table 5.2.4-1 The means of the total scores on the subject of “simple sentences”
Table 5.2.4-2 The rates of correctness on the subject of “simple sentences”
Table 5.2.5-1 The means of the total scores on the subject of “complex sentences”
Table 5.2.5-2 The rates of correctness on the subject of “complex sentences”
Table 5.2.6-2 The rates of correctness on the subject of “syntactic ambiguity and semantic analysis”
Table 5.2.7-1 The means of the total scores on the subject of “generation of Chinese”
Table 5.2.7-2 The rates of correctness on the subject of “generation of Chinese”
Table 5.2.8-1 The means of the total scores on the subject of “difficulties in ECMT”
Table 5.2.8-2 The rates of correctness on the subject of “difficulties in ECMT”
Table 5.2.9-1 The means of the total scores on the subject of “translation of long sentences”
Table 5.2.9-2 The rates of correctness on the subject of “translation of long sentences”
ACKNOWLEDGEMENTS
When this thesis was finally completed, I felt indebted to many people, without whom the thesis would have never been brought to the present shape. I’d like to take this opportunity to express my sincere gratitude to all of them.
First and foremost, my heartfelt thanks go to my supervisor, Prof. Ke Ping at Nanjing University who has been extremely supportive throughout this research. His lectures broadened my horizons in Translation Studies and machine translation research. His enlightening advice and help have been so important in rendering the present research possible as well as worthy of conducting. Most important of all, his patience and constant encouragement helped me to drag through the numerous difficulties I have been confronted with in writing and revising this thesis.
I am so grateful to Prof. Yu Shiwen, and his graduate students Liu Qun and Bai Xiaojing in the Institute of Computational Linguistics of Beijing University. They generously provided me with the test suite which they had prepared on the basis of many years of hard work. In the process of my research, I also benefited a great deal from their advice. These warm-hearted people are true devotees to machine translation research and will always be respected by students like me.
My sincere thanks also go to my friend Sha Junqing, who spared no efforts to help me find and test some of the programs. Meanwhile, I want to give my warm thanks to my colleague Yang Fei, who kindly helped me with her computer expertise whenever my computer went wrong.
Finally, I’d like to express my genuine thanks to my family and my husband for their love and encouragement, without which I would have never finished my graduate study, let alone this thesis.
ABSTRACT
An Evaluation of the Output Quality of Some Prevalent English-Chinese Machine Translation Programs
Yan Weiwei
This thesis reports an exploratory evaluation of the output quality of some prevalent English-Chinese Machine Translation (hereafter abbreviated to “MT”) programs and analyzes the translation technologies involved. The purpose of the research is to find out programs which produce relatively better output as well as the major linguistic bottlenecks English-Chinese MT programs would encounter in its processing of English texts.
Eight fully-automatic English-Chinese MT programs (DrEye Deluxe Pack, TranStar III, Oriental Express 2003, Homepage Translator 2000, SYSTRANBox, Kingsoft FastAIT 2003, LingoWare, and GBTP Global Access) and one MT evaluation (hereafter abbreviated to “MTE”) test suite containing 298 testing points (instituted by the MT Research Group in the Institute of Computational Linguistics of Beijing University) were selected for the evaluation. The 467 sentences in the test suite underwent translation by the selected programs. Criteria were established in light of structural linguistic theories to score the MT output from the perspectives of fidelity and intelligibility. For each program, the mean score it obtained for its output and the rate of correctness of its translations of the testing points were calculated.
An analysis of the mean score and the rate of correctness of each program was made in combination with an investigation into the mechanism of the translation engine used by the program. The analysis and investigation generated the following findings about the output quality of these programs and the technologies involved. GBTP Global Access, with IMT/EC technology in its engine, produces the best output. DrEye Deluxe Pack, supported by its human-computer interaction technology and a huge storage of words/phrases, ranks second. TranStar III, whose translation engine incorporates a logical semantic theory based on case grammar, ranks third. Oriental Express 2003, featuring its S-speed technology and 49 dictionaries, ranks fourth. Homepage Translator 2000, combining the Slot Grammar theory and the pattern-based translation technology and translating with the help of an English-Chinese pattern-based dictionary, ranks fifth. SYSTRANBox and Kingsoft FastAIT 2003, while each displaying some advantages, failed to come up with good enough output. LingoWare’s performance is the worst as, judging from its output, its translation procedure consists of only two simple steps, i.e., dictionary looking-up and substituting the dictionary translation for the original
Concentrating on the least adequately translated testing points, the researcher discovered the following major linguistic bottlenecks in English-Chinese MT:
(1) Syntactic ambiguity and semantic analysis
(2) Target text (i.e., Chinese) generation
(3) Some special linguistic phenomena (emphasis, ellipsis, inversion, etc.)
(4) Long sentences
Based on the above findings, this author draws some conclusions regarding the research most needed for the development of quality English-Chinese MT systems.
(1) Most of the tested programs performed fairly well in their treatment of simple language phenomena, but not in that of relatively complicated ones, and none fared sufficiently well so far as the generation of the target language is concerned. This is evidently due to the imperfection of the translation engines these programs depended on, which were not quite capable of handling complicated language phenomena. In future researches, more attention should be paid to the construction of working natural language models geared to MT R & D. Good natural language models are crucial to the design of good MT engines.
Secondly, it is very important to incorporate appropriate technologies into MT programs. GBTP Global Access, DrEye Deluxe Pack, and TranStar III are good cases in point. Viewed from the current trend of MT development, knowledge-based and corpus-based technologies have the greatest prospect of solving the many problems found in prevalent MT programs and significantly improving their output quality.
Given that all the eight tested programs adopt sentence as their basic translation unit, the present research evaluated their output quality mainly at the sentence level. Future researchers may pay attention to the institution of the test suite, which should account for not only every aspect of sentences (ranging from morphology, lexicon, and syntax to semantics) but also units larger than sentences. The testing points which test more than one language phenomenon at a time should also be included.
摘 要
对几种常用英汉机器翻译软件译文质量的评测
颜薇薇
本文对八种常用英汉机器翻译软件的译文质量进行了全面的评测，分析了与各软件相关的机器翻译技术，并且在此基础上归纳出英汉机器翻译处理中的主要语言障碍。
本研究选择八种常用的全自动英汉机器翻译软件（“译典通豪华版”、“译霸III”、“东方快车2003”、“翻译家2000”、“SYSTRANBox”、“金山快译2003”、“LingoWare”及“金桥译港世界通”）作为测试对象，采用我国机器翻译领域里的重要科研基地北京大学计算语言学研究所机器翻译课题组制定的“英汉机器翻译译文质量测试大纲”作为测试集（该测试集包括298个离散的测试点，467条例句，每一测试点只集中测试软件对某一具体语言现象的翻译情况），对被测软件的翻译质量进行了系统测试。研究者以结构主义语言学理论为基础，建立了一套评测译文质量的标准，从“忠实性”和“可懂性”两方面对机译软件输出的八套汉语译文中的所有译文进行评分，计算出各软件所得的平均分，同时计算出各软件以上述测试点为基准的翻译正确率。
根据各软件所得的翻译平均分、对测试点的翻译正确率以及研究者对各软件所用的翻译引擎的调查，本研究就各软件的译文质量与相关翻译技术得出如下结果：
(1) “金桥译港世界通”的翻译引擎采用了IMT/EC技术，其输出译文的质量最佳；
(2) “译典通豪华版”在庞大的单词/词组储备和人机交互技术的支持下，输出译文的质量位列第二；
(3) “译霸III”在分析句子的过程中运用了基于格语法的逻辑语义分析理论，其输出译文的质量位列第三；
(4) “东方快车2003”的翻译引擎采用了S-speed技术，并在翻译过程中充分利用其49部机器词典，其输出译文的质量位列第四；
(5) “翻译家2000”将空位语法理论和基于范型的翻译技术相结合，在翻译过程中借助了基于范型的英汉词典，其输出译文的质量位列第五；
(6) 尽管“SYSTRANBox”与“金山快译2003”各具优势，但其译文的总体质量不尽如人意。“LingoWare”的表现最不理想，从输出的译文来看，它的翻译过程仅由两个主要步骤组成，即查字典和中英文替换。
通过对所有译文的分析，笔者找出了这八种软件翻译得最不成功的测试点，并在此基础上归纳出全自动英汉机器翻译软件目前面临的主要语言障碍，即：
(1) 句法歧义与语义分析
(2) 目标语言（汉语）的生成
(3) 某些特殊的语言现象（如强调句、省略句、倒装句）
(4) 长句的处理
本研究的结果表明：
(1) 大多数英汉机器翻译软件在处理简单语言现象时的表现都是不错的，但在处理较为复杂的语言现象时就显得捉襟见肘了。这其中的主要原因应该在于翻译引擎的设计中未能把许多复杂的语言现象包含在内。在今后的研究中，语言学和计算语言学研究人员对面向机器翻译的自然语言模型应给予更多关注，并将自然语言模型研究方面的优秀成果应用于机器翻译引擎的设计之中。
(2) 如果仅依靠机器词典，全自动机器翻译软件一般不可能提供令人满意的译文；只有采用适当的机器翻译技术，尤其是IMT/EC技术以及逻辑语义分析技术，才能够提高全自动机器翻译软件的译文质量。在本项研究中，“金桥译港世界通”采用的IMT/EC技术、“译典通豪华版”的人机交互技术、“译霸III”的逻辑语义分析技术、“东方快车2003”的S-speed技术以及“翻译家2000”将空位语法理论和基于范型的翻译技术相结合的技术对测试集中例句的翻译都是相对成功的。但是，从全自动英汉机器翻译的发展趋势来看，机器翻译技术不能仅仅停留在处理语言结构的层次上，要根本解决全自动机器翻译软件目前面临的语言障碍、全面提高译文的质量，必须采用混合策略，在机器翻译的引擎中结合更多、更有效的技术，尤其是知识库技术和语料库技术。
本研究的结论对于自然语言处理中语言结构模型的研究以及机器翻译系统的开发有一定的理论意义和实践价值。
由于本研究所选的全自动英汉机器翻译软件在翻译处理中均以句子为单位，所以本研究只对句子一级译文的质量进行了评测，而没有进一步考察各软件在语篇层面上的翻译质量。在未来的研究中，研究者应关注测试集的选用与完善。测试集的内容必须更加全面和系统，既包括句子，也包括语篇。测试点既应涵盖句子的各个层面（从词法、词汇、句法到语义），也应体现各类语篇的差异。最后，综合测试两个或两个以上语言现象的测试点也应被纳入测试集之中。
CHAPTER ONE INTRODUCTION
As Doug Arnold (1994, pp. 4-6) claimed, Machine Translation (hereafter abbreviated to MT) is playing an increasingly important role in today’s world — socially, politically, commercially, scientifically and intellectually. MT programs have become helpful tools when it comes to the processing of multitudes of multilingual documents within a limited period of time. Since there are many MT programs for the users to choose from, questions naturally arise that what programs produce better translation? And what difficulties are most MT programs confronted with? This research attempts to probe into these questions. Because MT programs are still in their infancy, we want to pave the way for the future work of those researchers and developers dedicated to this realm.
This section begins with the explanation of some basic MT concepts involved in the present research. Then it introduces the rationale for the research and reviews previous researches. In the end, it displays the overall organization of the whole thesis.
Some basic MT concepts are briefly explained in the following.
Machine Translation
Machine translation is “the application of computers to the translation of texts from one natural language into another (Hutchins, 1986, p. 15)”. In other words, it is the process that utilizes computer programs to translate texts from one natural language into another. An MT program is a computer program which completes the task of machine translation.
MT Output
MT output refers to the translation (target language text) of the input (source language text), which is produced by the computer program.
MTE
MTE is the evaluation of an MT program or MT programs. There is no simple or unique way of conducting an MTE. The evaluation made in this research focused on the quality of the output, i.e., the translation of some prevalent English-Chinese MT programs.
Test Suite
In natural language processing (including machine translation), a test suite is a set of inputs, artificially constructed and designed to probe the system’s behavior with respect to some particular language phenomena. It should cover a wide range of language phenomena which MT systems may encounter.
Other important concepts are explained in respective chapters where they are discussed at length.
1.1.2 Rationale for the Present Research
According to Hutchins (1986, p. 15), the principal reason for MT is the fact that there are not enough translators to cope with the ever increasing volume of material that has to be translated. The use of computers for translation was first proposed in Warren Weaver’s memorandum in 1949. The next few years saw the beginning of MT research in the United States, the Soviet and Western Europe. The first “real” MT system in history was publicly demonstrated at Georgetown University in 1954. However, these early efforts actually resulted in laughable translations and there was widespread disappointment on the part of funding authorities at the practical results. Doubts in relation to the possibility of automating translation in general were most clearly voiced by Bar-Hillel’s report in 1959, in which he argued that fully-automatic, high-quality MT was impossible in principle. In 1964 the well-known ALPAC (the Automatic Language Processing Advisory Committee) report doubtlessly brought a disaster to MT by concluding that “there was little prospect of good-quality and/or cost-effective MT, and there were enough human translators to cope with demand (Hutchins, 2003, p. 510)”. It led to the virtual end of government funding in the USA and to a general loss of morale in this field. It was not until late 1970s that MT research underwent a period of renaissance.
During the 1980s, MT systems came into practical operation at numerous installations. At the same time, programs for personal computers began to be marketed. During the 1990s, the situation in the marketplace was transformed by low-cost (and inevitably low-quality) MT programs for personal computers, by the demand for immediate “less-than-perfect” translation on the Internet, and by the development of systems for cost-effective large-scale production of company documentation. Many big MT companies joined the competition, some of whose products achieved great success. Hutchins (1986, p. 335) said at the end of his famous work, Machine Translation: Past, Present and Future, that “MT is becoming a commercial product like other technical aids and office equipment”.
MT research in China can be traced back to the mid 1950’s. Like MT research elsewhere in the world, it had its ups and downs in the course of 47 years. It experienced 4 periods (Dong, 1988, pp. 85-87), i.e., initiation, standstill, recovery, and development. In the initiation period (1957 to 1965), research in Russian-Chinese and English-Chinese MT began and China’s first experimental MT system was demonstrated. Subsequent researches mostly focused on linguistic schemes or conceptual design. The standstill period (1966 to 1975) was caused by the insufficiency in linguistic studies, the inadequacy of the computers available for MT systems and the barrier of Chinese ideograms. The recovery period (1975 to 1982) began with MT research entering into China’s fifth five-year plan. During these eight years, great achievements were made both in MT techniques and in the training of professionals. The fourth period began after 1982, during which Chinese MT researchers started to build commercial systems on the basis of the experiments made in the recovery period.
The most successful project in China so far has been KY-1 developed at the Military Academy of Sciences with a claimed average accuracy of 75%. In 1987, it was marketed as Transtar-1, which is the first domestic commercialized MT program in China and “initiated the development of Chinese domestic MT programs (Ge, 2000, p. 2)”. Empowered by the overwhelming influence of the Internet, China’s MT program market thrived. In the subsequent 15 years, a considerable number of new programs were developed, including Light, HansVision, Instant Translator 2001 E to C T System, Internet Passport, Kingsoft FastAIT, Tongyi, and Huajian Instant-Trans. However, some of them were eliminated through the fierce competition. Only a few are still available in today’s market, among which are the well-known Kingsoft FastAIT and Oriental Express. According to one investigation, Oriental Express and Kingsoft FastAIT are the most popular in China’s present MT program market, “covering 73% and 56% of the market share separately (Ge, 2002, p. 78)”.
The vendors of MT programs are always overconfident of their products, purporting that “the rate of accuracy and readability of their translation have reached 70% to 80% (Xu, 1998, p. 47)”. However, unlike what they are boasting of, the fact is that the output of most programs is still laughable, which often contains misspellings, incorrect tense, wrong word order, untranslated words, so on and so forth. For example,
Example 1
Original text:
Chinese New Year is a month away, but Yao Ming still had reason to celebrate New Year’s Eve in Houston as members of his fan club attended the Buck-Rockets game.
Translation:
中国的“新的年”是 一月离去， 但是“Yao Ming ”静止的曾有原因以赞颂“新的年的前夕”在“Houston” 作为会员属于他的扇子俱乐部参加了雄鹿-火箭游戏。(Translated by LingoWare)
Example 2
Original text:
Seventeen of those immigrants were found dead when the truck’s trailer was discovered abandoned early Wednesday at a truck stop just south of Victoria. Two more of the immigrants died later. Deputies found 17 bodies inside and two more died later. Initial autopsy reports showed the victims died from dehydration and suffocation.
Translation:
当在维多利亚的货车停车站正义(公正)的南(方)发现了这个货车的尾部放(遗)(抛)弃的早星期三时那些移居的十七发现死。 两越来越多的移居的以后死(灭)亡了。 代表在里面找到了 17 个物(身)体和多二以后死(灭)亡。 初始尸体解剖报告显示(这些)牺牲品死于脱水和窒息。(Translated by DrEye Deluxe Pack)
In terms of their translation quality, “at present, most MT systems still have a great number of weaknesses and much needs to be improved, which resulted in people’s loss of confidence (Xu, 1999, p. 97).” Given the generally unsatisfactory performance of most MT programs, it is necessary to make an evaluation of the output quality of some prevalent English-Chinese MT programs. Detailed and comprehensive analyses should be made in relation to the true conditions of their quality, their linguistic bottlenecks, and the technologies they employ.
This section reviews some previous researches on the output quality of MT programs.
1.1.3.1 MTE Research Overview
As a routine part of its development, every field of science and technology must develop procedures for evaluating the quality of its results. MT is no exception. With the spurt of the MT research in 1950s and 1960s, MTE came into being. It is of interest to all the different groups involved in the creation, deployment, use and maintenance of MT programs. Funders want to know the return of their investments, researchers want to be informed of the latest progress in this field, and users want to see the usefulness of MT programs or the quality of their output. In recent years, MTE is gaining more and more attention in China with the surge of interest in MT research. Appraisal meetings are often held to evaluate different MT systems. MTE has been included as an important part in the “National Intelligent Interface Assessment” supported by the 863 High-technology Project of China.
There have been many types of MTE.
On the basis of evaluation purposes, White (2000, pp. 102-107) characterized five types of MTE, i.e., feasibility evaluation, internal evaluation, declarative evaluation, usability evaluation and operational evaluation. Among these five types, declarative evaluation was what most of the people think an MTE was, whose purpose was to measure the ability of an MT system to handle text representative of an actual end-use.
Luo Airong (1995) classified MTE into operational evaluation, declarative evaluation and typological evaluation in terms of evaluation methods.
Hirschman and Thompson (1997) broadly distinguished the following types appropriate to three different goals, i.e., adequacy evaluation, diagnostic evaluation and performance evaluation. When individual components were considered, they made a further distinction between intrinsic and extrinsic evaluation. They also drew a distinction between glass-box and black-box evaluation in terms of evaluation strategies, which sometimes appeared to differentiate between component-wise versus whole-system evaluation, and sometimes to a less clear-cut difference between a qualitative/descriptive approach and a quantitative/analytic approach.
Given that an MT program is in essence a piece of software (in a broad sense), it is profitable to see any of the stages of its creation, installation and maintenance as that of a software system. The Evaluation of Natural Language Processing Systems made by EAGLES[1] adopted the ISO 9126 Standard on software quality (Arturo, 1999, p. 261). This standard sets out six quality characteristics which should be considered in the evaluation of software products, i.e., functionality, reliability, usability, efficiency, maintainability and portability. Each characteristic is subdivided into a number of more specific sub-characteristics. Since these six characteristics are intended to apply to all software products, they could also apply to MT programs.
Yu Shiwen (1993, pp. 117-126) proposed two types of examination models ― a subjective and an objective one. In the former, the examination setter gave the marking criteria while the examinee filled the answer sheet. Then the graders checked the answers against the criteria. The latter was an automatic one.
1.1.3.2 Previous Approaches to the Evaluation of Output Quality
Although users and developers of MT programs share a common interest in the quality of MT output, the problem of defining it has not been solved yet. The standard quality characteristics (i.e., ISO 9126) are applicable only to a limited extent to natural language processing (NLP) products and do not answer questions concerning the output quality of MT programs. Establishing useful criteria for an objective MTE turns out to be an extremely difficult task, since definitions of translation quality (“What is a good translation?”) are mostly based on human judgments.
The actual implementation of the evaluation of the output quality of MT programs also proves difficult. Firstly, there is no set of exactly correct answers against which to compare the output. Since there are many ways to say the same thing in any language, it follows that there must be a possibly infinite number of “right” translations for any given text. Secondly, specific requirements on output quality vary significantly. Some people expect high-quality renderings while others need merely readable ones. Thirdly, it is especially difficult to put together MT systems of totally different types to compare. To summarize, just as Hutchins (1997, p. 419) said, “there is an urgent need for MT researchers, developers and vendors to agree on and implement objective, reliable and publicly acceptable benchmarks, standards and evaluation metrics.” However, a generally accepted standard has never been found so far.
Hirschman and Mani (2003, pp. 414-427) distinguished two approaches to the measure of output quality: intrinsic and extrinsic. For the former, the output can be considered by itself, compared against other outputs, or evaluated against the input. Both the “quality” (in a narrow sense) and the “informativeness” of the output are assessed. They defined “quality” as “the extent to which a text is well-formed, understandable, coherent, etc. to a native speaker” and “informativeness” as “the extent to which a text conveys information content, usually related to the preservation of text content”. For the latter, the impact of the MT technology on the efficiency of some task is often measured. It is natural to employ intrinsic measures in this research in that the aim here is to evaluate the quality of the output.
According to Arturo Trujillo (1999, pp. 258-259), the most common approach was to ask a translator or subject expert to rank the output quality, defined in terms of the “monolingual intelligibility” of the target text and the “bilingual accuracy” of content preservation. This approach was obviously very subjective. One of the first evaluations of this type was undertaken as part of the ALPAC report, which deeply influenced subsequent evaluation methodologies. In these evaluations, “intelligibility” assessed the “fluency” and the “grammaticality” of the TL text, without concern for whether it faithfully conveyed the meaning of the SL (Arturo, 1999, p. 258). For the measure of “intelligibility”, Arnold etc. (1994, pp. 169-173) proposed a minimum four-point scale ranging from “hopelessly unintelligible” to “perfectly intelligible” while a ten-point scale was used in ALPAC report. Nagao’s (1988, pp. 144-186) five-category scale was frequently used. “Accuracy” was defined as “an indication of how the translated text preserves the content of the source text” (Arturo, 1999, p. 259). Typically, a group of bilinguals were presented with a set of SL sentences and their translations along with a scale with which to rank the “accuracy”. Nagao’s scale (1988, pp. 144-186) of “accuracy” consisted of seven points.
The evaluation of MT output quality also belongs to what White (2000, p. 104) and Luo (1995) classified as “declarative evaluation”. In a declarative evaluation, “fidelity” (the accuracy and completeness of the information conveyed) and “intelligibility” (how fluent or understandable the output appears to be) were assessed. In their opinion, to measure the output quality was to measure its “informativeness”, “adequacy” and “fluency”. The first two measured “fidelity” while “fluency” measured “intelligibility”.
Different yardsticks were distinguished by Hutchins (1986, p. 330), i.e., “intelligibility” (of output text, e.g., via readability scales and cloze tests), “fidelity” (of SL text, e.g., via measure of information transfer), “acceptability” (to recipient of translation), time spent in revision (post-editing), number of “errors” corrected, and type.
The evaluation carried out by Xu Jian and Liang Maocheng (1999, pp. 97-102) measured the “intelligibility”, “fidelity” and “acceptability” of the output of 8 MT programs.
It is easy to perceive of the similarity the aforementioned measures shared. All employed intrinsic measures. “Informativeness”, “fidelity” or “accuracy” roughly referred to the same thing, i.e., the extent to which the output (translation) conveys or preserves the meaning of the original. Likewise, the definition of “intelligibility” was basically the same with grammaticality and fluency combined (sometimes style was also included).
Neither the “fidelity” nor the “intelligibility” of MT output is easy to measure, not to mention the measure of “style”. Among the various measures used to assess the quality of MT output, subjective grading containing a four-to-ten-point scale (or category) was often used. Since different evaluators adopted different scales (or categories), this kind of evaluation was inevitably highly subjective and led to unreliable results.
A set of criteria for measuring MT output quality (Xu, 1998, p. 47), which was established by Canadian experts, made it possible to quantify the measure and to some extent reduce its subjectivity. The criteria are presented in the following:
1 Accuracy
1.1 Misspelling (source or target)
1.2 Unclarity (source or target)
1.3 Source word not coded
1.4 Text missing
1.5 Incorrect translation
2 Language
2.1 Syntax
2.1.1 clause boundary
2.1.2 wrong part of speech analyzed
2.1.3 wrong word order
2.1.4 missing article
2.1.5 wrong preposition
2.1.6 wrong punctuation
2.2 Morphology
2.2.1 wrong agreement (number, gender)
2.2.2 wrong tense
2.2.3 wrong case
2.2.4 wrong article
2.3 Terminology
2.3.1 wrong word
2.3.2 wrong part of speech
2.3.3 clumsy compounding of noun phrase
2.3.4 bad dictionary coding
3 Style
3.1 Style deficient
3.2 Style inappropriate
Altogether twenty items were listed. Every 1000 words were assigned 1000 points. Words were the basic units for scoring. The output was scored in strict accordance with the nature (seriousness) of the errors, i.e., 5 points for minor ones and 10 points for serious ones. What was left in the end was the total score of the translation.
The measure of “accuracy” and “language” were better designed with relatively concrete items listed while the measure of “style” might easily fall into subjective judgment in that the specific border of a deficient style and an inappropriate style was not clearly set.
Based on the above researches, this thesis established more feasible criteria for measuring MT output quality in Chapter Two.
This thesis reports an exploratory evaluation of the output quality of some prevalent fully-automatic English-Chinese MT programs with an aim of providing answers for the following two research questions:
(1) Among the MT programs tested in this research, what programs produce output with relatively higher quality?
(2) What are the major linguistic bottlenecks in these MT programs?
1.3 Organization of This Thesis
Chapter One reviews the literature related to the present research and states the research purpose. Chapter Two describes the methodology of the research including the subjects, the instruments, data collection, and data analysis. Detailed criteria are established to assess the quality of MT output. The results are presented and analyzed in Chapter Three while the conclusions are made in Chapter Four.
CHAPTER TWO METHODOLOGY
This chapter elaborates on the methodology of the present research, embracing the subjects (MT programs), the instruments, data collection, and data analysis.
Eight prevalent English-Chinese MT programs (i.e., DrEye Deluxe Pack, TranStar III, Oriental Express 2003, Homepage Translator 2000, SYSTRANBox, Kingsoft FastAIT 2003, LingoWare, and GBTP Global Access) were selected as the subjects of this research. The criteria for selecting these programs were:
(1) commercially available in China or accessible on the Internet
(2) presently and popularly used
(3) fully-automatic
It is now an undisputable fact that Kingsoft FastAIT and Oriental Express are taking the lead in the China’s MT software market. TranStar III, though not so popular as the two best-selling ones, is welcomed by many users. LingoWare is very easy to use. DrEye Deluxe Pack comes from Taiwan and has achieved some success in mainland China. Both SYSTRANBox and GBTP Global Access are online translation programs. Homepage Translator 2000, developed by IBM Company, is favored by quite a number of users. All programs are easily accessible via purchase in the marketplace or download from the Internet.
Kingsoft FastAIT 2003
Kingsoft FastAIT 2003 was developed by Kingsoft Software Co., Ltd. It is the latest version of Kingsoft FastAIT. The core technology in its engine is the FastAIT (Fast Artificial Intelligence Translation) technology developed by NOVA, a Japanese company. This program has 49 dictionaries, focusing on medicine (western and Chinese), law, computing, finance, economics, psychology, geography, engineering, and other sciences. It can translate words, sentences, full texts as well as webpages and localize users’ menus.
In the process of translation, sentences are the basic units. The program first divides a text into sentences according to periods and then translates the whole text sentence by sentence. It transfers the English punctuations into corresponding Chinese punctuations.
Oriental Express 2003
Oriental Express 2003 was developed by Sunway Software Co., Ltd. It is the latest version of Oriental Express. Its translation engine employs S-speed technology which supports various operations including editing, sending mails, printing, and whole webpage translation and editing. Based on its embedding technology, it exclusively performs in-depth embedding. Its function on a single CD covers:
■ batch code localization
■ mutual conversion between traditional and simplified Chinese characters
■ switching between languages
■ translation between Chinese, Japanese and English
■ conversion between over 10 kinds of internal codes of Japan, Korea, Hong Kong and Taiwan
■ perpetual localization
■ translation of articles, webpages, words and programs
■ online updating of its dictionary
TranStar III
TranStar III was developed by Chinasoft Network Technology Co., Ltd. It is the latest commercial version of TranStar. It is an inexpensive and versatile program for translating Microsoft Word, Excel, PowerPoint, and PDF files in Windows XP and 2000. It can translate full texts and embedded texts. It has on-screen word lookup and dictionary maintenance. In the process of translation, TranStar III’s analysis involves a “logical semantic theory” based on case grammar. “Online word lookup” makes it easy to find definitions. Users can customize the user’s dictionary and translate a word or a sentence with a click of the mouse.
LingoWare
LingoWare is a small MT program and is easy to use. It was developed by LingoCom, an Israeli-based company founded in 1998. It allows users to enjoy the interface of any application in their native language, including Arabic, Chinese, Dutch, French, German, Hebrew, Italian, Japanese, Portuguese, Russian, Spanish, and Swedish. This program is now a shareware. It has three components: the “hook and interception technology” translation technology that allows English text to be captured and displayed in the target language; a global dictionary that is used for general translation; and specific dictionaries for selected programs created either by program developers or translators who have received authorizations from these developers.
DrEye Deluxe Pack
DrEye Deluxe Pack, the latest commercial version of DrEye, was developed by a Taiwan company and came into market at the end of 1996. It can translate sentences, full texts, webpages, desktop icons, etc. Its translation process involves syntactic analysis, semantic analysis, and human-computer interaction technology. By “human-computer interaction technology”, we here mean that it provides the users with several translations for some difficult parts (polysemants, ambiguities, etc.) in a sentence to choose from. This simple-to-use program has over 40,000 words/phrases, covering 90 domains (e.g., computer science, philosophy, etc.) with explanations on the words’ origin and grammar.
SYSTRANBox
SYSTRANBox was developed by the world’s longest-lasting MT supplier, SYSTRAN, which is the only independent fully-fledged supplier of the full range of machine translation solutions. It is a free program and accessible at http://www.systranbox.com/. It provides online translation, including text translation and webpage translation. The users only need to add a translation box to their websites and then can surf the Internet in their native languages. SYSTRANBox is an ideal MT solution for portals and content providers. Today, many leading portals have already chosen SYSTRANBox.
Homepage Translator 2000
Homepage Translator 2000 was developed by IBM’s China Research Lab. It is also small and convenient. Its translation engine combines the Slot Grammar theory[2] and the pattern-based translation technology[3]. In the process of translation, Homepage Translator firstly parses[4] the English sentence using Slot Grammar. Then it matches an English-Chinese pattern-based dictionary against the resulting parse tree and builds corresponding patterns for the translation.
GBTP Global Access
GBTP Global Access, accessible at http://www.netat.net/ is the first translation platform based on Internet application service. Its website was created by Huajian Electric Co., Ltd.[5] and two other companies. Its translation technology is based on Huajian’s key MT technology, i.e., IMT/EC technology (the latest MT technology that solely won the first prize of China National Award for Scientific and Technological Progress). Its main functions include browsing translation, real time translation, file translation, mail translation, bilingual BBS, bilingual chat, international transaction hall, English salon and other practical functions.
The instruments of the research included a computer, a test suite and detailed criteria for measuring the output. The computer was a desktop PC with Intel Pentium IV processor, operating in Windows 2000. The discussion in this section is focused on the test suite and the criteria.
Since “sentence is the basic unit in the translation process of MT programs (Ge, 2000, p. 30)”, it was chosen as the basic testing item. The great variety in different MT programs’ ability to deal with language phenomena necessitated a comprehensive test suite. The MTE Test Suite (Yu, 2003, see Appendix) instituted by the MT Research Group of the Institute of Computational Linguistics in Beijing University fitted well in with the research. It was published in 1998 as part of a National Natural Science Fund Project. In the course of its institution, letters were written to MT experts and system developers in China for their advice on the suite.
This test suite borrows from the language discrete-point testing method, which means each testing item (i.e., sentence) in the suite contains a testing point (i.e., single language phenomenon). It consists of three parts:
(1) Testing an MT System’s Ability to Analyze SL (from Subject 1 to Subject 6)
(2) Testing an MT System’s Ability to Synthesize TL (Subject 7)
(3) Testing an MT System’s Ability to Deal with Special Difficulties in MT (Subject 8 and Subject 9)
Altogether there are 467 sentences, covering 9 subjects and containing 298 testing points, namely, “lexical coverage”, “phrases”, “morphology”, “simple sentences”, “complex sentences”, “syntactic ambiguity and semantic analysis”, “generation of Chinese”, “special difficulties in English-Chinese machine translation”, and “long sentences”. To make the test suite more appropriate for MTE, the suite deliberately neglects the common grammatical terminology. Some testing points center on structures which are simple for human translators but comparatively difficult for MT. For instance, sub-subject 2.2 focuses its testing on “split phrases”, which are unusual in grammatical books. Its inclusion is necessary for MTE because various ways of splitting phrases often complicate MT analysis so much that an MT program often fails to recognize a split phrase as a whole.
2.2.2 Criteria for Measuring the Quality of MT Output
Since the evaluation of the quality of the output (translation) of some prevalent English-Chinese MT programs is a declarative evaluation, it should measure the “fidelity” and the “intelligibility” of the output. The former measured the extent to which the translation preserves the meaning of the original sentence. The latter measured “grammaticality” (i.e., how grammatically well-formed the translation is) and “fluency” (i.e., whether the translation reads like good Chinese sentence). But detailed criteria had to be established in that the criteria used previously, as mentioned in Chapter One, inevitably involved subjectivity and simplification in varying degrees.
2.2.2.1 Fidelity
“Fidelity” is the extent to which the output conveys or preserves the meaning of the original and should be the primary concern in this research. The focus here is on the preservation of meaning, which involves the comparison of the meaning in the output with that in the original.
In most cases, the analysis of language and semantics employs a bottom-up approach. We cannot content ourselves with it, though. Thus some explanations of sentence meaning should be made firstly.
Some linguists (Fries; Leech; Robins; Lyons, etc.) argued that the meaning of a sentence as a whole is not merely the sum of the meanings of the words and other constituents which compose it. Instead, the total meaning of a sentence consists of the lexical meanings (of the constituent lexemes) and grammatical meanings (of grammatical constructions that relate one lexeme, syntactically, to another). Chomsky (1965, p. 136) also assumed that the semantic interpretation of a sentence depends only on its lexical item, the grammatical functions and relative relations represented in the underlying deep structures.
Ke Ping (1996) distinguished three kinds of meanings in translation in a socio-semiotic approach, i.e., referential meaning, intralingual meaning (subdivided into phonological meaning, graphemic meaning, morphological/lexemic meaning, syntactic meaning, and textual meaning) and pragmatic meaning. As far as the state of the art of machine translation is concerned, the preservation of pragmatic meaning, phonological meaning, and graphemic meaning is almost impossible. And as far as this research is concerned, textual meaning is irrelevant.
On the basis of the above discussions, “fidelity” was measured in two dimensions: referential meaning and grammatical meaning. The measure of the referential meaning of a sentence was also conducted at two levels, i.e., lexical level and sentence level.
A. Referential Meaning
According to Ke Ping (1996), referential meaning is chiefly connected with the Topic in its broadest sense, since human language can be employed to talk about almost anything, universal or unique, real or imaginary. On most occasions of linguistic communication Referential meaning is the core element of a verbal message. It is also known as “conceptual meaning” or “cognitive meaning”. As mentioned above, the measure of the referential meaning of a sentence was conducted at lexical level and sentence level.
A lexeme, by definition, is “the smallest meaningful unit that is an item in the vocabulary of a language, which may be simply a word (word-lexeme) or a phrase (phrasal lexeme) (Ke, 1999, p. 78).” For the measure at the lexical level, the lexical meanings of the words as well as the phrases in the original sentences were checked against those in the output. This was done without paying attention to the strict equivalence at word level or phrase level due to the lexical gaps between the two different languages. A lexical gap is “the absence of a word in a particular place in a lexical field of a language (Ke, 1999, p. 96)”. A concept may be lexicalized in one language and described by circumlocution in another (ibid.). An English word may have a Chinese phrase equivalent, e.g., teenager, overweight, and pose (meaning “十几岁的年轻人”, “在重量上超过”, and “摆好姿势” separately). Likewise, an English phrase may have a word equivalent in Chinese. Therefore, as long as a lexical meaning is correctly represented, whether at the same level or not, it is preserved in the output.
Special attention should be paid to the meanings of function words. Lyons (1981, p. 158) distinguished the meanings of full words (including nouns, verbs, adjectives, adverbs, etc.), and function words (including the definite article, prepositions, conjunctives, the negative particle, etc.). The function words belong to classes of small membership and their distribution tends to be very strongly determined by the syntactic rules of the language. They are less lexical than full words, but some function words are more lexical than the others. In the limiting case, where a function word cannot but occur in a given syntactic construction, it has no lexical meaning at all, e.g., to in He wants to go, of in three pounds of butter. Nevertheless, there are many function words, which contribute some measure of lexical meaning to the sentence in which they occur. As a result, for function words, their lexical meanings were flexibly treated, depending on their specific usages. The function words with no lexical meaning (e.g., to in an infinitive phrase) were regarded as integrated into the sentence construction it belongs to. Similarly, the meaning of a phrase was also taken as a whole instead of adding up the individual word meanings, e.g., a great deal in She is a great deal better today.
Referential meaning at the sentence level often does not agree with the total lexical meanings of the separate lexemes. For example, You are welcome is usually translated into “不客气” instead of “你是受欢迎的”.
B. Grammatical Meaning
The grammatical meanings of English sentences are conveyed through inflectional morphology and syntax (Ke, 1999, pp. 120-122; Lyons, 1981, pp. 156-159). So the grammatical meaning of the output was measured in two dimensions, i.e., inflectional morphology and syntax.
Lyons (2000, pp. 52-53) also pointed out that the relation between lexical and grammatical meaning varies from language to language: what is encoded lexically (lexicalized) in one language sometimes may be encoded grammatically (grammaticalized) in another. Thus whether the grammatical meaning in the original sentence was grammaticalized or lexicalized in the output, it was rightly preserved.
1. Inflectional Morphology
Grammatical categories, by Ke’s (1999, pp. 124-132) definition, are “classes or groups of items which fulfill the same or similar functions in a particular language”. Different languages have different grammatical categories. In English, grammatical categories include tense, aspect, case, gender, mood, number, person, and voice (Crystal; Ke; Lyons; Robins).
Tense and aspect are grammatical categories in a large number of languages, but they are not in Chinese (Baker, 2000, pp. 98-99). Tense indicates the relationship between the form of a verb and the time of the action or state it describes (Ke, 1999, p. 128). Typical contrasts are made between the present, the past, and the future. English verbs conjugate to show different tenses (ibid.), whereas Chinese verbs, with no conjugations whatsoever, depend upon adverbs to convey the meaning of tense. In other words, tense in English is realized by grammatical means while in Chinese is by lexical means. Aspect deals with how the event described by a verb is viewed, i.e., whether it is in progress, habitual, repeated, momentary (Ke, 1999, p. 124). Likewise, English and Chinese have different means to express aspect though both have developed an aspect system.
Case shows the function of a noun or noun phrase (agency, possession naming, location, motion towards or from, etc.) in a sentence (Ke, 1999, p. 124). English case system includes the normative, the accusative, the dative, the possessive, and the vocative. Such a category does not exist in Chinese.
Gender is a grammatical distinction in which words such as nouns, articles, adjectives, and pronouns are marked according to a distinction between masculine, feminine, and sometimes neuter (Ke, 1999, p. 127). In English, gender is only applicable to the anaphoric pronominal links between nouns and he, she, it, etc., and the reflexive pronouns himself, herself, itself, etc. (Robins, 2000, p. 257). In Chinese, the meaning of gender is always lexicalized.
Mood expresses the speaker’s or writer’s attitude to what is said or written. Typical contrasts are made between the indicative (indicating factuality), the subjunctive (indicating possibility or uncertainty, its use now being restricted to formulaic or very formal situations), and the imperative (optative). The meaning of the indicative mood and the imperative mood in English may be found at the syntactic level in Chinese, and the meaning of the subjunctive mood is sometimes lexicalized.
Number distinguishes nouns, verbs, adjectives, etc. according to whether they are singular or plural, countable or uncountable (Ke, 1999, p. 127). In English, number recognizes a distinction between one and more than one (singular and plural), which has to be expressed morphologically, by adding a suffix to a noun or by changing its form in some other way to indicate whether it refers to one or more than one. Chinese prefers to express the same notion lexically and the form of a noun does not normally indicate whether it is singular or plural (e.g., my book and my books are both wo-de-shu in Chinese) (Baker, 2000, p. 87).
Person marks pronouns and, in most languages, corresponding verb forms, according to whether the pronoun represents or includes the person actually speaking or writing (“first person”), whether the pronoun represents or includes person or persons being addressed (“second person”), and whether the pronoun represents someone or something other than the speaker/writer or the listen/reader (“third person”) (Ke, 1999, pp. 127-128). Such a category is present in both English and Chinese.
Voice expresses the relationship between a verb and the noun phrase which are associated with it (ibid.). In English, typical contrasts are made between the active, the passive. Chinese verbs have no voice. Often, a bei-construction is used to conveyed the meaning of passive voice, which however, is not always the proper choice in translation. For example, The building is being built is usually translated into “这幢楼正在建着” instead of “这幢楼正在被建造着”.
To sum up, the output was examined to see whether the meanings of these grammatical categories in the original sentence were duly preserved, though a single English sentence may not simultaneously contain all the above mentioned categories.
2. Syntax
Grammatical meaning is not simply a matter of inflection and far more important are the syntactic differences between one grammatical construction and another (Lyons, 2000, pp. 52-53). With most modern languages, in which grammatical meanings are chiefly conveyed through syntactic and lexical means, syntax is much more important than inflectional morphology (Ke, 1999, p. 122).
When elaborating upon his theory of generative grammar, Chomsky (1965) discussed three major components: syntactic, phonological and semantic components. The syntactic component consists of a base that generates deep structures and a transformational part that maps them into surface structures. The semantic interpretation of a sentence depends only on its lexical item, the grammatical functions and relative relations represented in the underlying deep structures. It is conceivable that for some sentences, deep structures and surface structures are identical. But for others like the following sentences, transformations are involved in the derivation from their deep structures to their surface structures. He illustrated this point with the following example (1965, p. 22):
(1) I persuaded a specialist to examine John.
(2) I expected a specialist to examine John.
The underlying deep structures for (1) and (2) are, respectively, the following:
(1i) Noun Phrase — Verb — Noun
Phrase — Sentence
(I — persuaded — a specialist — a specialist will examine John)
(2i) Noun Phrase — Verb —
Sentence
(I — expected — a specialist will examine John)
Therefore, the measure of grammatical meaning at syntactic level should be based upon the deep structure instead of surface structure though they are sometimes identical.
Three kinds of grammatical meanings were identified by some linguists (Lyons, 1968, p. 435): (1) the meaning of grammatical items (function words and grammatical categories, i.e., tense, mood, case, etc.); (2) the meaning of such grammatical “functions” as “subject-of”, “object-of” or “modifier-of”, etc.; (3) the meaning associated with such notions as “declarative”, “interrogative” or “imperative” in the classification of different sentence types. The combination of Fries’ and Chomsky’s theories brought forth the measure of the grammatical meaning at the level of inflectional morphology and syntax. The measure of the grammatical meaning at morphological level was discussed in the previous section. The measure at syntactic level will be conducted from the three aspects on the basis of deep structures: (1) the meaning of function words; (2) the meaning of grammatical functions; (3) the meaning associated with different sentence types. The grammatical meanings of function words were separated from their lexical meanings (if they do have significant lexical meanings).
2.2.2.2 Intelligibility
Chomsky (1965, pp. 10-11) used the term “acceptability” (which we took as a synonym of “intelligibility” in this research) to refer to the “utterances that are perfectly naturally and immediately comprehensible without paper-and-pencil analysis, and in no way bizarre or outlandish”. Grammaticality was regarded as one factor to determine “acceptability”.
In this research, the measure of “intelligibility” revolved around two dimensions, i.e., grammaticality and fluency. Since both dimensions were elusive, especially the latter, feasible measures were also needed.
A. Grammaticality
Culicover (1976, pp. 3-4) defined grammatical sentences as “the sentences that sound good to the native speaker” and “the strings of words that can be formed by the rules of the syntactic component”. The grammaticality of a string of words is quite independent of whether it makes sense or not, which does not preclude the existence of a relationship between the syntactic structure of a linguistic expression and its semantic content. Lyons (1981, pp. 104-105) defined an ungrammatical sentence as “one in the formation of which the grammatical rules of the language system are not respected”. This definition applies not only to sentences, but also to phrases, e.g., *morning this, *late got up are ungrammatical. To measure the grammaticality of the output is to check whether these translations violate Chinese grammatical rules.
Grammar varies from language to language. According to Robins (2000, pp. 208-209), English has morphological word form variation in paradigms, i.e., inflections, through which many of its grammatical meanings or functions are realized. The role of inflections is non-existent in Chinese, but the syntactic classification and ordering of words in sentences are essential components of Chinese grammar. It is the syntactic relations that decide word class (Robins, 2000, pp. 210-215). Chinese sentences have fixed word order and the change in it may result in nonsynonymous sentences (Baker, 2000, p. 110). Thus word order and word class were subsumed into the measure of “grammaticality” of the output. In addition, Chinese and English also have different punctuation marks for the period: a hollow circle in Chinese and a dot for English. The usages of other punctuation marks are not always the same, either.
To summarize, grammaticality of the output (in Chinese) was measured from three dimensions: word order, word class, and punctuation marks.
B. Fluency
For a sentence, to be fluent is to let the readers read smoothly the string of words within it. The checking of collocation and idiomaticity was employed as the two dimensions for the measure of fluency.
Collocation is related to the co-occurrence of words and phrases (Ke, 1999, p. 90). Gramley (1992, p. 71) defined “collocations” as “the syntagmatic tendency of lexemes to work together in predictable ways”. By Halliday’s (2000, p. 333) definition, “collocation” is “a tendency to co-occur” and a particular association between the items in question. For example, there is a strong collocation between smoke and pipe.
Since the translations are in Chinese, they should read like Chinese. Here idiomaticity was used as an equivalent to what Newmark defined as “naturalness”. It meant that a translation should “read naturally” and “it is written in ordinary language, the common grammar, idioms and words that meet that kind of situation (Newmark, 1988, p. 24)”.
The fuzzy nature of language makes it inappropriate to adopt only a quantitative criterion as the sole criterion. Here the scoring procedure embraced two steps:
The first step was to decide, according to a native speaker’s linguistic knowledge, whether the meaning of a translation was completely unfaithful to that of the original or totally unintelligible. In either case, the translation scored zero.
In the second step, those translations which did not score zero were assessed against a set of scoring criteria set up on the basis of the above discussion
The following table shows how the scoring criteria were structured:
Table 2.2.2-1 Scoring criteria
|
Fidelity (5) |
Intelligibility (5) |
|||||||
|
Referential Meaning (2.5) |
Grammatical Meaning (2.5) |
Grammaticality (3) |
Fluency (2) |
|||||
|
Word / Phrase Level |
Sentence Level |
Inflectional Morphology (1.0) |
Syntax (1.5) |
Word Order (1.5) |
Word Class (1.0) |
Punctuation Mark (0.5) |
Collocation (1.0) |
Idiom-aticity (1.0) |
|
0.5 |
1.0 |
0.25 |
0.5 |
0.5 |
0.5 |
0.25 |
0.5 |
0.5 |
(unit: point)
1 Fidelity (5 points)
1.1 Referential Meaning (2.5 points)
1.1.1 Referential meaning at lexical level
1.1.1.1 Word/ phrase meaning missing
1.1.1.2 Additional word/phrase meaning
1.1.1.3 Wrong translation of word/phrase meaning
1.1.1.4 Unclear/improper translation of word/phrase meaning
1.1.1.5 Untranslated word/phrase meaning
1.1.2 Referential meaning at sentence level
1.2 Grammatical Meaning (2.5 points)
1.2.1 Inflectional Morphology (1.0 point)
1.2.1.1 Grammatical meaning missing
1.2.1.1.1 Tense
1.2.1.1.2 Aspect
1.2.1.1.3 Case
1.2.1.1.4 Gender
1.2.1.1.5 Mood
1.2.1.1.6 Number
1.2.1.1.7 Person
1.2.1.1.8 Voice
1.2.1.2 Wrong translation of grammatical meaning
1.2.1.2.1 Tense
1.2.1.2.2 Aspect
1.2.1.2.3 Case
1.2.1.2.4 Gender
1.2.1.2.5 Mood
1.2.1.2.6 Number
1.2.1.2.7 Person
1.2.1.2.8 Voice
1.2.2 Syntax (1.5 points)
1.2.2.1 Grammatical meaning missing
1.2.2.1.1 Function words
1.2.2.1.2 Grammatical functions
1.2.2.1.3 Sentence types
1.2.2.2 Wrong translation of grammatical meaning
1.2.2.2.1 Function words
1.2.2.2.2 Grammatical functions
1.2.2.2.3 Sentence types
2 Intelligibility (5 points)
2.1 Grammaticality (3 points)
2.1.1 Word order (1.5 points)
2.1.2 Word class (1.0 point)
2.1.3 Punctuation marks (0.5 point)
2.2 Fluency (2 points)
2.2.1 Collocation (1.0 point)
2.2.2 Idiomaticity (1.0 ponit )
The full mark of each translation was 10 points: 5 for “fidelity” and 5 for “intelligibility”. Different weights were assigned to “referential meaning”, “grammatical meaning”, “grammaticality”, and “fluency”. Every translation lost points according to the nature and number of the errors it made. For an error at “referential meaning at sentence level”, 1.0 point was missing. Once an error was made at “syntax”, 0.5 point was distracted. The same weight was also given to “word order”, “collocation”, and “idiomaticity” because of the difficulty in realizing them in the process of translation. For those below the sentence level, i.e., lexical meaning errors, wrong tenses, etc., 0.25 point was distracted. An error in punctuation mark also cost 0.25 point. After all the points caused by all the errors were distracted from the 10 points, the remaining points were the final score that reflected the overall quality of the output.
All the sentences in the suite were translated on the computer by the above 8 MT programs. The output, i.e., the translations, was gathered for further analysis. Then the criteria for measure were applied to score the output of each program. And whether the programs had correctly translated each testing point was judged.
The analysis of the output was mainly a quantitative one. The means of the total scores and the means on each subject were calculated. Then the overall rates of correctness of the translation of the testing points and the rates of correctness on each subject were also calculated. The means of the total scores and the overall rates of correctness were compared to answer the first research question. The means and the rates on each subject were compared and the translation of the testing points was analyzed to answer the second research question. There was one special kind of output. In some translations of DrEye Deluxe Pack, several choices were provided for the translation of some difficult parts of a sentence. In this research, the most appropriate choices were selected for the translation of such sentences.
CHAPTER THREE RESULTS AND DISCUSSION
Most translations fell somewhere between impressive and nonsensical. In general they were fairly understandable, if odd and stilted. They were discussed at length in this chapter, which went from the general comparison to the detailed analysis in accordance with the 9 subjects of the test suit.
The means of the total scores and the rates of correctness of the output of all the 8 programs are shown in Table 5.1-1.
Table 5.1-1 Means of total scores & rates of correctness
|
Program |
Mean of Total Scores (point) |
Rate of Correctness (%) |
|
Kingsoft FastAIT 2003 |
7.78 |
61.03 |
|
Oriental Express 2003 |
8.53 |
73.88 |
|
TranStar III |
8.56 |
74.95 |
|
LingoWare |
4.41 |
30.62 |
|
DrEye Deluxe Pack |
9.11 |
80.73 |
|
SYSTRANBox |
8.07 |
64.45 |
|
Homepage Translator 2000 |
8.11 |
69.52 |
|
GBTP Global Access |
9.32 |
85.01 |
It could easily be seen from Table 5.1-1 that GBTP Global Access performed best in terms of means of total scores and rates of correctness. But the ranking of the means of total scores and that of the rates of correctness displayed some disparities. For the former, the ranking of the programs was:
Table 5.1-2 Ranking of the programs in terms of means of total scores
|
Ranking |
Program |
|
1 |
GBTP Global Access |
|
2 |
DrEye Deluxe Pack |
|
3 |
TranStar III |
|
4 |
Oriental Express 2003 |
|
5 |
Homepage Translator 2000 |
|
6 |
SYSTRANBox |
|
7 |
Kingsoft FastAIT 2003 |
|
8 |
LingoWare |
In Table 5.1-2, GBTP Global Access ranked the first place. It scored 0.21 point higher than the running-up DrEye Deluxe Pack, which was 0.55 point higher than TranStar III. There was a slight difference (0.03 point) between TranStar III and Oriental Express 2003. It was the same case with SYSTRANBox and Homepage Translator 2000 (0.04 point). The difference was 0.43 point between Oriental Express 2003 and SYSTRANBox and 0.31 point difference between Homepage Translator 2000 and Kingsoft FastAIT 2003. In the lowest place ranked LingoWare with 3.37 points lower than the preceding Kingsoft FastAIT 2003.
For the latter, the ranking is:
Table 5.1-3 Ranking of the programs in terms of rate of correctness
|
Ranking |
Program |
|
1 |
GBTP Global Access |
|
2 |
DrEye Deluxe Pack |
|
3 |
TranStar III |
|
4 |
Oriental Express 2003 |
|
5 |
Homepage Translator 2000 |
|
6 |
SYSTRANBox |
|
7 |
Kingsoft FastAIT 2003 |
|
8 |
LingoWare |
Table 5.1-3 was the same with Table 5.1-2. It should be noticed that the difference between the mean of Homepage Translator 2000 and that of SYSTRANbox would have been higher if it were not for the fact that the former lost 0.25 point for each declarative sentence due to its failure to transfer the English period into its Chinese counterpart. Given this reason, the overall output quality of Homepage Translator 2000 was regarded as better than that of SYSTRANBox.
On the basis of the above data and analysis, a general conclusion concerning the first research question could be drawn. It was reflected by the ranking in terms of the overall quality of the 8 MT programs as shown in Table 5.1-4.
Table 5.1-4 Ranking of the programs in terms of overall output quality
|
Ranking |
Program |
|
1 |
GBTP Global Access |
|
2 |
DrEye Deluxe Pack |
|
3 |
TranStar III |
|
4 |
Oriental Express 2003 |
|
5 |
Homepage Translator 2000 |
|
6 |
SYSTRANBox |
|
7 |
Kingsoft FastAIT 2003 |
|
8 |
LingoWare |
3.2 Analysis of the Output Quality in Terms of Separate Subjects
One thing demonstrated by all of the effort in MT research is that language is far more complex than even linguists ever imagined. As a result, further analysis of the output can not be conducted in a meticulous way. It was based on the 9 subjects of the test suite instead of on the scrutinized criteria for measure established in Chapter Two. The discussion below aimed to provide the answer to the other research question.
This subject comprised 3 sub-subjects including “the most common words”, “common words” and “less common words”. There were 13 testing points in all. The means of the total scores and the rates of correctness on this subject were shown in Table 5.2.1-1 and Table 5.2.1-2.
Table 5.2.1-1 The means of the total scores on the subject of “lexical coverage”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
9.77 |
|
2 |
TranStar III |
9.35 |
|
3 |
Homepage Translator 2000 |
9.31 |
|
4 |
DrEye Deluxe Pack |
9.30 |
|
5 |
Oriental Express 2003 |
9.12 |
|
6 |
SYSTRANBox |
8.58 |
|
7 |
Kingsoft FastAIT 2003 |
8.38 |
|
8 |
LingoWare |
5.60 |
Table 5.2.1-2 The rates of correctness on the subject of “lexical coverage”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
100 |
|
2 |
DrEye Deluxe Pack & TranStar III |
84.62 |
|
3 |
Homepage Translator 2000 & Oriental Express 2003 |
76.92 |
|
4 |
SYSTRANBox |
69.23 |
|
5 |
Kingsoft FastAIT 2003 |
61.53 |
|
6 |
LingoWare |
53.85 |
A comparison of the above two tables displayed an evident disagreement. In the former, all the means except that of Lingoware was relatively high while in the latter, the rates varied significantly. The reason was that the assessment of the overall quality of a translation was far more complicated (as shown in the establishment of the criteria for measure in Chapter Two) than the yes-no judgment of the translation of a single testing point. The correct translation of the testing point contributed to the overall quality but did not necessarily bring forth a high score. It was the same vise versa. The same reason might help explain another disagreement. In Table 5.2.1-1, the means of DrEye Deluxe Pack, Homepage Translator 2000 and TranStar III were very near while in Table 5.2.1-2, the rate of Homepage Translator 2000 was apparently much lower than those of the other two programs. Presumably, Homepage Translator 2000’s advantages in other dimensions compensated its relatively narrow lexical coverage. Similar disagreements would emerge later in the succeeding sections.
The 13 sentences in this subject were simple sentences whose word order resembled that of their Chinese translation. As a result, judging from Table 5.2.1-1, all programs except LingoWare did a fairly good job. Among the testing points subsumed in this subject, the most unsuccessfully translated one was “most common numerals” (testing point 1.1.5).
Original sentence:
School begins at nine.
Translations:
1. 学校开始九。(Translated by Kingsoft FastAIT 2003)
2. 在9点学校开始。(Translated by Oriental Express 2003 )
3. 学校从九开始。(Translated by TranStar III)
4. 学校开始在九. (Translated by LingoWare)
5. 学校在九点开始。(Translated by DrEye Deluxe Pack)
6. 学校开始在九。(Translated by SYSTRANBox)
7. 学校从九开始. (Translated by Homepage Translator 2000)
8. 上课 9 点开始。(Translated by GBTP Global Access )
Among the all the 8 translations, only GBTP Global Access correctly translated the whole sentence. Strictly speaking, none of the other 7 translations was correct. But in terms of the testing point, translation 2 and 5 were right in translating nine into “九点” (or “9点”). Other difficult points were “the most common numerals” and “common verbs”.
Superficially, a dictionary (or dictionaries) with a large enough number of entries seemed to suffice to widen an MT system’s lexical coverage. However, sometimes the cause of these lexical errors lied in the inability to deal with the semantic ambiguities in words (lexical ambiguities). That explained why GBTP Global Access and TranStar III excelled and why Kingsoft FastAIT 2003 fell behind. DrEye Deluxe Pack’s distinguished itself by providing several possible equivalents for the same words or phrases.
This subject comprised 3 sub-subjects including “ordinary phrases consisting of adjacent words”, “split phrases”, and “phrases with other phrases inserted”. There were 12 testing points in all. Unlike the one-word testing points in the preceding sub-subject, the testing points here consisted of two words or more. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.2-1 and Table 5.2.2-2.
Table 5.2.2-1 The means of the total scores on the subject of “phrases”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
9.65 |
|
2 |
DrEye Deluxe Pack |
9.40 |
|
3 |
TranStar III |
9.19 |
|
4 |
Oriental Express 2003 |
8.88 |
|
5 |
Homepage Translator 2000 |
7.81 |
|
6 |
Kingsoft FastAIT 2003 |
6.65 |
|
7 |
SYSTRANBox |
5.05 |
|
8 |
LingoWare |
3.52 |
Table 5.2.2-2 The rates of correctness on the subject of “phrases”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
100 |
|
2 |
DrEye Deluxe Pack & Oriental Express 2003 |
91.67 |
|
3 |
TranStar III & Homepage Translator 2000 |
83.33 |
|
4 |
Kingsoft FastAIT 2003 |
58.33 |
|
5 |
SYSTRANBox |
50.00 |
|
6 |
LingoWare |
16.67 |
Except LingoWare whose word-for-word translations involving no disambiguation or restructuring, all the other programs did fairly well in translating ordinary phrases consisting of adjacent words. It was not the same case with the translation of “split phrases” and “phrases with other phrases inserted”, however. Testing point 2.2.4, i.e., “V-P phrases with a reflexive pronoun (corresponding to the subject inserted)”, was taken as an example.
Original sentence:
We must never divorce ourselves from the masses.
Translations:
1. 我们一定从不从块与我们自己办离婚。(Translated by Kingsoft FastAIT 2003)
2. 我们必须从来不使我们自己从群众中分离。(Translated by Oriental Express 2003 )
3. 我们必须从不把我们自己与群众相离婚。(Translated by TranStar III)
4. 我们从未必须离婚我们自己从群众. (Translated by LingoWare)
5. 我们自己从不(决不)一定(要)离婚从(这些)群众那里。(Translated by DrEye Deluxe Pack)
6. 我们必须从未与离婚从大量。(Translated by SYSTRANBox)
7. 我们一定不要使我们自己脱离群众. (Translated by Homepage Translator 2000)
8. 我们决不能脱离群众。(Translated by GBTP Global Access)
Translation 1, 4, 5, and 6 were totally unintelligible and nonsensical. Translation 3 was intelligible but nonsensical. Translation 2 was both intelligible and sensible with minor faults (incorrect equivalent for “never” and unnatural Chinese). Translation 7 was a correct one except for the punctuation mark. Only Translation 8 was perfectly correct.
The translation of split phrases and phrases with other phrases inserted involved detailed analysis of syntactic structures and correct rearrangement of word order in the target language. This time GBTP Global Access, DrEye Deluxe Pack and Oriental Express 2003 took the lead with TranStar III and Homepage Translator 2000 following, and LingoWare fell far behind again.
This subject comprised 9 sub-subjects, among which were “plural forms of nouns”, “case of nouns and pronouns”, “the comparative degree and the superlative degree of adjectives”, “the comparative degree and the superlative degree of adverbials”, “irregular changes of verbs”, “infinitive”, “present participle of verbs”, “gerund”, and “past participle”. Not only the translation of the morphological categories but also that of the non-finite verbs was tested. There were 99 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.3-1 and Table 5.2.3-2.
Table 5.2.3-1 The means of the total scores on the subject of “morphology”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
DrEye Deluxe Pack |
9.67 |
|
2 |
GBTP Global Access |
9.38 |
|
3 |
TranStar III |
9.02 |
|
4 |
Oriental Express 2003 |
8.67 |
|
5 |
SYSTRANBox |
8.07 |
|
6 |
Homepage Translator 2000 |
7.97 |
|
7 |
Kingsoft FastAIT 2003 |
7.91 |
|
8 |
LingoWare |
4.77 |
Table 5.2.3-2 The rates of correctness on the subject of “morphology”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
92.93 |
|
2 |
DrEye Deluxe Pack |
90.30 |
|
3 |
Oriental Express 2003 & TranStar III |
80.81 |
|
4 |
Homepage Translator 2000 |
77.78 |
|
5 |
SYSTRANBox |
72.73 |
|
6 |
Kingsoft FastAIT 2003 |
67.68 |
|
7 |
LingoWare |
39.39 |
Generally speaking, all programs performed quite well on “plural forms of nouns” and “case of nouns and pronouns”. For other sub-subjects, their performance varied. The most unsuccessfully translated testing points were:
■ Special usage of nominal possessive case (Testing Point 3.2.4)
■ Infinitive as adverbial (Testing Point 3.6.6)
■ Present participle as adverbial denoting time, result, reason, condition and purpose (with the same function as that of a sentence or a clause) (Testing Point 3.7.5)
■ The perfect form and the passive voice of present participle (Testing Point 3.7.6)
■ The perfect form and passive voice of a gerund (Testing Point 3.8.6)
■ Past participle as objective complement of a complex object (Testing Point 3.9.4)
Here the translations of a sentence from testing point 3.8.6 were taken as an example.
Original sentence:
He was punished by being sent to bed without any supper.
Translations:
1. 他被处罚藉由被送去不需要任何的晚餐就能固定。(Translated by Kingsoft FastAIT 2003)
2. 他被没有任何晚饭被送到床惩罚。(Translated by Oriental Express 2003)
3. 他受到没有任何晚饭而发送到床的处罚。(Translated by TranStar III)
4. 他被惩罚了通过是发送到床没有任何晚餐. (Translated by LingoWare)
5. 他由没有任何晚餐送到床惩罚。(Translated by DrEye Deluxe Pack)
6. 他由被送惩罚了到床没有任一顿晚饭。(Translated by SYSTRANBox)
7. 他通过没有任何晚饭让上床去被罚. (Translated by Homepage Translator 2000)
8. 他通过被送上床而没有任何晚饭被处罚。(Translated by GBTP Global Access)
The testing point (i.e., being sent) was correctly treated in Translation 1, 2, 6, 7 and 8. But none of these translations was near-correct. The combination of gerund and passive voice complicated the syntactic structure and led to different degrees of confusion. The co-occurrence of a passive voice (was punished) and a by-phrase added to the confusion in that the latter usually constituted part of a passive voice construction indicating the agent of the verb but was the adverbial of manner in this sentence. The meaning of the original sentence could only be guessed from these scrambled strings of words.
However, a closer observation revealed significant differences. Translation 3 was the best of all. Except for the inappropriate translation of sent, it was both faithful to the original sentence and intelligible. Clearly, GBTP Global Access and TranStar III had made an accurate analysis of the original sentence, grammatically and semantically. As mentioned in Chapter Two, TranStar III’s translation process involved a “logical semantic theory” based on case grammar. It followed that such a semantic analysis was useful for translating sentences with complicated structures. Likewise, GBTP Global Access’s translation process might have also involved similar analysis.
This subject comprised eight sub-subjects, among which were “basic sentence patterns of declarative sentences”, “tense in a declarative sentence”, “passive voice in declarative sentences”, “interrogative sentences”, “imperative sentences and exclamatory sentences”, “negative sentences”, “auxiliary verbs”, and “coordination”. There were 75 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.4-1 and Table 5.2.4-2.
Table 5.2.4-1 The means of the total scores on the subject of “simple sentences”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
9.75 |
|
2 |
Oriental Express 2003 |
9.30 |
|
3 |
DrEye Deluxe Pack |
9.11 |
|
4 |
Kingsoft FastAIT 2003 |
8.74 |
|
5 |
SYSTRANBox |
8.47 |
|
6 |
TranStar III |
8.18 |
|
7 |
Homepage Translator 2000 |
7.97 |
|
8 |
LingoWare |
5.65 |
Table 5.2.4-2 The rates of correctness on the subject of “simple sentences”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
93.33 |
|
2 |
Oriental Express 2003 |
86.67 |
|
3 |
DrEye Deluxe Pack |
81.33 |
|
4 |
TranStar III |
78.67 |
|
5 |
Homepage Translator 2000 |
74.67 |
|
6 |
Kingsoft FastAIT 2003 |
72.00 |
|
7 |
SYSTRANBox |
70.67 |
|
8 |
LingoWare |
50.67 |
Undoubtedly, this time Oriental Express 2003’s performance was remarkable, showing the excellence of the S-speed technology in dealing with simple sentence translation. The ranking of the other programs did not show any significant difference.
It could be observed that the more similar the word order of the original was to Chinese word order, the better the output would be.
The most unsuccessfully translated points were:
■ “Subject + linking verb + adjective + complement” construction (Testing Point 4.1.7)
■ Future indefinite (present participle indicating future indefinite), (Testing Point 4.2.2)
■ Future continuous (Testing Point 4.2.5)
■ Different forms of interrogative sentences with have as predicate verb, which is an important clue to their translation. (Testing Point 4.4.2)
■ Combination of imperative sentences (Testing Point 4.5.4)
■ “Imperative sentence + or + declarative sentence” construction (Testing Point 4.5.5)
■ Coordination of nominal elements (including non-finite verbs) (Testing Point 4.8.1)
Again these points were relatively complicated ones. Take the translations of Testing point 4.2.5 as an example:
Original sentence:
I shall be working when you come.
Translations:
1. 当你来的时候, 我将工作。(Translated by Kingsoft FastAIT 2003)
2. 当你来时我将工作。(Translated by Oriental Express 2003 )
3. 当你来时，我将工作。(Translated by TranStar III)
4. 我shall正在工作当您来. (Translated by LingoWare)
5. 当你(们)来了时, 我将工作。(Translated by DrEye Deluxe Pack)
6. 我工作当您来。(Translated by SYSTRANBox)
7. 当你来的时候,我将工作. (Translated by Homepage Translator 2000)
8. 当你来时，我将工作。(Translated by GBTP Global Access)
All the programs failed to correctly translate the testing point. Translation 1, 2, 3, 5, 7, and 8 lexicalized the grammatical meaning of “future continuous” with “将” (indicating future indefinite in Chinese). Translation 4 did not translate the auxiliary shall and in Translation 6 the meaning was simply missing. Failures of this kind revealed that the rules the translation engines operated on were incomplete.
This subject comprised 8 sub-subjects, among which were “object clause”, “attributive clause”, “subject clause”, “appositive clause”, “predicative clause”, “adverbial clause”, “embedding”, and “compound sentences”. There were 61 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.5-1 and Table 5.2.5-2.
Table 5.2.5-1 The means of the total scores on the subject of “complex sentences”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
DrEye Deluxe Pack |
8.92 |
|
2 |
GBTP Global Access |
8.91 |
|
3 |
Oriental Express 2003 |
8.21 |
|
4 |
TranStar III |
8.19 |
|
5 |
Kingsoft FastAIT 2003 |
8.02 |
|
6 |
SYSTRANBox |
7.82 |
|
7 |
Homepage Translator 2000 |
7.14 |
|
8 |
LingoWare |
4.12 |
Table 5.2.5-2 The rates of correctness on the subject of “complex sentences”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
DrEye Deluxe Pack |
88.52 |
|
2 |
GBTP Global Access |
85.97 |
|
3 |
Oriental Express 2003 |
85.25 |
|
4 |
TranStar III |
81.97 |
|
5 |
Homepage Translator 2000 |
72.13 |
|
6 |
SYSTRANBox |
70.49 |
|
7 |
Kingsoft FastAIT 2003 |
67.21 |
|
8 |
LingoWare |
34.43 |
The rankings in the above two tables were consistent with the ranking in the general quality in section 5.1.
The most unsuccessfully translated testing points were:
■ Attributive clause introduced by “preposition + which” or “preposition + whom” construction (Testing Point 5.2.7)
■ Subject clause introduced by when (Testing Point 5.3.3)
■ Subject clause introduced by where or wherever (Testing Point 5.3.8)
■ Adverbial clause of comparison (Testing Point 5.6.9)
■ The main clause or the nominal clause is also a complex sentence. (Testing Point 5.7.1)
■ The attributive clause is also a complex sentence. (Testing Point 5.7.2)
■ Neither of the two clauses in one sentence is embedded in the other clause. (Testing Point 5.7.4)
Such result was not difficult to imagine and understand. Again all the programs resorted to their respective resources as well as technologies. The second sentence of Testing Point 5.2.7 was taken as an example:
Original sentence:
In our class there are fifteen students, three of whom are from Beijing.
Translations:
1. 在我们的班级那里中是十五位学生, 三谁来自北京。 (Translated by Kingsoft FastAIT 2003)
2. 在我们的高材生有15名学生，谁的的3从北京。 (Translated by Oriental Express 2003)
3. 在我们班中有十五名学生，其中三个来自北京。 (Translated by TranStar III)
4. 在我们的类别有十五学生，三属于谁是从“北京”. (Translated by LingoWare)
5. 在我们的班级(课)中有十五个学生, 来自北京的三个。 (Translated by DrEye Deluxe Pack)
6. 在我们的类有十五名学生, 三谁是从北京。(Translated by SYSTRANBox)
7. 三来自北京十五个学生是在我们那里课中. (Translated by Homepage Translator 2000)
8. 在我们班有15 个学生，其中3 人是来自北京。 (Translated by GBTP Global Access)
Only GBTP Global Access, DrEye Deluxe Pack, and TranStar III correctly translated the attributive clause introduced by the of + whom construction. DrEye Deluxe Pack did relatively better in that it rendered the whole sentence in a more natural way and the compensated classifier measure word of students in Chinese was also consistent with the compensated one of the omitted students in the clause.
3.2.6 Syntactic Ambiguity and Semantic Analysis
This subject comprised 6 sub-subjects, among which were “words belonging to two word classes”, “words belonging to three word classes”, “verbs belonging to different subclasses”, “polysemants”, “ambiguity”, and “unrecorded words”. There were 87 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.6-1 and Table 5.2.6-2.
Table 5.2.6-1 The means of the total scores on the subject of “syntactic ambiguity and semantic analysis”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
DrEye Deluxe Pack |
9.23 |
|
2 |
GBTP Global Access |
9.14 |
|
3 |
TranStar III |
8.89 |
|
4 |
Homepage Translator 2000 |
8.51 |
|
5 |
Oriental Express 2003 |
8.42 |
|
6 |
SYSTRANBox |
8.30 |
|
7 |
Kingsoft FastAIT 2003 |
7.94 |
|
8 |
LingoWare |
3.93 |
Table 5.2.6-2 The rates of correctness on the subject of “syntactic ambiguity and semantic analysis”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
DrEye Deluxe Pack |
85.60 |
|
2 |
TranStar III |
82.76 |
|
3 |
Homepage Translator 2000 & GBTP Global Access |
81.61 |
|
4 |
Oriental Express 2003 |
77.01 |
|
5 |
SYSTRANBox |
71.26 |
|
6 |
Kingsoft FastAIT 2003 |
63.21 |
|
7 |
LingoWare |
29.89 |
The rankings in the above two tables differed perceivably from the ranking of the general quality in section 5.1 in that Homepage Translator 2000 and Oriental Express 2003 exchanged their places and GBTP Global Access downgraded.
The most unsuccessfully translated testing points were:
■ Words belonging to noun and adjective (Testing Point 6.1.2)
■ Words belonging to adverbial and adjective (Testing Point 6.1.4)
■ Verbs with different meanings in accordance with what follows (Testing Point 6.3.4)
■ Subordinate conjunctions which have various meanings and introduce different types of clauses (Testing Point 6.4.6)
■ Nouns with three different meanings (Testing Point 6.4.7)
■ To judge the coordination and the modification in “N1 and N2 of N3” construction and “N1 of N2 and N3” construction (Testing Point 6.5.1)
■ Complex sentences which are semantically compound ones (Testing Point 6.5.4)
■ To judge whether who is the logical subject or the object of the infinitive (Testing Point 6.5.5)
■ To judge whether a V-ing + N construction is a predicate-object construction or an attribute-head construction (Testing Point 6.5.6)
■ Unrecorded names of places (Testing Point 6.6.2)
■ Unrecorded proper names in appositive phrases (Testing Point 6.6.3)
Like the case in the preceding sections, the above points were complicated grammatical phenomenon (e.g., Testing Point 6.5.5), uncommon names (e.g., Testing Point 6.6.2), and were easy for a human translator but difficult for an MT program (e.g., Testing Point 6.1.2).
Testing Point 6.5.5 was taken as an example:
Original sentences:
a. Who does she want to come here?
b. Who does she want to reproach?
Translations:
1. Kingsoft FastAIT 2003
1a. 世界卫生组织她来这里吗?
1b. 世界卫生组织她责备吗?
2. Oriental Express 2003
2a. 她想要谁来这里？
2b. 她想要谁到责备？
3. TranStar III
3a. 她想要谁来这里？
3b. 她想要责备谁？
4. LingoWare
4a. 谁她是否想到这里来?
4b. 谁她是否想以叱责?
5. DrEye Deluxe Pack
5a. 谁她想要来这里?
5b. 她想要责备谁?
6. SYSTRANBox
6a. 谁她想要来这里?
6b. 谁她想要责备?
7. Homepage Translator 2000
7a. 她想要谁来这里?
7b. 她想要责备谁?
8. GBTP Global Access
8a. 她想要谁来这里？
8b. 她想要责备谁？
Only GBTP Global Access, TranStar III and Homepage Translator 2000 produced satisfactory translations. But those of Oriental Express 2003, DrEye Deluxe Pack and SYSTRANBox were faithful and intelligible to varying degrees.
Testing Point 6.5.6 was taken as another example:
Original sentences:
a. Flying planes is dangerous.
b. Flying planes are dangerous.
Translations:
1. Kingsoft FastAIT 2003
1a. 飞的飞机是危险的。
1b. 飞的飞机是危险的。
2. Oriental Express 2003
2a. 开飞机是危险的。
2b. 飞的飞机是危险的。
3. TranStar III
3a. 架飞行的飞机是危险的。
3b. 飞行的飞机是危险的。
4. LingoWare
4a. 飞行的飞机是dangerous.
4b. 飞行的飞机是dangerous.
5. DrEye Deluxe Pack
5a. 飞飞机危险。
5b. 驾驶飞机危险。
6. SYSTRANBox
6a. 架飞行的飞机是危险的。
6b. 飞行飞机是危险的。
7. Homepage Translator 2000
7a. 驾驶飞机是危险.
7b. 乘飞机飞机是危险.
8. GBTP Global Access
8a. 驾驶飞机是危险的。
8b. 驾驶飞机是危险的。
The focus of this testing point is to test a program’s ability to treat structural ambiguity. The key to solve this ambiguity lies in the copula (to be). For the first sentence, GBTP Global Access, Oriental Express 2003, TranStar III, and Homepage Translator 2000 correctly translated the predicate-object gerundial phrase. For the second, Kingsoft FastAIT 2003, Oriental Express 2003, TranStar III, and Lingoware correctly translated the attribute-head gerundial phrase. For both, only Oriental Express 2003 and TranStar III did it right while SYSTRANBox and DrEye Deluxe Pack failed. Interestingly, GBTP Global Access, Kingsoft FastAIT 2003 and Lingoware provided the same translation for both sentences. It stood to reason to say that the relatively better translation of TranStar III and Homepage Translator 2000 achieved here was rooted in the semantic model in their respective translation engines.
This subject comprised 6 sub-subjects, among which were “the translation of articles”, “the usage of Chinese classifier measure words”, “expressions of negation”, “the passive sentences translated into active ones”, “change of word order”, and “proper decision of referential relationship and exchange of word orders”. There were 56 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.7-1 and Table 5.2.7-2.
Table 5.2.7-1 The means of the total scores on the subject of “generation of Chinese”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
9.40 |
|
2 |
DrEye Deluxe Pack |
9.32 |
|
3 |
Homepage Translator 2000 |
8.91 |
|
4 |
Oriental Express 2003 |
8.73 |
|
5 |
TranStar III |
8.65 |
|
6 |
SYSTRANBox |
8.61 |
|
7 |
Kingsoft FastAIT 2003 |
8.52 |
|
8 |
LingoWare |
5.76 |
Table 5.2.7-2 The rates of correctness on the subject of “generation of Chinese”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
78.57 |
|
2 |
DrEye Deluxe Pack |
75.00 |
|
3 |
TranStar III |
64.29 |
|
4 |
Homepage Translator 2000 |
62.50 |
|
5 |
Oriental Express 2003 |
60.71 |
|
6 |
SYSTRANBox |
55.36 |
|
7 |
Kingsoft FastAIT 2003 |
51.79 |
|
8 |
LingoWare |
8.93 |
The rankings in the above two tables were slightly different in that Homepage Translator 2000 and Oriental Express 2003 exchanged their places. In terms of rates of correctness, the performance of all the 8 programs was comparatively unsatisfactory.
The most unsuccessfully translated testing points were:
■ The indefinite article a before a noun as a unit of measure should be translated into “每” or “一”. (Testing Point 7.1.4)
■ The positive degree of some adjectives used in combination with the denotes one kind of things and should be translated into nouns. (Testing Point 7.1.7)
■ The English structure “numeral + adjective + nouns” should be translated into Chinese structure “numeral + classifier measure word + adjective + noun” (numeral including indefinite article). (Testing Point 7.2.3)
■ The choice of “不”, “没”, “” and “未” when translating the adverb not (Testing Point 7.3.1)
■ The translation of negative pronouns (none, nothing, nobody, neither, no one, etc.) (Testing Point 7.3.2)
■ Negative sentences should be translated into assertive ones. (Testing Point 7.3.7)
■ Logical direct object + passive form of verb + indirect object + by + logical subject (Testing Point 7.4.1)
■ Logical indirect object + passive form of verb + direct object + by + logical subject (Testing Point 7.4.2)
■ When some passive sentences are translated into active ones, logical subjects “有人”, “人们”, etc. should be added. (Testing Point 7.4.3)
■ In translation, some English modifiers before the modified words are translated into post modifiers. (Testing Point 7.5.1)
■ Not only word order of the modifier and the modified words but also the word order within the modifying components should be changed. (Testing Point 7.5.2)
■ The change of word order of apposition (Testing Point 7.5.7)
■ The main clause is translated into subordinate clause, while the subordinate clause is translated into main clause. (Testing Point 7.5.8)
■ When the adverbial is put before the main clause in translation, the pronoun in the subordinate clause should be translated into the noun in the main clause. (Testing Point 7.6.1)
The low rates of correctness and the long list of unsuccessfully translated testing points implied the relatively poor ability of these programs to generate Chinese, as shown in the translations of the sentences in Testing Point 7.5.2.
Original sentence:
There were quite a few people on the platform waiting for the train coming at night.
Translations:
1. 在月台上的相当多的人正在等候火车在晚上过来。(Translated by Kingsoft FastAIT 2003)
2. 在等在晚上来的火车的月台上有很多的人。(Translated by Oriental Express 2003 )
3. 在等候夜间来的火车的平台上有许多人们。(Translated by TranStar III)
4. 当以前是几人在操作平台正在等待为火车来临在夜晚. (Translated by LingoWare)
5. 在等待夜里来的火车的站台有相当不少人。(Translated by DrEye Deluxe Pack)
6. 有一些人们在平台等火车来在晚上。(Translated by SYSTRANBox)
7. 在站台上有相当多人,晚上等待火车来到. (Translated by Homepage Translator 2000)
8. 关于政纲等在夜里来的火车有相当多人。(Translated by GBTP Global Access)
3.2.8 Special Difficulties in English-Chinese Machine Translation
This subject comprised 7 sub-subjects, among which were “emphasis”, “omission”, “inversion”, “the usages and the translation of some pronouns”, “split structures”, “subjunctive mood”, and “punctuation marks”. There were 39 testing points in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.8-1 and Table 5.2.8-2.
Table 5.2.8-1 The means of the total scores on the subject of “difficulties in ECMT”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
9.23 |
|
2 |
DrEye Deluxe Pack |
8.89 |
|
3 |
TranStar III |
7.94 |
|
4 |
Oriental Express 2003 |
7.47 |
|
5 |
Homepage Translator 2000 |
7.30 |
|
6 |
Kingsoft FastAIT 2003 |
7.25 |
|
7 |
SYSTRANBox |
7.13 |
|
8 |
LingoWare |
1.64 |
Table 5.2.8-2 The rates of correctness on the subject of “difficulties in ECMT”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access |
70.49 |
|
2 |
DrEye Deluxe Pack |
60.66 |
|
3 |
TranStar III |
54.10 |
|
4 |
Oriental Express 2003 |
42.62 |
|
5 |
Homepage Translator 2000 |
40.98 |
|
6 |
Kingsoft FastAIT 2003 |
37.70 |
|
7 |
SYSTRANBox |
34.45 |
|
8 |
LingoWare |
8.20 |
The rankings in the above two tables were consistent with the ranking in the general quality in section 5.1. The rates of correctness became even lower.
The most unsuccessfully translated testing points were:
■ The emphatic sentence pattern: it is/was + the emphasized part + that/who/whom/which + the remaining part (Testing Point 8.1.3)
■ To be or the base-form verbs following to are often omitted in an objective complement or in a subjective complement. (Testing Point 8.2.1)
■ In simple sentences, in order to express liveliness, adverbials are sometimes placed at the beginning and predicate verbs are placed before the subject. (Testing Point 8.3.1)
■ In compound sentences connected by not only … but also, predicate verbs, modal verbs or auxiliary verbs are placed before the subject when not only is at the beginning of the sentences. (Testing Point 8.3.2)
■ When nor or neither means “also not”, the word order of the sentence is: nor/neither + be/have/modal verb/auxiliary verb + the subject. (Testing Point 8.3.4)
■ When so means “the same case”, the word order of the sentence is: so + be/have/modal verb/auxiliary verb + the subject. (Testing Point 8.3.5)
■ Linking verbs or auxiliary verbs are placed before the subject in sentences beginning with only, never, little, etc. (Testing Point 8.3.6)
■ It as an introductory word (Testing Point 8.4.1)
■ That (also this, these and those) (Testing Point 8.4.2)
■ As as a pronoun (Testing Point 8.4.3)
■ The translation of One (Testing Point 8.4.4)
■ All, each, few, little, another (Testing Point 8.4.6)
■ Demonstrative pronouns (Testing Point 8.4.8)
■ Split attributive clauses or participle phrases in a complex sentence (Testing Point 8.5.2)
■ Split prepositional phrases (Testing Point 8.5.3)
■ Subjunctive mood in the objective clauses introduced by that after some verbs (Testing Point 8.6.4)
■ Other forms of subjunctive mood (Testing Point 8.6.6)
■ Punctuation marks (Testing Point 8.7.1-8.7.5)
Testing Point 8.3.2 was taken as an example:
Original sentence:
Not only is this problem very important, but it is also difficult to solve at once.
Translations:
1. 不但是这一个问题很重要, 而且它也对立刻解决是困难的。(Translated by Kingsoft FastAIT 2003)
2. 不仅是很重要的这个问题，但是立刻解决也是困难的。 (Translated by Oriental Express 2003)
3. 不仅是这十分重要的问题，而且立即解决是也困难。(Translated by TranStar III)
4. 没有仅仅是此问题非常重要的， 但是它也是困难的以解决在一次. (Translated by LingoWare)
5. 这个问题不仅是很重要, 而且立即解决, 这也困难。(Translated by DrEye Deluxe Pack)
6. 不仅是这个问题非常重要, 但它还难立即解决。(Translated by SYSTRANBox)
7. 不仅这问题是非常重要, 但是也立即作解答是困难的. (Translated by Homepage Translator 2000)
8. 不仅这个问题是非常重要的，而且同时解决也是困难的。(Translated by GBTP Global Access)
The 8 programs reorganized their Chinese translation in 8 different ways. However, it was noteworthy that most of them were understandable (except translation 4).
This subject comprised no subcategory. There were 3 sentences in all. The means of the total scores and the rates of correctness of the output in this subject were shown in Table 5.2.9-1 and Table 5.2.9-2.
Table 5.2.9-1 The means of the total scores on the subject of “translation of long sentences”
|
Ranking |
MT Program |
Mean (point) |
|
1 |
GBTP Global Access |
7.67 |
|
2 |
Oriental Express 2003 |
7.16 |
|
3 |
DrEye Deluxe Pack |
6.17 |
|
4 |
Kingsoft FastAIT 2003 |
6.00 |
|
5 |
TranStar III |
5.75 |
|
6 |
SYSTRANBox |
5.00 |
|
7 |
Homepage Translator 2000 |
2.25 |
|
8 |
LingoWare |
0 |
Table 5.2.9-2 The rates of correctness on the subject of “translation of long sentences”
|
Ranking |
MT Program |
Rate of Correctness (%) |
|
1 |
GBTP Global Access, DrEye Deluxe Pack, TranStar III & SYSTRANBox |
66.67 |
|
2 |
Kingsoft FastAIT 2003 |
33.33 |
|
3 |
Oriental Express 2003, Homepage Translator 2000 & LingoWare |
0 |
Among the 9 subjects, the translations of the sentences in this subject scored the lowest due to the complexity of the sentence structures.
The most unsuccessfully translated testing point was “simple sentences with many or long modifiers” (Testing Point 9.1.1).
Original sentence:
The conservation laws have enabled us to reach far-reaching conclusions concerning the stability of atoms, without resorting to any hypothesis about the conditions within the nucleus on the forces operative in it.
Translations:
1. 保护法律已经使我们能够达成广大的结论关于原子的安定，不在它里面在力量职员上的核心里面诉诸任何的假设有关情况的事而。(Translated by Kingsoft FastAIT 2003)
2. 守恒定律已经使我们能够关于原子的稳定性得出影响深远的结论，没有关于条件在核子以内在在它有作用的力量上采取任何假设。(Translated by Oriental Express 2003)
3. 保护法律已使我们达成有关原子的稳定的深远的结论成为可能，没有在在它中操作的力量上的核之内诉诸任何假设关于条件。(Translated by TranStar III)
4. 保存法律已经启用了我们以达到远的-正在达到结束关于稳定属于原子，没有诉诸到任何假设关于conditions 在核心之内在强制operative 在它. (Translated by LingoWare)
5. 保护法(定)律允许我们得出涉及原子的稳定性的深远的结论, 没有在它(这)中关于力量(势力)操作方面的核之内的条件诉诸任何假设。(Translated by DrEye Deluxe Pack)
6. 保护法律使我们得出广远的结论关于原子的稳定,没有依靠对任一个假说关于条件在中坚力量之内在力量有效在它。(Translated by SYSTRANBox)
7. 保存法律已使我们能获得深远的,不关于在它中武装力量技工上核心以内状况诉诸任何假说与原子的稳定性有关结论. (Translated by Homepage Translator 2000)
8. 守恒定律已经使我们能够得出深远的关于原子的稳定的结论，在常去任何假说大约条件在核心内在在内操作的部队上时。(Translated by GBTP Global Access)
Unsurprisingly, the co-occurrence of several difficult grammatical phenomena brought about the total failure. However, Oriental Express 2003 and GBTP Global Access did provide readable translations for the main clause.
CHAPTER FOUR CONCLUSIONS
This chapter summarizes the major findings of the research, presents the implications for the MT programs development, and finally recognizes the limitations of the research and makes suggestions for future research.
In this research, the researcher endeavored to make an evaluation of the output (translations) quality of some prevalent English-Chinese Machine Translation programs. 8 fully-automatic MT programs and one test suite were chosen for the evaluation. The sentences in the test suite were translated by these programs and the output was gathered. Detailed criteria for measure were established in light of structural linguistic theories to score the output. The means of the scores and the rates of correctness were calculated. Based on the analysis of the means and the rates of correctness, the researcher arrived at the fsollowing conclusions concerning the output quality as well as the technologies of these programs:
(1) GBTP Global Access, with IMT/EC technology in its translation engine, produces the best output.
(2) DrEye Deluxe Pack, supported by its storage of over 40,000 words/phrases (covering 90 domains with explanations on the words’ origin and grammar) as well as a human-computer interaction technology, produces the second best output.
(3) TranStar III, which analyzes the source text using a logical semantic theory based on case grammar, wins the third place.
(4) Oriental Express 2003, featuring its S-speed translation engine and 49 dictionaries, wins the fourth place.
(5) Homepage Translator 2000, combining the Slot Grammar theory and the pattern-based translation technology and translating with the help of an English-Chinese pattern-based dictionary, wins the fifth place.
(6) SYSTRANBox and Kingsoft FastAIT 2003 also have advantages of their own but both failed to come up with good enough output. LingoWare’s performance is the worst among all the 8 programs. Judging from its output, its translation procedure simply consists of two steps: dictionary lookup and substitution.
Based on the most unsuccessfully testing points, the research also obtained findings concerning the major linguistic bottlenecks of these MT programs:
(1) Syntactic ambiguity and semantic analysis (as listed in Section 5.2.6)
(2) Target text (i.e., Chinese) generation (as listed in Section 5.2.7)
(3) Some special linguistic phenomena (emphasis, ellipsis, inversion, etc. as listed in Section 5.2.8)
(4) Long Sentences (as listed in Section 5.2.9)
The above 4 subjects contain the most unsuccessfully translated testing points which are mainly connected with the complexity in the syntactic and semantic analysis, or the difference between English and Chinese.
4.2 Implications of the Findings
In light of the findings of the present research, some useful implications can be obtained for future theoretical and practical research on fully-automatic English-Chinese MT programs.
Firstly, most programs performed fairly well in their treatment of relatively simple language phenomena, i.e., common words and phrases, morphology, and simple sentences. But it was not the same case with their performance on the relatively complicated language phenomena, especially syntactic ambiguity and long sentences, and the generation of the target language as well. The main reason may possibly lie in the imperfection of their translation engines, which failed to take many complicated language phenomena and other difficulties in MT into consideration. In the future research, more attention should be paid to the further studies on the language model of natural languages. Unlike the traditional linguistic studies oriented to human understanding, studies on the model of natural languages should be oriented to MT research. Importance should also be attached to the improvement of the design of MT engines. The achievements obtained in the studies of language model should be applied to MT engines, e.g., the rules underlying split phrases.
Secondly, the findings reveal the importance of incorporating appropriate technology (or technologies) into MT programs. If an MT program solely depends on a machine dictionary to translate, its output will inevitably be unreadable. The case in point is LingoWare. Unlike LingoWare, GBTP Global Access, DrEye Deluxe Pack, TranStar III, Oriental Express 2003, and Homepage Translator 2000 incorporate different MT technologies into their translation engines. As a result, their translation of both the sentences and the testing points were fairly satisfactory. In this sense, these technologies are relatively successful. If future MT program developers can incorporate more technologies like those above mentioned into the engines of their programs, the output quality will probably become better. However, seen from the general trend of MT development, MT technologies should not simply focus on the treatment of language phenomena. In order to fundamentally solve the problems confronted by present MT programs and to significantly improve their output quality, more knowledge-based technologies and corpus-based technologies should be developed and incorporated. The combination of different technologies, i.e., a hybrid approach, will prove highly effective.
4.3 Limitations of the Study and Suggestions for Further Research
As stated at the beginning of this thesis paper, this research is only an exploratory case study of the evaluation of the output quality of some prevalent English-Chinese Machine Translation programs. Consequently, several limitations or drawbacks constrained the extent to which the findings of the present research could be generalized. They are outlined as follows:
In the first place, the small sample size prevented further generalizations to all programs. The selection of the MT programs for evaluation complied with three criteria. However, there are enormous low-cost fully-automatic MT programs commercially available in China or accessible on the Internet. We could not exhaust them all but choose some typical ones. But there is no denying that some other typical programs were neglected unintentionally in this research.
In the second place, as a result of human involvement, this evaluation could not completely avoid subjectivity. The subjectivity was mainly rooted in the two steps of the evaluation process:
(1) In the field of MTE, there is no agreement as to what and how to access and thus the criteria for measure were established by the researcher. Every aspect in the criteria was decided by the researcher, from the various dimensions of measure to the value assigned to an error.
(2) When scoring the output, the researcher’s judgment inevitably involved some degree of subjectivity which might karmically affect the correctness of the result.
In the third place, given that sentence is the basic unit in the translation process of all the programs evaluated, this research only evaluated the output quality of sentences. In the future research, special attention should be paid to the institution of the test suite. It should be more comprehensive and systematic, not only every aspect of sentences (ranging from morphology, lexicon, and syntax to semantics) but also passages of different types. The testing points which test two or more language phenomena should also be included.
In the fourth place, this research is a black-box research. We concentrated on the evaluation of the output quality instead of what is really going on in these MT programs. But a clear picture of their workflow is surely conducive.
Despite its various drawbacks, the present research does provide some insight into possible avenues of investigation that may be conducted by future researchers who work in this direction. For example, continued researches should make a more comprehensive selection of typical MT programs, and the test suite can include more sentences for each testing point. Further studies can also make a white-box study of the output quality of MT programs.
NOTES
[1] EAGLES is the Expert Advisory Group on Language Engineering Standards at the University of Geneva.
[2] Slot Grammar Theory is established by the Language Analysis & Translation Research Group at IBM. A slot is a placeholder for the different parts of a sentence associated with a word. A word may have several slots associated with it, and these form a slot frame for the word. For example, in the sentence I give the chocolate to you, the word give has three slots: a subject (I), a direct object (chocolate) and an indirect object (you). In order to translate a sentence, the system first analyzes it. For each word, the Slot Grammar parser draws on the word’s slot frames to cycle through the possible sentence constructions. For example, the indirect object slot for give might be filled by to you or by you. Using a series of word relationship tests to establish context, the system then tries to determine the meaning of the sentence.
[3] Pattern-based translation uses a huge collection of translation patterns, each of which is a pair of source context-free-grammar (CFG) rule and its corresponding target CFG rule, and makes a translation by matching these translation patterns to the input.
[4] To parse a sentence is to assign labeled brackets to each constituent of a sentence.
[5] The parent company of Huajian Electronic Co., Ltd. is Huajian Group of CAS (Chinese Academy of Sciences), one of the six biggest high-tech enterprises directly under the CAS which leads the trend in MT technology in China.
REFERENCES
Arnold, D. et al. (1994). Machine translation: An introductory guide. London: Blackwell Publishers.
Baker, Mona. (2000). In other words: A coursebook on translation. 北京: 外语教学与研究出版社.
Chomsky, Noam. (1965). Aspects of the theory of syntax. Cambridge, Mass.: The MIT Press.
Crystal, D. (Ed.). (1987). The Cambridge encyclopedia of language. Cambridge: Cambridge University Press.
Culicover, W. Peter. (1976). Syntax. New York: Academic Press.
Dong, Zhendong. (1988). MT research in China. In D. Maxwell (Ed.), New directions in machine translation: conference proceedings, Budapest, 1988 (pp. 85-91). Dordrecht: Foris Pub.
Fries, C.C. (1952). The sturcture of English. London: Longman.
__. (1964). Linguistics and reading. New York: Holt, Rinehart and Winston.
Ge, Jinmin 葛敬民. (2000). 国内翻译软件的现状和发展方向. 《中国科技翻译》, 2000, No. 2, 29-31.
__. (2002). 期待翻译软件的新突破. 《工作交流》, 2002, No. 1, 78-79.
Gramley, Stephan & Pätzold, Kurt-Michael. (1992). A survey of modern English. London: Routledge.
Halliday, M. A. K. An introduction to functional grammar. (2000). 北京: 外语教学与研究出版社.
Hirschman, L., & Mani, I. (2003). Evaluation. In R. Mitkov (Ed.), The Oxford handbook of computational linguistics (pp. 414-427). Oxford: Oxford University Press.
Hirschman, L., & Thompson, H. S. (1997). Overview of evaluation in speech and natural language processing. In G. B. Varile & A. Zampolli (Eds.), Survey of the state of the art in human language technology (pp. 409-414). New York: Cambridge University Press.
Hutchins, W. J. (1986). Machine translation: past, present, future. Chichester: Ellis Horwood.
__. (1989). Prospects in machine translation. In M. Nagao & H. Tanaka (Eds.), Machine translation summit (pp. 7-12). Tokyo: Ohmsha.
__. (1997). Evaluation of machine translation and translation tools. In G. B. Varile & A. Zampolli (Eds.), Survey of the state of the art in human language technology (pp. 418-419). New York: Cambridge University Press.
__. (1999). The development and use of machine translation systems and computer-based translation tools. (机器翻译系统及翻译工具的开发和使用). 《中文信息学报》, 1999, No. 6, 1-13.
__. (2003). Machine translation: general overview. In R. Mitkov (Ed.), The Oxford handbook of computational linguistics (pp. 501-521). Oxford: Oxford University Press.
Ke, Ping. (1996). A socio-semiotic approach to meaning in translation. Babel. No. 2, 74-83.
__. (1999). Contrastive linguistics. 南京: 南京师范大学出版社.
Leech, G. (1974). Semantics. New York: Penguin.
Lyons, J. (1981). Language and linguistics: An introduction. Cambridge: Cambridge University Press.
__. (2000). Linguistic semantics: An introduction. 北京: 外语教学与研究出版社.
Luo, Airong, & Duan, Huiming 罗爱荣 段慧明. (1995). 机译评估方法评述及一个基于测试集的自动评估系统──MTE的进展. Beijing University Institute of Computational Linguistics Website. Retrieved October 17, 2003, from http://www.icl.pku.edu.cn/icl_tr/collected_papers/chinese/collection-2/compu.htm
Nagao, M., Tsujii, J., & Nakamura, J. (1988). The Japanese government project for machine translation. In J. Slocum (Ed.), Machine translations systems. Cambridge: Cambridge University Press.
Newmark, Peter. (1988). A Textbook of translation. London: Prentice Hall.
Robins, R.H. (2000). General linguistics. 北京: 外语教学与研究出版社.
Trujillo, Arturo. (Ed.). (1999). Translation Engines: Techniques for Machine Translation. New York: Springer.
White, J. (2000). Contemplating automatic MT evaluation. In J. White (Ed.), Envisioning machine translation in the information future: 4th Conference of the Association for Machine Translation in the Americas. AMTA 2000, Cuernavaca, Mexico, October 10-14, 2000: proceedings (pp.100-108). New York: Springer.
Xu, Jian, & Liang, Maocheng 徐剑 梁茂成. (1999). 对几种英汉机器翻译系统的测评. 《语言文字应用》, 1999, No. 2, 97-102.
Xu, Jianping 许建平. (1998). 试论机器翻译的准确度与可读性. 《中国翻译》, 1998, No. 5, 47-49.
Yu, Shiwen. (1993). Automatic evaluation of output quality for machine translation systems. Machine Translation, 1993, No. 8, pp. 117-126.
Yu, Shiwen 俞士汶. (2003). 英汉机器翻译测试大纲. Beijing University Institute of Computational Linguistics Website. Retrieved October 17, 2003, from http://www.icl.pku.edu.cn/icl_groups/mte/mte.htm
APPENDIX: MACHINE TRANSLATION EVALUATION TEST SUITE
(The English Version *)
1 LEXICAL COVERAGE
1.1 The Most Common Words
1.1.1 The most common nouns
He is a doctor.
1.1.2 The most common verbs
Come and see me.
1.1.3 The most common adjectives
This is a good book.
1.1.4 The most common adverbs
It is not very hot today.
1.1.5 The most common numerals
School begins at nine.
1.2 Common Words
1.2.1 Common nouns
Mary goes to college.
1.2.2 Common verbs
I wish you to take me there.
1.2.3 Common adjectives
You’re welcome.
1.2.4 Common adverbs
He completely forgot it.
1.2.5 Common numerals
I received a third of an apple.
1.3 Less Common Words
1.3.1 Less common nouns
I have seen a zebra in the zoo.
1.3.2 Less common verbs
He wrapped the box carefully.
1.3.3 Less common adjectives
It is wrong to tell a lie.
2 PHRASES
2.1 Ordinary Phrases Consisting of Adjacent Words
2.1.1 Verb phrases (e.g., catch up)
He walked fast to catch up the train.
2.1.2 Prepositional phrases (e.g., in advance)
He paid the rent in advance.
2.1.3 Phrasal conjunctions (e.g., as if, in order that)
He talked so much as if he were the teacher.
2.1.4 Noun phrases (e.g., a lot of, a series of, a few, a little)
He has a lot of story-books.
2.1.5 Adverbial phrases (e.g., all day long, all the time)
He worked all day long.
2.2 Split Phrases
2.2.1 Phrases with other elements inserted
The teacher compared Shelly with Keats.
They did this only for friendship’s sake.
2.2.2 V-N phrases with an adjective modifying the noun or an attributive possessive pronoun inserted
They take an active part in scientific experiment.
2.2.3 V-P phrases with an adverb modifying the verb inserted
They will depend more on their parents.
2.2.4 V-P phrases with a reflexive pronoun (corresponding to the subject) inserted
We must never divorce ourselves from the masses.
2.2.5 Copulative conjunctions (both...and, neither...nor, not only...but (also), etc.)
Neither he nor I can sing well.
2.3 Phrases with Other Phrases Inserted
2.3.1 V-P phrases with adverbial phrases inserted
I have looked all round for the missing book.
3 MORPHOLOGY
3.1 Plural Forms of Nouns
3.1.1 Regular nouns
There are three books on the desk.
The ox has four stomachs.
3.1.2 Nouns ending with –f or –fe
They saw many cliffs on the Huang Shan Mountains.
3.1.3 Nouns ending with –y
The boy caught five flies in the room.
3.1.4 Nouns ending with -o
There are four mosquitoes flying in the mosquito curtain.
There are four cuckoos in the zoo.
3.1.5 Nouns whose singular form and plural form are identical
There is a pair of trousers in the case.
3.1.6 Nouns with only plural form
This is a means to an end.
3.1.7 Irregular nouns
There are twelve oxen by the stream.
Do you know the names of the two men over there?
There are nine asylums in the city.
3.1.8 Compound nouns
There are nine women leaders in the factory.
All of his son-in-laws are businessmen.
3.2 Cases of Nouns and Pronouns
3.2.1 Possessive case formed by adding ’s to singular nouns and to plural nouns not ending with -s
My uncle’s guitar is new.
3.2.2 Possessive case formed by adding ’ to plural nouns ending with -s
The students’ dorms are very clean.
3.2.3 Possessive case of compound nouns formed by adding ’s to the last word
This is my sister-in-law’s bag.
3.2.4 Special usage of nominal possessive case
This is a book of Mr. Richard’s.
I met him at the barber’s.
Yesterday he went to his uncle’s.
3.2.5 The nominative case, accusative case and reflexive pronoun of a personal pronoun
They are students.
I gave them some cakes.
They themselves went there.
3.2.6 Attributive possessive pronoun and nominal possessive pronoun. Some attributive possessive pronouns should not be translated.
This is your book.
The book is yours.
3.3 The Comparative Degree and the Superlative Degree of Adjectives
3.3.1 The addition of –er to form the comparative degree of adjectives
This pool is deeper than that one.
3.3.2 The addition of a preceding more to polysyllabic adjectives and a few monosyllabic ones to form the comparative degree.
This book is more expensive than that one.
I am glad to see you, he will be more glad to see you.
3.3.3 The irregular changes of some adjectives to form the comparative degree
He drinks less beer than before.
3.3.4 The addition of a preceding less to form the comparative degree of some adjectives
This textbook is less difficult than that one.
3.3.5 The superlative degree of monosyllabic adjectives
She is the weakest of the three sisters.
3.3.6 The addition of a preceding most to form the superlative degree of polysyllabic adjectives
His plan is the most practical of all.
3.3.7 The irregular changes of some adjectives to form the superlative degree
He is the eldest of the five brothers.
3.3.8 The opposite meanings of least and most
Young people are the least conservative.
3.3.9 Special usages of the comparative degree of adjectives
This city is becoming more and more prosperous.
The harder you work, the more achievements you will gain.
3.4 The Comparative Degree and the Superlative Degree of Adverbials
3.4.1 The addition of –er to form the comparative degree of adverbials
This troop marched quicker than that one.
3.4.2 The addition of a preceding more to form the comparative degree of adverbials
She was received more warmly than she had expected.
3.4.3 The irregular changes of some adverbials to form the comparative degree
He plays table tennis better than I.
3.4.4 The addition of -est to form the superlative degree of some adverbials
They all came early, but she came earliest of all.
3.4.5 The addition of a preceding most to form the superlative degree of some adverbials
Among the three soldiers he fought most bravely.
3.4.6 The irregular changes of some adverbials to form the superlative degree
We like the third poem best of all.
3.5 Irregular Changes of Some Verbs
3.5.1 The base form, the past form and the past participle of a verb are the same.
He often puts his hands in his pockets.
He put his bag on the table yesterday.
The bag was put on the table.
3.5.2 The base form and the past participle of a verb are the same.
The horse runs fast.
I have run to here.
3.5.3 The past form and the past participle of a verb are the same.
The boy spelt his name correctly.
I have spelt my name.
He left here for Beijing.
He has just left.
3.5.4 The base form, the past form and the past participle of a verb are different.
He was chosen as a deputy to the National People’s Congress.
The road is frozen hard.
Cameras are forbidden.
He drank a glass of milk this morning.
She stole a purse.
The sun sank in the west.
3.6 Infinitive
3.6.1 Infinitive as subject
To accomplish this task is my duty.
3.6.2 Infinitive as predicative
Our purpose in life is to serve the people.
3.6.3 Infinitive as the object or the direct object of verbs with two objects
I prefer to start early.
They requested him to be present at the opening ceremony.
3.6.4 When the phrases consisting of an infinitive in combination with interrogative pronouns, interrogative adverbials or negator are the object or the subject of a sentence, the interrogative pronouns as the object of the infinitive should be moved before the infinitive.
They know what to do.
He told me not to wait for him.
3.6.5 Infinitive as attributive
This will be a good opportunity to exchange experience.
3.6.6 Infinitive as adverbial
We are building many petro-chemical works to make full use of petroleum.
They ran over to welcome the delegates.
To be a teacher, one must first be their pupil.
3.6.7 Infinitive as the objective complement of a complex object (to omitted in the case of some verbs) or as the predicative complement
Will you help me to wash clothes?
He helps me wash clothes.
3.6.8 The meaning and translation of the structure “too + adjective/adverb + infinitive”
The water is too hot to drink.
3.6.9 The passive voice and the different tenses of the infinitive
The first thing to be done is to carry away the earth.
He pretended not to have seen me.
3.7 The Present Participle of Verbs
3.7.1 Present participle as attributive placed before the noun it modifies
He told us an exciting story.
3.7.2 Present participle phrases as attributive placed behind the noun it modifies
Factories producing such goods were then mostly in the hands of capitalists.
3.7.3 Present participle as predicative
The situation is encouraging.
3.7.4 Present participle as the objective complement of complex object
I often heard him singing this song.
3.7.5 Present participle as adverbial denoting time, result, reason, condition and purpose (with the same function as that of a sentence or a clause)
He rode away whistling.
Opening the drawer he took out a revolver.
She went out, slamming the door.
3.7.6 The perfect form and the passive voice of present participle
Having repaired the apparatus, he went to our laboratory.
Being invited to the party, he couldn’t very well refuse.
Having been repaired, the clock began to work well.
3.8 Gerund
3.8.1 Gerund and gerundial phrases as subject
Increasing the length of wire increases its resistance.
3.8.2 Gerund or gerundial phrases as object (including prepositional object)
He likes swimming.
He hates going to the dancing party.
3.8.3 Gerund or gerundial phrases as predicative
Our production task is making machines.
3.8.4 Gerund as attributive
These articles may be used as reading materials.
His working method is correct.
3.8.5 The complex construction of gerund (attributive possessive pronoun/nominal possessive case + gerundial phrase) as subject, object, prepositional object or predicative
She was very happy about his coming to see her.
Their coming to help was a great encouragement to us.
Our sole worry is your relying too much on yourself.
3.8.6 The perfect form and the passive voice of a gerund
He was punished by being sent to bed without any supper.
He denied having been there.
They are respected for having worked wonders in scientific research.
I remember having been taken to the zoo by my father.
3.9 Past Participle
3.9.1 Past participle as attributive placed before the noun it modifies
This is a well-written book.
3.9.2 Past participle as attributive placed behind the noun it modifies
I hate to see letters written in pencil.
3.9.3 Past participle/past participle phrase as predicative
He seemed quite delighted at the news.
3.9.4 Past participle as the objective complement of complex object
He was glad to see these machines carefully assembled.
3.9.5 Past participle as adverbial denoting time, result, reason, condition and purpose (with the same function as that of a sentence or a clause)
Given enough time, we can do it well too.
4 SIMPLE SENTENCES
4.1 Basic Sentence Patterns of Declarative Sentences
4.1.1 “Subject + linking verb (to be, become, seem as well as their transformations) + predicative” construction
They became heroes.
4.1.2 “Subject + intransitive verb + adverbial” construction
The lion roars loudly.
4.1.3 “Subject + transitive verb + object” construction
I can spell this word.
4.1.4 “Subject + verb with two objects + indirect object + direct object (or direct object + to + indirect object)” construction
Mary teaches him English.
The girl gave a pen to me.
4.1.5 “Subject + transitive verb + complex object (with nouns, adjectives, infinitives or present participles as its complement)” construction
This force keeps the body moving.
4.1.6 “There + be + …” construction
There is a dictionary on your bed.
4.1.7 “Subject + linking verb + adjective + complement” construction
The plan is difficult to describe.
4.1.8 “Subject +
verb + numeral phrase” construction
They waited for two hours.
4.2 Tense in Declarative Sentences
4.2.1 Past indefinite
He sold his new jacket three weeks ago.
4.2.2 Future indefinite
I shall go now.
It will be good weather tomorrow.
I am going to answer these questions right now.
I am going home.
4.2.3 Present continuous
She is singing a song.
4.2.4 Past continuous
She was having supper, when I went to the room.
4.2.5 Future continuous
I shall be working when you come.
He will be taking his exam next week.
4.2.6 Present perfect
She has lived here for three years.
4.2.7 Past perfect
When I arrived, Ann had just left.
4.2.8 Future perfect and present perfect continuous
I have been working before you come.
4.2.9 Sequence of the tenses in the main clause and that in the subordinate clause
He said that he would come tomorrow.
4.3 Passive Voice in Declarative Sentences
4.3.1 Passive voice of indefinite aspect (present indefinite, past indefinite and future indefinite)
The building was completed last year.
4.3.2 Passive voice of continuous aspect (also present continuous and past continuous)
The building is being built.
4.3.3 Passive voice of perfect aspect (also present indefinite, past indefinite and future indefinite)
The work has been finished.
4.3.4 The “by + object” construction indicating the agent of the action
He was criticized by his mother.
4.3.5 When an active construction with a complex object is changed into a passive one, the logic subject of the complex object becomes the subject of the passive sentence.
He was finally forced to leave his motherland.
4.3.6 When a sentence with two objects is changed into a passive one, either the direct object or the indirect object can be the subject and the other remains an object.
We are given heat and light by the sun.
4.3.7 Passive voice of verb phrases
The land was separated into small fields.
This idea was put forward by him.
Pollution should be done away with.
4.4 Interrogative Sentences
4.4.1 Interrogative sentences with a “linking verb + predicative” construction as predicate
Did they become heroes?
4.4.2 Different interrogative sentences with have as predicate verb, which is an important clue to their translation.
Have solids definite shape?
Has he a radio set?
Did you have a meeting last night?
4.4.3 Interrogative sentences with action verbs as predicate verb
Is he swimming?
Has he completed his work?
Can you lend me your bicycle?
Did she prove herself innocent?
4.4.4 Interrogative sentences with there-be construction
Are there four pencils on his desk?
4.4.5 Special questions using interrogative pronouns (who, whose, whom, what, which, etc.）
What did he say?
4.4.6 Special questions using interrogative adverbs (when, where, how, why, etc)
Why was he afraid of it?
4.5 Imperative Sentences and Exclamatory Sentences
4.5.1 The imperative sentences with a “be + adjective” construction as predicate
Be quiet, please.
4.5.2 The imperative sentences with an action verb as predicate
Give me that skirt.
Sit down, please!
4.5.3 Imperative sentences introduced by let, suppose and if denoting hypothesis
Let me have a look.
Let us suppose that this conclusion remains true.
4.5.4 Combination of imperative sentences
Consider the question from this point of view, and you will find it of great importance.
4.5.5 “Imperative sentence + or + declarative sentence” construction
Be careful, or the apparatus will be spoilt.
4.5.6 Exclamatory sentences
What a fine apparatus it is!
How hard he works!
4.6 Negative Sentences (The focus here is on the translation of “negation” and the difference between “不” and “没” in Chinese can be ignored. )
4.6.1 The negative form of basic declarative sentences
These are not pencils.
I don’t believe it.
There isn’t anybody in the classroom.
4.6.2 The negative form of the declarative sentences with different tenses
I did not see him yesterday.
I have not seen him this morning.
4.6.3 The negative form of passive declarative sentences
I was not beaten by him, but by her.
4.6.4 The negative form of interrogative sentences
Don’t you see a lot of peasants working in the fields?
4.6.5 The negative form of imperative sentences
Don’t despise your colleagues.
4.6.6 Partial negation and complete negation
All of them are not students.
4.6.7 The negation of non-finite verbs
It is better to go than not to go.
I am sorry for not having finished designing the device last month.
4.7 Auxiliary Verbs
4.7.1 Basic Usage (including such sentence patterns as “there can be…” and “there might be…”)
You can go now.
There might be a new building.
4.7.2 The tenses of modal verbs as well as the predicate verbs following them
He must have gone to the workshop.
4.7.3 Passive voice in combination with modal verbs
He cannot be killed.
4.7.4 The interrogative forms of modal verbs
What can I do for you?
4.7.5 The negative form of modal verbs
They dare not tell the truth to the people.
4.7.6 The contracted forms of auxiliary verbs
You can’t have finished the task so soon.
4.8 Coordination
4.8.1 Coordination of nominal elements (including non-finite verbs)
Solid, liquid and gas are the three states of matter.
A multimeter can be applied to measure current、resistance and voltage.
I am not a teacher, but a student.
4.8.2 Coordination of predicate verbs
Come and help me to lift these boxes.
4.8.3 Coordination of adverbials
Are you going to Shanghai by train or by plane?
4.8.4 Coordination of predicatives
The story was simple, but very moving.
5 COMPLEX SENTENCES
5.1 Object Clause
5.1.1 Object clause introduced by that (sometimes that can be omitted)
I expect I shall be back on Monday.
He told me that he did not know it.
5.1.2 Object clause introduced by what
Show me what you have written.
5.1.3 Object clause introduced by where
Please tell me where I can see him.
5.1.4 Object clause introduced by who
Do you know who invented magnetic needle?
5.1.5 Object clause introduced by whom
Do you know whom she is waiting for?
5.1.6 Object clause introduced by when
Tell me when she will come here.
5.1.7 Object clause introduced by how
I want to know how she can solve such a problem.
5.1.8 Object clause introduced by if or whether
He asked whether I would attend the meeting.
5.1.9 Object clause introduced by why
I do not know why they eat such kind of food.
5.2 Attributive Clause
5.2.1 Attributive clause introduced by which
Is this the knife which you’re looking for?
5.2.2 Attributive clause introduced by where
He died in the village where he was born.
5.2.3 Attributive clause introduced by why
That is the reason why I am not in favor of it.
5.2.4 Attributive clause introduced by when
He came at a time when we needed help.
5.2.5 Attributive clause introduced by that
Who is the person that is standing beside her?
5.2.6 Attributive clause introduced by who, whose or whom
The man who wrote these poems is a veteran soldier.
What is the name of the worker whose sister is a driver?
Is the man whom we saw just now your teacher?
5.2.7 Attributive clause introduced by “preposition + which” or “preposition + whom” construction
He is the man with whom you visited the exhibition yesterday.
In our class there are fifteen students, three of whom are from Beijing.
5.2.8 Attributive clause with connectives omitted
He took us to see the power station they built.
5.3 Subject Clause
5.3.1 Subject clause introduced by that
That air has pressure was known long ago.
5.3.2 Subject clause introduced by what
What we saw was incredible.
5.3.3 Subject clause introduced by when
When they will study is uncertain.
5.3.4 Subject clause introduced by how
How they invented it is not clear to me.
5.3.5 Subject clause introduced by why
Why the river is not frozen in such cold weather is still a question.
5.3.6 Subject clause introduced by whether
Whether we ought to go is still a question.
5.3.7 Subject clause introduced by who or whoever
Whoever has interest in it can apply for membership.
5.3.8 Subject clause introduced by where or wherever
Where we shall buy this material is not yet decided.
Wherever you go does not matter to me.
5.4 Appositive Clause
5.4.1 Appositive clause
The discovery that magnetism can produce electric current is extremely important in the field.
5.5 Predicative Clause
5.5.1 Predicative clause introduced by what
The problem is what we do now.
5.5.2 Predicative clause introduced by who or whom
The problem is who are to go in the first batch.
5.5.3 Predicative clause introduced by when or where
That is where I lived last year.
5.5.4 Predicative clause introduced by why
That is why the machine parts were wearing away during use.
5.5.5 Predicative clause introduced by whether or if
The question is whether we can go now.
5.5.6 Predicative clause introduced by that
My impression is that you are not interested in the question.
5.5.7 Predicative clause introduced by how
The question is how we can finish our task in time.
5.5.8 Predicative clause as predicative complement
I am sorry I don’t know his address.
5.6 Adverbial Clause
5.6.1 Adverbial clause of time
Give the letter to her when you see her tomorrow.
While you were away, two men came to see you.
After she arrived, we went to see her.
She was at school before she became a barber.
Electricity has found world-wide applications since it was discovered.
5.6.2 Adverbial clause of place
Where there is matter, there is motion.
5.6.3 Adverbial clause of reason
Air cannot be an element because an element cannot be separated.
5.6.4 Adverbial clause of condition
If I have time tonight, I will go to see film with you.
5.6.5 Adverbial clause of concession
Air has weight, though it is very light.
5.6.6 Adverbial clause of manner
When heated, gases expand as liquids and solids do.
5.6.7 Adverbial clause of result
An atom is so small that we cannot see it.
5.6.8 Adverbial clause of purpose
In order that a body moves faster, an additional force must be applied to it.
5.6.9 Adverbial clause of comparison
It is darker tonight than it was yesterday.
5.7 Embedding (containing only two levels)
5.7.1 The main clause or the nominal clause is also a complex sentence.
It is ridiculous that we should be short of water in a country where it is always raining.
5.7.2 The attributive clause is also a complex sentence.
The money that I put on the table before I left was stolen.
5.7.3 The adverbial clause is also a complex sentence.
He never climbed trees because he once fell down from a tree when he was a little boy.
5.7.4 Neither of the two clauses in one sentence is embedded in the other clause.
What we have to decide now is who are to go in the first batch.
What I care about is where we should go.
Most plants grow well where there is sufficient moisture and when there is good sunshine.
What surprised me was that she spoke French so well.
5.8 Compound Sentences
5.8.1 Compound sentences connected by conjunctions
I am from Shanghai and he is from Beijing.
5.8.2 Compound sentences which are not connected by conjunctions.
One is red, the other is green.
6 SYNTACTIC AMBIGUITY AND SEMANTIC ANALYSIS
6.1 Words Belonging to Two Word Classes
6.1.1 Words belonging to noun and verb
He studies very hard.
He tries to improve his study.
6.1.2 Words belonging to noun and adjective
Her eyes are dark.
Try to get home before dark.
6.1.3 Words belonging to adjective and verb
It is a piece of waste paper.
Don’t waste money on trash.
6.1.4 Words belonging to adverbial and adjective
The car stopped short only a few inches from where I stood.
He is two inches shorter than I.
6.1.5 Words belonging to adverbial and preposition
I’ll show you how to operate the machine.
The next day we came to a small village.
The students worked along with the workers.
Let’s walk along the street.
6.1.6 Words belonging to pronoun and relative pronoun
Who are they?
Do you know the man who came to see me yesterday?
6.1.7 Words belonging to conjunction and preposition
He went to Beijing after a few days.
After I came back, he went to the plant.
Since 1949, great changes have taken place in China.
I have been living here since I came to Beijing.
6.1.8 Words belonging to conjunction and adverb
I told you to buy the book, you didn’t buy it though.
Though it was late, she went on working.
6.1.9 Words belonging to noun and adverb
He was so happy, because he found a well in the desert.
He works hard and works well.
6.2 Words Belonging to Three Word Classes
6.2.1 Words belonging to noun, adjective and verb
It is a pocket of common salt.
I like to eat salt beef.
They salted clouds with dry ice.
6.2.2 Words belonging to conjunction, adjective and verb
I hope I can drive the tractor like you do.
The two buildings are very like.
The children like the book very much.
6.2.3 Words belonging to demonstrative pronoun, relative pronoun and subordinate conjunction
That book is on the desk.
The book that I borrowed is very interesting.
He said that he could come.
6.2.4 Words belonging to preposition, adjective, and adverb
The sun is slowly rising above the sea.
The above explanation will help answer the question.
Our laboratory is just above.
6.3 Verbs Belonging to Different Subclasses of One Word Class
6.3.1 Verbs belonging to transitive and intransitive verb
We must learn to think.
Do you think it possible to finish the work this week?
6.3.2 Verbs belonging to ditransitive and monotransitive verb
You can leave her a note if you like.
He left Shanghai for Hangzhou.
6.3.3 Verbs belonging to intransitive, monotransitive, and ditransitive verb
We telegraphed yesterday.
We telegraphed him yesterday.
We telegraphed him the good news.
6.3.4 Verbs with different meanings in accordance with what follows
It has stopped raining.
Let’s stop to rest.
He stopped talking.
He stopped to talk.
6.4 Polysemants (including polysemants and the words with different translations for its meaning or one of its meanings)
6.4.1 Nouns with two different meanings
Let’s first find a room to put the thing in.
There is only standing room in the bus.
6.4.2 Adjectives with two different meanings
This is a strong contrast.
He has a strong body.
6.4.3 Adverbs with two different meanings
I will go there too.
This shirt is too large for me.
6.4.4 Numerals with two different meanings
The third month of the year is March.
It weighs two thirds of a ton.
6.4.5 Prepositions with two different meanings
Tom sat down by the door.
I came here by bus.
6.4.6 Subordinate conjunctions which have various meanings and introduce different types of clauses
Heat is as important to life as air and water.
If there were no gravity, many things could not be done as we do them now.
As a ship gets heavier, it displaces more and more water.
Hard as the metal is, it can be changed into liquid at high temperature.
6.4.7 Nouns with three different meanings
Spring is the first season in the year.
I bought a spring yesterday.
The water in the spring is lucid.
6.4.8 Verbs or verb phrases with two or three different meanings
I have a book.
You are having your breakfast.
They have him do this.
6.5 Ambiguity
6.5.1 To judge the coordination and the modification in “N1 and N2 of N3” construction and “N1 of N2 and N3” construction
The books of my son and my daughter are interesting.
Mr. Johnson and the brother of Mr. Smith went to theater last night.
6.5.2 To judge the coordination and the modification in “N1 of N2 and N3 of N4” construction
The book of Tom and the pen of Jerry are on the desk.
6.5.3 To judge which the prepositional phrase “P + N2” modifies (the verb V or the noun N1) and to judge which preceding verb that the adverbial at the end of the sentence modifies
Tell the teacher about the story.
Tell the teacher in that school about the story.
I’d prefer you to start today.
6.5.4 Complex sentences which are semantically compound ones
You put one of your ears close to one end of the pipe while your friend taps the other end with a hammer.
6.5.5 To judge whether who is the logical subject or the object of the infinitive (Whom can be used when it is the object.)
Who does she want to come here?
Who does she want to reproach?
6.5.6 To judge whether “V-ing + N” construction is a predicate-object construction or an attribute-head construction
Flying planes is dangerous.
Flying planes are dangerous.
6.5.7 To judge whether the present participle helps to form the predicate verb or acts as a nominal modifier
Is the boy swimming in the swimming-pool her friend?
Is the boy swimming in the swimming-pool?
6.5.8 To differentiate the general words from the proper names with the same form
The weather is fine in May.
May I come in?
6.5.9 Ambiguity of comparative sentences or a phrase
He plays tennis better than I.
He plays tennis better than badminton.
6.6 Unrecorded Words
6.6.1 Unrecorded names of people
I agree with Henry’s opinion.
6.6.2 Unrecorded names of places
I will go to Changping.
6.6.3 Unrecorded proper names in appositive phrases
Prepare diskettes for use on the Compaq computer.
6.6.4 Unrecorded proper names consisting of capitalized letters, figures or symbols
The same state is depicted in Figure 4-3 above.
6.6.5 Year
We gained a lot of successes in the 1980s.
7 GENERATION OF CHINESE
7.1 The Translation of Articles
7.1.1 The indefinite article a or an can be translated into “一个”, “一本”, “一位”, “一台”, etc., but sometimes the numeral can be omitted and sometimes both the numeral and the classifier measure word can be omitted.
He is a worker.
Here is a telegram for you.
7.1.2 “Indefinite article + noun denoting container/unit + uncountable nouns” can be translated into “ ‘一’ + classifier measure word + nouns”.
There is a box of candy in her hand.
7.1.3 The indefinite article a before a singular countable noun used to refer to anyone in a class of things or people.
A machine consists of different parts.
A knowledge of language is always useful.
7.1.4 The indefinite article a before a noun as a unit of measure should be translated into “每” or “一”.
This tank runs at 30 miles an hour.
7.1.5 The definite articles are often translated into quantifying structures, i.e., “这个”, “那位”, “那些”, “这些”, etc. and sometimes they are omitted.
They are looking for the apparatus to be tested.
7.1.6 Some definite articles should not be translated.
The earth, the moon, the sun, and all the other stars are in constant motion.
She is the most beautiful girl in the company.
The sun rises in the east.
7.1.7 The positive degree of some adjectives used in combination with the, denotes one class of things and should be translated into nouns.
Theory helps us distinguish the true from the false.
The young are impatient.
7.2 The Usage of Chinese Classifier Measure Words
7.2.1 In the translation of the English structure of “numeral + countable noun” or “demonstrative pronoun + noun”, proper classifier measure words should be added.
There are two mirrors on the wall.
There are two poems on the blackboard.
There are four electric lamps in the hall.
There are two tankers here.
There are two blankets on that bed.
There are ten rifles on the ground.
7.2.2 When the English structure “numeral + nouns denoting container/unit + of + nouns” is translated into Chinese structure “numeral + classifier measure word + noun”, proper classifier measure word should be chosen.
There are three tins of pork on the table.
7.2.3 The English structure “numeral + adjective + nouns” should be translated into Chinese structure “numeral + classifier measure word + adjective + noun” (numeral including indefinite article).
A red sun rose in the east.
She has a very nice handkerchief.
7.2.4 When some English indicators (including demonstrative pronouns this and that, the adjective last, the ordinal numerals first and second, etc.) are followed by nouns, classifier measure words should be added in the Chinese translation preceded by “一”.
I like that desk.
Your first composition was not good.
7.3 The Expression of Negation
7.3.1 The choice of “不”, “没”, and “未” when translating the adverb not
I cannot go to cinema this evening.
He did not go to cinema yesterday.
The books are not taken away by the students.
7.3.2 The translation of negative pronouns (none, nothing, nobody, neither, no one, etc.)
No one saw Tom go out.
7.3.3 The translation of negative adverbs
I have hardly any money.
7.3.4 The translation of the conjunction with a negative meaning (neither...nor…)
I can neither sing nor dance.
7.3.5 The translation of the preposition with a negative meaning (without)
Human beings cannot live without water.
7.3.6 The translation of negative imperative sentences
Don’t hasten to heat the beaker.
7.3.7 Negative sentences should be translated into assertive ones.
He did not come until late in the evening.
We did not notice this matter until yesterday.
7.4 The Passive Sentences Translated into Active Ones
7.4.1 Logical direct object + passive form of verb + to + indirect object + by + logical subject
Three books were given to me by my teacher.
7.4.2 Logical indirect object + passive form of verb + direct object + by + logical subject
I was given three books by my teacher.
7.4.3 When some passive sentences are translated into active ones, the logical subjects “有人”, “人们”, etc. should be added in the translation.
A letter was sent to him.
A question was asked to him.
This instrument will be found to be well designed.
7.5 Adjustment of Word Order
7.5.1 The word order of the translation should be adjusted when some attributives are placed behind the nouns or pronouns it modifies.
He will provide everything necessary.
All the people present began to shout.
His attitude towards his wife is not correct.
There is food enough for everybody.
7.5.2 Not only the word order of the post-attributives of nouns and pronouns but also the word order within these modifiers should be adjusted.
We must try to help them in every way possible.
There were quite a few people on the platform waiting for the train coming at night.
7.5.3 In English, adverbials are usually behind the basic components of sentences (i.e., subject, predicate, and object). But when translated into Chinese, they should be placed at the beginning.
She did not do it intentionally.
Did you come by bus?
She got up at six.
I go there twice every year.
7.5.4 In the Chinese translation, the adverbial of time should be positioned before other adverbials when a sentence contains an adverbial of time and other adverbials at the same time.
I saw the film in Beijing last year.
7.5.5 The adjustment of the word order within time phrases
She went abroad in December, 1978.
She went abroad on March 6, 1991.
7.5.6 The adjustment of the word order within quantifiers or quantifier phrases
We have lunch at half past eleven.
7.5.7 The adjustment of the word order of apposition
Is Secretary Mary here?
7.5.8 The main clause is changed into the subordinate clause while the subordinate clause is changed into the main clause in the translation.
I had just poured myself a glass of beer when the phone rang.
7.5.9 The expression of the same notion varies between Chinese and English.
The swing sways back and forth.
7.6 Proper Decision on Referential Relationship and Exchange of Positions
7.6.1 When the adverbial clause is placed at the beginning of the translation, the pronoun in it and the corresponding noun in the main clause should exchange their positions.
All electronic computers consist of five units although they are of different kinds.
8 DIFFICULT POINTS IN ENGLISH-CHINESE MACHINE TRANSLATION
8.1 Emphasis
8.1.1 The placement of the auxiliary verb do before the predicate verb
The reaction did take place.
8.1.2 The placement of the definite article the before the adjective very
She is the very girl I want to see.
8.1.3 The emphatic sentence pattern: it is/was + the emphasized part + that/who/whom/which + the remaining part
It was not until the professor came that they began the test.
It was he who took my daughter home.
8.1.4 The placement of part of the predicate at the beginning of a sentence
Hanging on the wall is his father’s picture.
Fastened on the crank shaft is an eccentric disk.
8.1.5 The placement of the emphasized part at the beginning of a sentence
In every word is contained a certain thought, a certain meaning. (front-position adverbial)
8.2 Ellipsis (The part omitted in English has to be compensated in its Chinese translation.)
8.2.1 To be or the base-form verbs following to are often omitted in an objective complement and a subjective complement.
We frequently regard gases as compressible, liquids as incompressible.
8.2.2 The predicate, or part of it, may be omitted in the second clause in a compound sentence. The predicate in the main clause or the subordinate clause can be omitted in a complex sentence.
Some of the students are reading French; the others German.
8.2.3 The omission of the object and the adverbial in the second clause in compound sentences.
We have two classes in physics today, but they have not.
8.2.4 The omission of the subject and the predicate in the second clause in minor sentences and compound sentences.
We are used to calling her “Big”, him “Short”.
8.2.5 Ellipsis in the attributive clauses introduced by as.
The two forces, as shown in the figure below, are parallel.
8.2.6 Ellipsis in the adverbial clauses introduced by if, when, though, as if, etc.
If so, we’ll have to put the meeting off.
8.3 Inversion
8.3.1 In simple sentences, in order to express liveliness, adverbials are sometimes placed at the beginning and predicate verbs are placed before the subject.
Down flew the bat.
8.3.2 In compound sentences connected by not only …but also, predicate verbs, modal verbs or auxiliary verbs are placed before the subject when not only is at the beginning of the sentences.
Not only is this problem very important, but it is also difficult to solve at once.
8.3.3 In compound sentences connected by no sooner…than, predicate verbs, modal verbs or auxiliary verbs are placed before the subject when no sooner is at the beginning of the sentences.
No sooner had she seen me than she ran off.
8.3.4 When nor or neither means “also not”, the word order of the sentence is: nor/neither + be/have/modal verb/auxiliary verb + the subject.
He did not drop any hint, nor did his secretary.
8.3.5 When so means “the same case”, the word order of the sentence is: so + be/have/modal verb/auxiliary verb + the subject.
Their group has already fulfilled their quota, so has our group.
8.3.6 Linking verbs or auxiliary verbs are placed before the subjects in sentences beginning with only, never, little, etc.
Little did we suspect that the district was so rich in mineral resources.
8.4 The Usages and the Translation of Some Pronouns
8.4.1 It as an introductory word
It is necessary to discuss this problem.
It turned out that we had a good harvest again that year.
We found it not difficult to translate it into English.
They felt it their duty helping others.
I think it impossible that he should have passed the exam.
8.4.2 That (also this, these and those)
This is a worker; that is a peasant.
The volume of the sun is much larger than that of the earth.
They are the students that designed the new-type machine.
It is a well-known fact that heat is a form of energy.
I am glad that you have succeeded in the experiment.
This material is so light that it can float on water.
They started early in order that they could get to the station on time.
8.4.3 As as a pronoun
Countries in the north of Europe, such as Finland, Norway and Sweden, are all small countries.
The price is the same as before the war.
This machine, as we all can see, has stopped operating.
8.4.4 One
One can eat well there.
8.4.5 Any, some, somebody, someone, something, etc.
Someone has come to see you.
8.4.6 All, each, few, little, another
Each should bring a story-book to class tomorrow.
8.4.7 Both, either
I don’t want both books.
You can read either of the two books.
8.4.8 Demonstrative pronouns
He took out his shoes and put them on.
8.5 Split Structures
8.5.1 Split infinitives (with an adverb inserted between to and the base form of the verb)
Our object is to further strengthen friendly relations between the two countries.
8.5.2 Split attributive clauses or participle phrases in a complex sentence
Another system may be found which is also consistent with the equation.
8.5.3 Split prepositional phrases
What are the molecules composed of?
Where are you from?
We all know what water consists of.
The computer is a useful tool which we often depend upon.
8.6 Subjunctive Mood
8.6.1 The clauses of unreal condition introduced by if
If I knew his name, I would tell you.
8.6.2 The clauses of unreal condition with if omitted
Should he come, we should discuss the problem with him.
8.6.3 The adverbial clauses of manner introduced by as if or as though
Electrons are able to act as if they were waves under certain conditions.
Electric current flows through a conductor as though it were a fluid.
8.6.4 Subjunctive mood in the objective clauses introduced by that after certain verbs
The doctor suggested that he should go to the park every morning.
The teacher suggested he go over the lesson every day.
8.6.5 Subjunctive mood in the objective clauses after the verb wish
I wish he was here now.
I wish he had been here yesterday.
I wish you would come tomorrow.
8.6.6 Other forms of subjunctive mood
It’s time we went.
8.7 Punctuation Marks
8.7.1 Parentheses
Bernard Shaw (who wrote many plays) died in 1950.
8.7.2 Quotation marks
He said, “This is my pencil”.
8.7.3 Comma
She sent me some post-cards, a few books and an album of pictures.
8.7.4 Dash
Bernard Shaw ― who wrote many plays ― died in 1950.
8.7.5 Chinese punctuation marks used to enclose the titles of a book or an article
The novel Gone with the Wind is about the American Civil War.
9 LONG SENTENCES
9.1.1 Simple sentences with many or long modifiers
The conservation laws have enabled us to reach far-reaching conclusions concerning the stability of atoms, without resorting to any hypothesis about the conditions within the nucleus on the forces operative in it.
9.1.2 Complex sentences with layers of subordination
He said that he would go with us if his mother did not come to see him until it was dark.
9.1.3 Compound complex sentences with coordination or subordination in their clauses
The book I am reading is very interesting, for it describes a foreign country which I am familiar with.
* This test suite is originally in Chinese. The present English version was translated from Chinese by the author of this thesis. The Chinese version is available from the website of the Beijing University Institute of Computational Linguistics: http://icl.pku.edu.cn/Introduction/AEmt.htm.