(Translation
Studies / Bilingual Lexicography)
Department of
English, School of Foreign Studies,
南京大学外国语学院英语系
|
姓名 |
吴志杰 |
学号 |
MG0209051 |
||||||||||||||||||||||||||||||
|
年级 |
二年级 |
导师姓名 |
张柏然、刘华文 |
||||||||||||||||||||||||||||||
|
1. Specific topic of research
Chinese Word Segmentation in Machine Translation: A Cognitive Approach |
|||||||||||||||||||||||||||||||||
|
2. Significance of the topic (the rationale for your proposed research)
Word-segmentation systems available are either linguistically-oriented or statistically-oriented. Both types, however, have some innate defects that cannot be overcome due to the pragmatic-oriented feature of the Chinese language. As machine translation programs are concerned, they do not produce very satisfactory results in terms of Chinese word segmentation. A cognitive approach is designed in the light of how human beings carry out word-segmentation process in their reading, aiming at addressing the word-segmentation problem in machine translation from a different perspective. |
|||||||||||||||||||||||||||||||||
|
3. Existing research on this topic (literature review)
I. Status Quo of Chinese Word Segmentation Systems Chinese word segmentation is one of the most important components of Natural Language Processing for Chinese language, and is often referred to as the bottleneck in Chinese language understanding and processing. a. Linguistically-based word-segmentation systems Although the significance of Chinese word segmentation has long been recognized, the pioneering research in this field did not begin until the 1980s. Since the first Chinese word segmentation system, i.e. CDWS, was developed in 1983 by the Computer Department of Beijing University of Aeronautics and Astronautics, multiple studies have been conducted and quite a few models have been established (Wang et al 2003). First came the mechanical matching method for Chinese word segmentation, which was employed in the CDWS system. By this method, we first establish a large lexicon that contains all the possible words in Chinese, and then apply certain rules to divide the input sentence into small linguistic blocks, which are to be compared with the items in the lexicon. If all these linguistic blocks match lexons from the lexicon, then word segmentation is accomplished. Otherwise, an alternative division of the sentence is carried out and the above process is repeated. The process, however, might be applied several times before an acceptable result can be obtained. Based on this method, there exist some subcategories, such as Maximum Matching and Minimum Matching, and Obverse Matching and Reverse Matching, with the first two divided by the criterion of priority to long words or short words, and the latter pair, by the direction of processing. Among them, the Maximum Reverse Matching Method has been the one most widely used. This method, however, is notorious for its poor treatment of ambiguity processing of word segmentation. In order to improve its performance, feature lexicon, binding matrix, and grammar analysis have recently been incorporated into this method. Feature lexicon, by extracting functional words (e.g., “了”), words with affixes (e.g., “老虎”), and words formed by doubling the same character (e.g., “明明白白”), is employed as a kind of pre-processing before applying the mechanical matching process. Binding matrix is used to check results of the mechanical matching process by a grammar matrix and a semantic matrix on the level of phrases. Grammar analysis also applies grammar rules for the purpose of word-segmentation, but it is performed synchronously with the mechanical matching both on the level of phrase and sentence, therefore it is different from the binding grammar matrix. (Liu 2000; Wang et al. 2003; Luo et al. 1997; Yin 1998.) These, although different from each other, all belong to the formal rule-based methods. The subsequent systems based on them do not produce very satisfactory results. b. Statistically-based word-segmentation systems There is another underdeveloped
approach for Chinese word segmentation, that is, a statistically-based
word-segmentation processing system. It usually employs word frequency and
character cooccurrence probability to decide word boundaries, the only example
of which is the system designed by
The difference between these two approaches can be described as “deductive” vs. “inductive”. The fundamental difference between these is the source of knowledge that eventually determines the behavior of the system. Deductive systems rely on linguists and language engineers, who create or modify sets of rules in accordance with their knowledge, expertise, and intuition. While inductive systems depend on examples, which usually take the form of a corpus. The rules of inductive systems are often derived by the system itself from the examples. Neither of these has so far given a satisfactory response to the increasing need for Chinese word segmentation. The problem is that both approaches, in addition to their obvious advantages, have a number of serious drawbacks.
II. Shortcomings discussed Why cant these systems segment Chinese words very well? According to the information concerning the designs for these translating systems, most of programs adopt either a linguistically-based or a statistically-based word-segmentation processing system. However, neither segmentation approach can achieve a very satisfactory result. The rule-based linguistically-oriented translation systems do not produce very satisfactory results due to the fact that most Chinese words can serve more than one part of speech and have couples of, or even dozens of, meanings, with a single character capable of forming words with many different other characters, before or after. As to a statistically-based word-segmentation processing system, it cannot solve the problem either. A statistically-based word-segmentation processing system can only make sure a certain percentage of word segmentations are all right while leaving the remaining words poorly processed and making ridiculous segmentation mistakes. This approach improves the performance of unusual word segmentation, but does a very poor job concerning common words, components of which are very flexible in forming words with other characters, and in most cases multiple in the meaning (Liu 2000; Wang et al. 2003). As some scholars have argued, European languages are mainly syntactic-oriented while Chinese is basically pragmatic-oriented. In other words, pragmatic contexts and co-texts play an important role in understanding Chinese texts, hence they are significant in Chinese word-segmentation. All the above mentioned systems, however, fail to take pragmatic contextual and co-textual information into consideration, which largely accounts for their poor word-segmentation performance. Therefore, how to incorporate into word-segmentation systems pragmatic contextual and co-textual information becomes very relevant to the problem here. |
|||||||||||||||||||||||||||||||||
|
4. Specific, real questions existing research does not answer (or does not answer adequately) but you attempt to answer. Also, state if these are definition, basic data, descriptive, or causes-and-effects questions and what you have done to avoid “reinventing the wheel” in your proposed study.
The question my study intends to address while existing research does not answer is how we can incorporate into word-segmentation systems pragmatic contextual and co-textual information so that the word segmentation problem will be more satisfactorily solved and the performance of machine translation will be greatly improved. To avoid “reinventing the wheel” in my study, I first conducted a pilot study to assess all the translation programs available on the market, the results of which show that a relatively large proportion of the translation errors can be attributed to mistakes in word-segmentation. This means my research question is a genuine one and has relatively great significance. Second, I have tried my best to obtain a rather thorough literature so that I am sure that my approach has never been tried before. |
|||||||||||||||||||||||||||||||||
|
5. Your tentative answer to the questions (your hypothesis): interpretive, descriptive, explanatory, or predicative? How will you operationalize your hypothesis so that it can be tested (e.g. by measuring its key concepts)? If your hypothesis is an interpretive one, does it have testable consequences?
The tentative answer to my hypothesis is predicative in nature. That is, if we incorporate into word-segmentation systems pragmatic contextual and co-textual information, the word segmentation problem will be more satisfactorily solved and the performance of machine translation will be greatly improved. In order to operationalize my hypothesis, I will conduct one test to see how well the available machine translation programs on the market can deal with Chinese word-segmentation (My pilot study shows that a relatively large proportion of the translation errors can be attributed to mistakes in word-segmentation.). Besides, I will carry out two survey studies and several interviews to find out how Chinese people segment a Chinese sentence into words and how similar they carry out this word-segmenting process. |
|||||||||||||||||||||||||||||||||
|
6. Methods by which you will obtain and analyze your evidence: What theoretical model will you use? Do you propose to conduct conceptual research, empirical research, or applied research? If you opt for empirical research, what specific kind(s) of empirical research (naturalistic, experimental, qualitative, quantitative?) will you do and what specific empirical research method(s) (case study, corpus study, survey, historical/archival study, etc.) will you use?
The main theoretical model of my study will be a process model, which will introduce the word-segmentation process of my newly designed model step by step. However, the other two theoretical models, i.e., the comparative model and causal model, will be employed in my study as well. The comparative model will be used when comparing the results of human segmentation and those of translation programs while the causal mode will be applied to justification of the new model. My study, in the first place, is conceptual research, since the focus of study is a new model for Chinese word segmentation. However, the part that supports my new model is empirical. In this part, one test, two surveys and six interviews will be carried out to obtain my evidence. The test and surveys are mainly quantitative studies whereas the interviews aim at obtaining qualitative data. |
|||||||||||||||||||||||||||||||||
|
7. Evidence you will use or expect to find. Are they textual or contextual? Why do you think your evidence supports your hypothesis?
Evidence that I will use is mainly contextual variables. To be more specific, I will use source-text variables, especially word-segmentation of source texts. Chinese word segmentation is one of the most important components of Natural Language Processing for Chinese language, and is often referred to as the bottleneck for Chinese language understanding and processing. It is obvious that how source texts are segmented will affect the quality of machine translation. My new model of Chinese word-segmentation takes pragmatic contextual and co-textual information into consideration, which, hopefully, will contribute to the better performance of machine translation systems. |
|||||||||||||||||||||||||||||||||
|
8. Preliminary findings (Give one example, e.g. variables you’ve identified and patterns or regularities you’ve found in your pilot study.)
In my pilot study, I discovered that computer translation programs available on the market, which are either linguistically-oriented or statistically-oriented, are unable to tackle Chinese segmentation very well in Chinese-English translating. It shows that a relatively large proportion of the translation errors can be attributed to mistakes in word-segmentation. And in my survey studies and interviews, I found that people carry out word-segmentation process quite similarly and in most cases they make use of the pragmatic contextual and co-textual information, rather than linguistic principles. |
|||||||||||||||||||||||||||||||||
|
9. Major references
Lamb, Sydney.
(1999). Pathways of the brain:
Neurocoginitive basis of language. Liu, Kaiying 刘开瑛. (2000).《中文文本自动分词和标注》. 北京: 商务印书馆. Luo, Zhengqing et al. 骆正清等. (1997). 《汉语自动分词研究综述》.《浙江大学学报》[Journal of Zhejiang University]. 第31卷第3期. Taylor, John.
(2002). Cognitive grammar. Wang, Ke et al. 王科 等. (2003). 《汉语分词的主要技术及其应用展望》 [Main techniques in Chinese
word-segmentation and their prospective applications].《通信技术》[Communications
Technology]. 2003年第6期. Wu, Andi.
(2003). Chinese word segmentation in MSR-NLP. The Second SIGHAN Workshop on
Chinese Language Processing, Yin, Jianping 殷建平.
(1998). 《汉语自动分词方法》[Automatic word segmentation methods for Chinese]. 《计算机工程与科学》[Computer Engineering and Science]. 第20卷第3期. |
|||||||||||||||||||||||||||||||||
|
10. The timetable for your research
|
|||||||||||||||||||||||||||||||||
|
导师签名 (1) |
|
日期 |
|
||||||||||||||||||||||||||||||
|
导师签名 (2) |
|
日期 |
|
||||||||||||||||||||||||||||||