How far we are from good MT system between Arabic/English?

In this post I will try to perfom some MT testing using Google translate and MS-Windows live translator service in order to evaluate its accuracy and – to some extend – the quality of the translation. This evaluation is not automated (i.e. based on my own human translation). The reason I wrote this post is to know how far we are from good MT system that can really provide the minimum acceptable translated text.

I will divide the evaluation to main two aspects, the first one is English related issues and second one is Arabic related issues.

I used two indicators, (1) translation accuracy to measure the translation from syntactic point of view. (2) translation quality to measure if this translation convey the message (semantic).

A. From English to Arabic:

1. Space-Delimited Compound Words
   Source Language (SL):  Take off your jacket!
   Target Language (TL) – Google:   خلع سترته الخاصة بك! (he took off your jacket!)
   Target Language (TL) – MS:تقلع الغلاف الخارجي الخاص بك! (you took off your outer shell!)
   Tranlation Accuracy: Bad
   Translation Quality: Bad
2. Compound Verbs with inserted words between the verb parts:
   SL: Take your jacket off!
   TL-Google:   خذ خارج الغلاف! (take out the shell!)
   TL-MS: تقلع الغلاف الخارجي الخاص بك! (take off your outer shell!)
   Tranlation Accuracy: Bad
   Translation Quality: Bad
3. Hyphenated/Solid/Open:
   SL: I used a school-bus.
   TL-Google: كنت في حافلة المدرسة.  (I was in the school-bus)
   TL-MS:استخدام حافلة مدرسية. ( The use of the school-bus)
   Tranlation Accuracy: bad
   Translation Quality: good
4. Hyphenated/Solid/Open:
   SL: New York is a big city in US.
   TL-Google: نيويورك هي مدينة كبيرة في الولايات المتحدة. (New York is a big city in US.)
   TL-MS:نيويورك مدينة كبيرة في الولايات المتحدة.(New York- is- a big city in US.)
   Tranlation Accuracy: Very good
   Translation Quality: Very good
5. Textual Meaning:
   SL: Ali is a pure man.
   TL-Google: علي هو محض رجل.(Aly is a plain man)
   TL-MS:علي رجل محض.(Aly is a plain man)
   Tranlation Accuracy: bad
   Translation Quality: bad
6. Textual Meaning:
   SL: I studied pure mathematics.
   TL-Google: درست الرياضيات البحتة.(I studied pure mathematics.)
   TL-MS:درست الرياضيات محض.(I studied plain mathematics.)
   Tranlation Accuracy:Very Good
   Translation Quality: Very Good
7. Textual Meaning:
   SL: he is trying to contain acts.
   TL-Google:انه يحاول احتواء الأعمال.(he is trying to include acts)
   TL-MS:يحاول تحتوي على الأعمال.(trying including of acts.)
   Translated text meaning:
   Tranlation Accuracy:Bad
   Translation Quality: Bad
8. Suggestive Meaning (Culture dependent translation):
   SL: The girl is as white as snow.
   TL-Google:الفتاة كما الثلج الأبيض(the girl is like the white ice)
   TL-MS:كما بيضاء كما الثلوج الفتاة. (like white like snow the girl)
   Tranlation Accuracy:Bad
   Translation Quality: Bad
9. Suggestive Meaning (Culture dependent translation):
   SL: UNESCO is a big organization.
   TL-Google:اليونسكو منظمة كبيرة(UNESCO is a big organization)
   TL-MS:اليونسكو منظمة كبيرة.(UNESCO is a big organization)
   Tranlation Accuracy:Very good
   Translation Quality: Very good
10. Proper Arabic morphological generation:
    SL: The man said they will write it.
    TL-Google:وقال الرجل أنها سوف اكتبها.
    TL-MS: وقال الرجل سوف تكتب عليه
    The correct translation: وقال الرجل أنهم سيكتبونها.
    Tranlation Accuracy:Bad
    Translation Quality: Bad

A. From Arabic to English:

1. Suggestive Meaning (Culture dependent translation):
   SL: الفتاة كالقمر في جمالها.
   TL-Google:Moon girl beauty.
   TL-MS:Girls as Moon in aesthetics
   The correct translation: The girl is as fair as Snow White.
   Tranlation Accuracy:Bad
   Translation Quality: Bad
3. Arabic morphological translation:
   SL: وقال الرجل انهم سيكتبونها.
   TL-Google:The man said they Sictbunha.
   TL-MS:And the man they سيكتبونها
   The correct translation: The man said they will write it.
   Tranlation Accuracy:Bad
   Translation Quality: Bad

Google translate service: http://translate.google.com

MS Live translate service: http://www.microsofttranslator.com/Default.aspx

I plan to add more results in future posts, God willing.

The man said they Sictbunha

My first published paper

An English-To-Arabic Web Translation Agent paper here imam-195. This paper published in ICENCO’2004 conference.

Arabic/English Online Free Dictionaries

Did you ever wonder about the meaning of any English or Arabic word or its pronunciation. There are many useful online dictionaries that can help you out. I decided to put a list of them so any one can find them easily and use them. This is as well related to my Masters thesis, as the dictionary part is important in MT research.

1. Dicts http://dicts.info/
2. Alburaq http://www.alburaq.net/dictionary/transform.cfm (also provide Arabic roots dictionary http://www.alburaq.net/mukhtar/root.cfm)
3. tos.edu http://www.tps.edu.ee/nastik/ar-en/ (very simple)
4. ECTACO http://www.ectaco.co.uk/English-Arabic-Dictionary/ (organized output with similar words)
5. Dict.info http://www.dicts.info/2/english-arabic.php (English pronunciation available)
6. Sakhr http://dictionary.sakhr.com/ (You will notice that you need to put the word in the website search input box , the output is very good with diacritics (i.e. tashkeel) )
7. Lexilogos http://www.lexilogos.com/english/arabic_dictionary.htm (linked to many other dictionaries).
8. Albalqa http://dictionary.bau.edu.jo/Search.aspx
9. Sensagent http://dictionary.sensagent.com/ (Rich output ).
10. Babylon http://www.babylon.com/define/98/English-Arabic-Dictionary.html