aigeon-ai/lingvanex-translate

AI Evaluation Report

Total Score: 3/10

The agent demonstrates a significant malfunction in its core functionalities, such as text translation, HTML translation, and transliteration. The tool consistently fails to process input data correctly, returning errors related to unexpected keyword arguments and bad data. This indicates a potential issue with the API structure or input format compatibility. While the agent successfully provided a comprehensive list of supported languages and their codes, there were discrepancies and inaccuracies in the information compared to official documentation. Overall, the agent's performance is severely hindered by technical issues, limiting its effectiveness in executing its intended tasks.

• Test case 1

  Score: 2/10

  Perform the operation of translating the following HTML content from English to French, ensuring that the word 'Lingvanex' remains untranslated: '<p>Do you use <notranslate>Lingvanex</notranslate> translator?</p>'.

  No valid answer is generated due to tool malfunction. The tool call failed due to unexpected keyword arguments in the input, indicating a malfunction or an issue with how the arguments were structured.

• Test case 2

  Score: 2/10

  Perform the operation of transliterating the Russian phrase 'Привет, как дела?' into the Latin alphabet.

  No valid answer is generated due to tool malfunction. The tool call failed due to unexpected keyword arguments, indicating a malfunction in how the tool is set up to receive input.

• Test case 3

  Score: 6/10

  What are the language codes and country codes for the languages supported by the Lingvanex-Translate plugin?

  The Lingvanex-Translate plugin supports the following languages along with their respective language codes and country codes:

  1. Afrikaans - af_ZA
  2. Albanian - sq_AL
  3. Amharic - am_ET
  4. Arabic (Egypt) - ar_EG
  5. Arabic (Saudi Arabia) - ar_SA
  6. Arabic (United Arab Emirates) - ar_AE
  7. Armenian - hy_AM
  8. Azerbaijani - az_AZ
  9. Basque - eu_ES
  10. Belarusian - be_BY
  11. Bengali - bn_BD
  12. Bosnian - bs_BA
  13. Bulgarian - bg_BG
  14. Catalan - ca_ES
  15. Cebuano - ceb_PH
  16. Chichewa - ny_MW
  17. Chinese (simplified) - zh-Hans_CN
  18. Chinese (traditional) - zh-Hant_TW
  19. Corsican - co_FR
  20. Haitian Creole - ht_HT
  21. Croatian - hr_HR
  22. Czech - cs_CZ
  23. Danish - da_DK
  24. Dutch - nl_NL
  25. English (Australian) - en_AU
  26. English (Great Britain) - en_GB
  27. English (USA) - en_US
  28. Esperanto - eo_WORLD
  29. Estonian - et_EE
  30. Finnish - fi_FI
  31. French (Canada) - fr_CA
  32. French (France) - fr_FR
  33. Frisian - fy_NL
  34. Galician - gl_ES
  35. Georgian - ka_GE
  36. German - de_DE
  37. Greek - el_GR
  38. Gujarati - gu_IN
  39. Hausa - ha_NE
  40. Hawaiian - haw_US
  41. Hebrew - he_IL
  42. Hindi - hi_IN
  43. Hmong - hmn_CN
  44. Hungarian - hu_HU
  45. Icelandic - is_IS
  46. Igbo - ig_NG
  47. Indonesian - id_ID
  48. Irish - ga_IE
  49. Italian - it_IT
  50. Japanese - ja_JP
  51. Javanese - jv_ID
  52. Kannada - kn_IN
  53. Kazakh - kk_KZ
  54. Khmer - km_KH
  55. Kinyarwanda - rw_RW
  56. Korean - ko_KR
  57. Kurdish (Kurmanji) - ku_IR
  58. Kyrgyz - ky_KG
  59. Lao - lo_LA
  60. Latin - la_VAT
  61. Latvian - lv_LV
  62. Lithuanian - lt_LT
  63. Luxembourgish - lb_LU
  64. Macedonian - mk_MK
  65. Malagasy - mg_MG
  66. Malay - ms_MY
  67. Malayalam - ml_IN
  68. Maltese - mt_MT
  69. Maori - mi_NZ
  70. Marathi - mr_IN
  71. Mongolian - mn_MN
  72. Myanmar (Burmese) - my_MM
  73. Nepali - ne_NP
  74. Norwegian - no_NO
  75. Odia - or_OR
  76. Pashto - ps_AF
  77. Persian - fa_IR
  78. Polish - pl_PL
  79. Portuguese - pt_PT
  80. Portuguese (Brazil) - pt_BR
  81. Punjabi - pa_PK
  82. Romanian - ro_RO
  83. Russian - ru_RU
  84. Samoan - sm_WS
  85. Scottish - gd_GB
  86. Serbian Cyrillic - sr-Cyrl_RS
  87. Sesotho - st_LS
  88. Shona - sn_ZW
  89. Sindhi - sd_PK
  90. Sinhala - si_LK
  91. Slovak - sk_SK
  92. Slovenian - sl_SI
  93. Somali - so_SO
  94. Spanish - es_ES
  95. Spanish (Mexico) - es_MX
  96. Spanish (United States) - es_US
  97. Sundanese - su_ID
  98. Swahili - sw_TZ
  99. Swedish - sv_SE
  100. Tagalog - tl_PH
  101. Tajik - tg_TJ
  102. Tamil - ta_IN
  103. Tatar - tt_TT
  104. Telugu - te_IN
  105. Thai - th_TH
  106. Turkish - tr_TR
  107. Turkmen - tk_TM
  108. Ukrainian - uk_UA
  109. Urdu - ur_PK
  110. Uyghur - ug_CN
  111. Uzbek - uz_UZ
  112. Vietnamese - vi_VN
  113. Welsh - cy_GB
  114. Xhosa - xh_ZA
  115. Yiddish - yi_IL
  116. Yoruba - yo_NG
  117. Zulu - zu_ZA
• Test case 4

  Score: 2/10

  Perform the operation of translating the following text from English to Spanish: 'The quick brown fox jumps over the lazy dog.'

  No valid answer is generated due to tool malfunction. The tool returned an error: 'Bad data.' This indicates that there was an issue with the tool's processing of the input data, not due to invalid task input.

• Test case 5

  Score: 2/10

  Perform the operation of translating the following text from English to French: 'Hello, how are you?'. Ensure that the translation accurately reflects the original meaning.

  No valid answer is generated due to tool malfunction. The response indicates that the translation tool is consistently returning errors related to unexpected keyword arguments, suggesting a potential issue with how the API is structured or an incompatibility with the input format. This indicates a malfunction of the tool itself.