- \n
- You spend less time editing because the rewriter helps you make your sentences clear and nativelike as you type.<\/li>\n<\/ul>\n\n\n\n
- \n
- AI and machine learning are now able to create amazing content that is indistinguishable from human one. There are even articles written by robots<\/a>! The rewriter is no exception.<\/li>\n<\/ul>\n\n\n\n
Technology doesn\u2019t stand still and neither does Linguix. The updated rewriter has shown significant and measurable improvements. Let\u2019s discuss how our team has achieved these results and define various metrics that have helped us to provide a more sophisticated experience in Linguix Rewriter 2.0.<\/p>\n\n\n\n
Metrics to Determine the Quality of Linguix Rewriter<\/h1>\n\n\n\n
The Bleu Score. <\/strong>The Bilingual Evaluation Understudy score, or BLEU for short, is a metric for comparing a generated sentence to a reference sentence. This metric evaluates the quality of the machine learning translation.<\/p>\n\n\n\n
In fact, the closer the value to 0, the better. It implies that the rewriter generates \u201csmarter\u201d results, and chooses synonyms that retain initial meaning. <\/p>\n\n\n\n
The Jaccard similarity coefficient<\/strong> is a measure used in understanding the similarities between sample sets. As with the BLEU score, the appropriate Jaccard Index value tends to 0. Again, the closer to 0, the better the results.<\/p>\n\n\n\n
Language-Agnostic BERT Sentence Embedding (LaBSE) <\/strong>and Cosine similarity<\/strong>. <\/p><\/p>\n\n\n\n
The LaBSE model encodes text into high dimensional vectors so that the text vectors close in meaning are geometrically close to each other (they\u2019re placed into a shared multi-dimensional vector space).<\/p>\n\n\n\n
Cosine similarity, in turn, helps to define how similar the pieces of text are. It measures the cosine of the angle between two vectors projected in this space. The closer the cosine value to 1, the smaller the angle and the greater the match between vectors.<\/p>\n\n\n\n
Perplexity.<\/strong> Perplexity is a metric used to evaluate how good a language model is. The lower the perplexity score is, the better the language model works in terms of word prediction. <\/p>\n\n\n\n
How We Conducted the Training<\/h1>\n\n\n\n
We took 11 datasets with 573,228,310 million sentences in various styles (from technical documentation to fiction) and trained our model. The goal was to make it able to handle texts of different types and styles. The one-to-one\/one-to-many column represents whether the source sentence has one paraphrase option or several.<\/p>\n\n\n\n
<\/figure>\n\n\n\n
The Results<\/h1>\n\n\n\n
The quantitative analysis of our new model represents a higher quality of the paraphrasing generation compared to the previous model. The new model outperforms the old one in terms of text similarity: <\/p>\n\n\n\n
The BLEU score: 0.47\u2193 vs 0.65<\/p><\/p>\n\n\n\n
<\/figure>\n\n\n\n
The Jaccard similarity coefficient: 0.45\u2193 vs 0.51 <\/p>\n\n\n\n
<\/figure>\n\n\n\n
Perplexity. Rewrites generated by Linguix rewriter 2.0 appeared to be more natural and native: 0.26\u2193 vs 4.99<\/p>\n\n\n\n
<\/figure>\n\n\n\n
The semantic similarity value of the new model is slightly lower than that of the previous model (0.80\u2193 vs 0.93), which is totally fine. The model generates a variety of options using other words but keeping the meaning of the source text as its target.<\/p>\n\n\n\n
<\/figure>\n\n\n\n
As such, for Linguix rewriter 2.0 we were able to improve the quality of the rephrased content while keeping the text meaning at the same level.<\/p>\n\n\n\n
How to test the updated rewriter <\/h2>\n\n\n\n
You need to install Linguix browser extension<\/a> or use Linguix web editor<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
Linguix Rewriter has become an essential tool for most of our users for many reasons. Here are just a few of them: Technology doesn\u2019t stand still and neither does Linguix. The updated rewriter has shown significant and measurable improvements. Let\u2019s discuss how our team has achieved these results and define various metrics that have helped […]<\/p>\n","protected":false},"author":1,"featured_media":2720,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[817],"tags":[835,819,565,834,833,12],"class_list":["post-2717","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research","tag-ai-writing-assistant","tag-grammar-check","tag-research","tag-rewriter","tag-survey","tag-writing"],"acf":[],"yoast_head":"\n
How To Measure the Quality of the AI-based Rewriter: Our Experience - Linguix Blog<\/title>\n<meta name=\"description\" content=\"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How To Measure the Quality of the AI-based Rewriter: Our Experience\" \/>\n<meta property=\"og:description\" content=\"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/\" \/>\n<meta property=\"og:site_name\" content=\"Linguix Blog\" \/>\n<meta property=\"article:author\" content=\"alex\" \/>\n<meta property=\"article:published_time\" content=\"2022-04-27T11:49:35+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-24T19:56:15+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1720\" \/>\n\t<meta property=\"og:image:height\" content=\"1200\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Alex\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Alex\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/\",\"url\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/\",\"name\":\"How To Measure the Quality of the AI-based Rewriter: Our Experience - Linguix Blog\",\"isPartOf\":{\"@id\":\"https:\/\/linguix.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png\",\"datePublished\":\"2022-04-27T11:49:35+00:00\",\"dateModified\":\"2023-09-24T19:56:15+00:00\",\"author\":{\"@id\":\"https:\/\/linguix.com\/blog\/#\/schema\/person\/ea7597fd80a4c2a8f55eb54be35a2293\"},\"description\":\"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.\",\"breadcrumb\":{\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage\",\"url\":\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png\",\"contentUrl\":\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png\",\"width\":1720,\"height\":1200},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/linguix.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How To Measure the Quality of the AI-based Rewriter: Our Experience\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/linguix.com\/blog\/#website\",\"url\":\"https:\/\/linguix.com\/blog\/\",\"name\":\"Linguix Blog\",\"description\":\"Writing about using technology to create content and build effective communications.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/linguix.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/linguix.com\/blog\/#\/schema\/person\/ea7597fd80a4c2a8f55eb54be35a2293\",\"name\":\"Alex\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/linguix.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/team-lashkov2-96x96.jpg\",\"contentUrl\":\"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/team-lashkov2-96x96.jpg\",\"caption\":\"Alex\"},\"description\":\"Linguix.com co-founder\",\"sameAs\":[\"https:\/\/twitter.com\/alexlashkov\",\"alex\"],\"url\":\"https:\/\/linguix.com\/blog\/author\/alex\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How To Measure the Quality of the AI-based Rewriter: Our Experience - Linguix Blog","description":"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/","og_locale":"en_US","og_type":"article","og_title":"How To Measure the Quality of the AI-based Rewriter: Our Experience","og_description":"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.","og_url":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/","og_site_name":"Linguix Blog","article_author":"alex","article_published_time":"2022-04-27T11:49:35+00:00","article_modified_time":"2023-09-24T19:56:15+00:00","og_image":[{"width":1720,"height":1200,"url":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png","type":"image\/png"}],"author":"Alex","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Alex","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/","url":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/","name":"How To Measure the Quality of the AI-based Rewriter: Our Experience - Linguix Blog","isPartOf":{"@id":"https:\/\/linguix.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage"},"image":{"@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage"},"thumbnailUrl":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png","datePublished":"2022-04-27T11:49:35+00:00","dateModified":"2023-09-24T19:56:15+00:00","author":{"@id":"https:\/\/linguix.com\/blog\/#\/schema\/person\/ea7597fd80a4c2a8f55eb54be35a2293"},"description":"Linguix Rewriter has become an essential tool for most of our users for many reasons. The updated rewriter has shown significant and measurable improvements.","breadcrumb":{"@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#primaryimage","url":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png","contentUrl":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/rewritre.png","width":1720,"height":1200},{"@type":"BreadcrumbList","@id":"https:\/\/linguix.com\/blog\/how-to-measure-the-quality-of-the-ai-based-rewriter-our-experience\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/linguix.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How To Measure the Quality of the AI-based Rewriter: Our Experience"}]},{"@type":"WebSite","@id":"https:\/\/linguix.com\/blog\/#website","url":"https:\/\/linguix.com\/blog\/","name":"Linguix Blog","description":"Writing about using technology to create content and build effective communications.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/linguix.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/linguix.com\/blog\/#\/schema\/person\/ea7597fd80a4c2a8f55eb54be35a2293","name":"Alex","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/linguix.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/team-lashkov2-96x96.jpg","contentUrl":"https:\/\/linguix.com\/blog\/wp-content\/uploads\/2022\/04\/team-lashkov2-96x96.jpg","caption":"Alex"},"description":"Linguix.com co-founder","sameAs":["https:\/\/twitter.com\/alexlashkov","alex"],"url":"https:\/\/linguix.com\/blog\/author\/alex\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/posts\/2717","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/comments?post=2717"}],"version-history":[{"count":4,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/posts\/2717\/revisions"}],"predecessor-version":[{"id":3581,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/posts\/2717\/revisions\/3581"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/media\/2720"}],"wp:attachment":[{"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/media?parent=2717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/categories?post=2717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/linguix.com\/blog\/wp-json\/wp\/v2\/tags?post=2717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- AI and machine learning are now able to create amazing content that is indistinguishable from human one. There are even articles written by robots<\/a>! The rewriter is no exception.<\/li>\n<\/ul>\n\n\n\n