Language coverage
Multilingual Website Chatbots: A Source and Language Test
Evaluate multilingual website answers with a source-parity matrix, policy-exception tests and fluent review. Separate language coverage from market availability.
By HostDesk · Updated · 6 min read
Before offering a website assistant in another language, check whether it can preserve the meaning, limits and authority of your approved information in that language. Fluent wording is only one part of the job. A confident answer based on an obsolete translation is still wrong.
Use a topic-by-language matrix, review important exceptions with a fluent person and test the route to a human. This is a product-agnostic evaluation method. It does not certify a translation, establish legal compliance or claim that HostDesk supports a particular set of languages.
Separate language, audience and business scope
Define the audience from actual enquiries and the service you can provide. An audience in Canada does not establish that every visitor wants French; an audience in the United States does not establish that every visitor wants English. Record requested languages rather than deriving them from a person's name.
Market scope is a separate question. Currency, service area, delivery options and contract terms may differ between offerings or locations. Translation must not silently carry a policy into a market where the business has not authorized it.
Write a short coverage statement for each task: which language is supported, which approved sources apply and what the visitor can do if those sources are incomplete. A narrower, well-reviewed scope is easier to evaluate than an unqualified promise to handle every topic in every language.
| Dimension | Example of a decision to record |
|---|---|
| Task | Explain the published change-request process. |
| Language | The visitor's selected language, with relevant terminology. |
| Business scope | The offering and location to which the source applies. |
| Source authority | The approved policy version, not simply the newest-looking translation. |
| Fallback | A real route when a language or topic is outside the reviewed scope. |
Build the source-parity matrix
Choose the topics that matter before a visitor takes the next step. For each language, record the source URL, business-approved version, effective date and any missing exception. Mark a topic ready only after both the factual meaning and the language have been reviewed.
Parity does not require identical page layouts or a word-for-word translation. It requires an accurate answer for the same scoped question. If policies legitimately differ by location, record that difference explicitly instead of forcing the pages to agree.
The following matrix is a fictional teaching fixture. Its versions and findings are invented to demonstrate the method, not observed customer data or results from a multilingual assistant.
| Source | Approved meaning | Review state | Required action |
|---|---|---|---|
| English example policy v3 | Requests more than 48 hours after the original booking need staff review. | Current in the fixture. | Use for this scoped policy. |
| French example page v2 | Older wording uses 24 hours after the original booking. | Conflict in the fixture. | Have the policy owner and fluent reviewer update it. |
| Spanish example page | No corresponding approved page exists. | Coverage missing. | Do not imply a reviewed Spanish policy exists. |
| Contact route | Staff review is required; no automatic approval. | Destination needs its own verification. | Check receiving language and actual route before publishing. |
Test the exception, not just the easy sentence
Prepare expected meaning before reading the assistant's answer. Include the condition, exception, next step and any statement the answer must avoid. A reviewer should be able to mark the important clauses separately.
Use a fluent reviewer who understands the business context. A machine translating an answer back into the source language may help find a question to investigate, but it is not independent confirmation that the original answer is accurate or natural.
Include spelling variation, a language switch and a follow-up with an implied reference. Keep the test task constant so you can see whether a language change altered the policy. Use approved harmless fixtures rather than real private customer records.
| Test field | What to write |
|---|---|
| Question | Visitor wording and selected language. |
| Expected meaning | The approved facts and exceptions that must survive. |
| Forbidden addition | A guarantee, entitlement or completed action the source does not support. |
| Reference context | What 'that', 'it' or 'next month' refers to in the conversation. |
| Observed reply | Actual output copied only after the test runs. |
| Reviewer verdict | Meaning correct, language revision needed, source conflict, or unsupported. |
| Evidence | Source version and reviewer/date; keep the verdict traceable. |
Worked review: a polite answer can still change the policy
Consider the fictional English policy at https://maple-workshop.example/changes, version Example-v3: 'Change requests made more than 48 hours after the original booking require staff review. Sending a request does not confirm a change.' No real business or executed test is represented.
A French-speaking visitor asks, 'Puis-je modifier ma demande?' The expected meaning depends on the request's timing and requires staff review where the policy says so. Do not assume the person is asking to change the language merely because the question is in French.
Imagine an illustrative reply whose meaning is 'Yes, your change is approved.' Even if a fluent reviewer likes the wording, the factual verdict is fail: the source authorizes a review process, not approval. A corrected answer must preserve that boundary and offer the actual request route.
Now ask a follow-up whose meaning is 'So it is confirmed?' The expected answer still cannot claim confirmation without the appropriate outcome evidence. Record the real output when you run this test; these sample verdicts describe a deliberately constructed fixture.
Review the interface as well as the answer
Language choice should remain understandable in the launcher, instructions, errors and handoff form. Check dates, numeric formats and units where a misunderstanding would change the request. A label such as '03/04' is ambiguous without a stated format.
W3C explains that programmatically identifying the page language helps browsers and assistive technology handle text appropriately. Its language-of-parts guidance addresses passages in another language. Ask the person implementing the interface to check the actual markup and assistive-technology behavior.
Those technical language markers do not validate translation quality or establish overall accessibility conformance. Keep the review specific: can the visitor understand the question, identify required fields, correct an error and use the human route in the language offered?
Publish only the scope you can maintain
Require an owner for each important translated policy. When the authoritative policy changes, reopen its language rows and retest affected exceptions. A new translation date alone does not prove that the business meaning was approved.
If a topic is missing in one language, make the limit and alternative clear. Do not quietly use a conflicting older translation to avoid refusing. The receiving team must also know the language needed for follow-up; a fluent automated greeting cannot establish staffed support.
Use a public-site preview to inspect which information is available, then perform the actual language review with the chosen tool and fluent reviewers. The language matrix is an operating worksheet you maintain, not an advertised HostDesk language-management feature.