UADetox

Rewrites toxic Ukrainian comments into polite literary Ukrainian, keeping the author's point. A fine-tuned Gemma 4 E2B, 1.84 GB, that runs on a CPU, with a small classifier in front that decides what needs rewriting.

Gemma 4 E2B · LoRA · Q4_0 GGUF · llama.cpp · Ukrainian

Try the Telegram bot
Usage
Benchmark
Weights and dataset: release pending

Examples

Outputs of the deployed model with greedy decoding, not selected. The inputs were written for this page.

Comment	Rewrite
Ти дебіл, закрий рота	Ти, добродію, помиляєшся, тож помовч-но краще.
Та йди ти нахуй зі своїми порадами	Та йди собі, чоловіче добрий, своєю дорогою зі своїми порадами.
Що за довбойоби пишуть такі новини, хоч би перевіряли щось	Що ж то за новини пишуть такі люди, добродію? Хоч би перевіряли щось, а не вірили на слово.
Знову ці клоуни в раді щось вигадали, задовбали вже	Знову ті панове в раді щось вигадали, та вже, по щирості, несила терпіти.
Пиздець, а не дороги, мер тільки бабло пиляє	Ой лишенько, а не дороги, а мер, бач, тільки гроші розкрадає!
Купив цей телефон, гівно повне, батарея здохла за тиждень	Купив я цей телефон, та й нічого з нього не вийшло: батарея, бач, за тиждень і згасла.
Дякую за новину, дуже корисно	Not rewritten. Classifier score 0.01, threshold 0.21.

The criticism stays. The mayor is still accused and the phone is still bad; profanity and insults are removed.

Benchmark

A rewrite is counted as good only if it keeps the meaning and stance, is fluent, has no toxicity left and is in the literary register. Verdicts are from an LLM judge (DeepSeek V4.1 Flash, fixed provider); intervals are 95% paired bootstrap.

300 comments from the development split.

System	Params	Good, %	Toxicity left, %	Meaning lost, %
Reference rewrites	–	91.6	0.0	21.2
Gemma 4 31B, 4-shot	31B	74.7	3.3	32.7
Gemma 4 26B-A4B, 4-shot	26B	68.7	3.7	42.3
UADetox E2B	2B eff.	57.3	4.7	42.7
Gemma 4 E2B, 4-shot	2B eff.	20.5	44.1	59.3

Differences from UADetox E2B in good rate: 31B +17.3 [+10.0, +25.0], 26B +11.3 [+3.7, +18.3], reference +34.7 [+28.3, +41.1]. On the full development split (1,966 comments) the fine-tuned model scores 59.7 and the on-device build 59.5.

On-device build

Step	Size	Good, %	Effect
LoRA merged into Gemma's Q4_0 QAT weights	3.35 GB	–4.8 [–6.9, –2.7]	Naive merge undoes the quantization-aware training
LoRA trained through the Q4_0 rounding	3.35 GB	–1.4 [–3.6, +0.7]	Within noise of full precision
Vocabulary trimmed 262k → 88k tokens	1.84 GB	+0.05	84–87% of outputs byte-identical

Effects are changes in good rate: the two merges against the same model in bf16, the trim against the untrimmed build. The trim keeps Ukrainian, punctuation and the 40k most frequent Latin tokens, closed under BPE merges, including the per-layer embedding tables.

CPU serving

llama.cpp, Q4_0, one request at a time, about 74 prompt and 70 output tokens per comment.

Machine	Generation, tok/s	Per comment, s	Sustained, comments/min
Hetzner CAX21, 4 ARM cores, 8 GB	20.4	4.2	22–31
Hetzner CX23, 2 shared x86 cores, 4 GB	6.2	12.4	~8

The classifier (multilingual-e5-small, ONNX) handles about 20 comments/s on one core. About 30% of comments in public Telegram threads are flagged, so a CAX21 keeps up with roughly 80 comments per minute.

Usage

# serve; set -t to the number of cores
llama-server -m uadetox_e2b_distill_q4_trim_q4_0.gguf \
  -c 1024 -np 1 -t 4 -fa on --no-cache-prompt \
  --chat-template-kwargs '{"enable_thinking":false}'

# the comment is the whole prompt; decode greedily
curl -s localhost:8080/completion -d '{
  "prompt": "<|turn>user\nТи дебіл, закрий рота<turn|>\n<|turn>model\n",
  "n_predict": 256, "temperature": 0, "top_k": 1,
  "stop": ["<turn|>", "<|turn>"]
}' | jq -r .content

No system prompt or instructions. This is the format the model was trained and evaluated with.

POST /v1/rewrite
{"text": ["Ти дебіл, закрий рота", "Дякую за новину"], "mode": "auto"}

{"results": [
  {"needs_rewrite": true,  "score": 0.977, "rewrite": "Ти, добродію, помиляєшся, тож помовч-но краще."},
  {"needs_rewrite": false, "score": 0.006, "rewrite": null}
 ],
 "threshold": 0.208}

mode: "auto" rewrites only what the classifier flags; "always" skips the classifier.
POST /v1/classify returns scores only. Up to 32 texts of 1,000 characters per request.
Bearer-token auth, result cache, HTTP 429 when the queue is full, Prometheus metrics at /metrics.

Direct messages to @uadetoxdevbot are always rewritten, with buttons to rate the result.
In any chat, type @uadetoxdevbot and a comment, then pick the suggestion to send the rewrite.
In a group, the bot replies to flagged messages only. Admins can use /uadetox_off@uadetoxdevbot and /uadetox_on@uadetoxdevbot.

Messages the bot processes and its rewrites are stored without names or user ids to improve the model.

Limitations

About 6 in 10 rewrites pass every check. The most common failure is lost detail (about 40% of outputs), then fluency errors (about 33%).
Quality drops on comments longer than about 40 words and on sexual or threatening content.
The register is old-fashioned courtesy («добродію», «бач»), which some readers take as irony.
The classifier reliably catches profanity and slurs, but often misses mockery and insults without swear words.
All figures are on the development split with one LLM judge, which is stricter than our human labels. A locked test-set evaluation with a second human rater is in progress.

Citation

@misc{uadetox2026,
  title  = {UADetox: Polite Literary Rewriting of Toxic Ukrainian Comments on Device},
  author = {{UADetox project}},
  year   = {2026},
  url    = {https://uadetox.pages.dev}
}