Llm
class LlmEngine(context: Context, modelPath: String, template: ChatTemplate = ChatTemplate.GEMMA)(source)
On-device text generation via MediaPipe LLM Inference, backed by whatever .task model the user downloaded (Qwen, Gemma, …) — fully local, no network at inference time.
Two generation paths:
generate (one-shot): wraps the whole prompt in the chat template and generates. Simple, always correct.
generate (prefix/suffix): KV prefix-cache — the stable preamble is prefilled into a base session once, then each turn clones that session and only processes the variable suffix, skipping re-prefill of the (~500-token) preamble. Falls back to the one-shot path on any failure, so it can never regress correctness; flip PREFIX_CACHE off to disable entirely.