Opt.Gear: On-Device Language Models Trained on 0.5T Tokens

🤗 HUGGING FACE 🏠 OptAI ✉️ CONTACT

Opt.Gear is an open-source model family developed by OptAI. Rather than simply shrinking server-scale language models, it is designed for on-device text generation in environments constrained by SRAM, memory bandwidth, power, and latency: smartphones, laptops, robots, vehicles, and embedded boards. The 270M and 1B Dense models provide a 65,536-token Context Window. This release includes Supervised Fine-Tuning (SFT) weights and deployment binaries for ONNX, Qualcomm NPU, and Apple ANE.

Summary of Opt.Gear

Gear-1B balances quality and efficiency, while Gear-270M focuses on lower latency and smaller mobile and edge environments. Both share the same hybrid design and 64K context, supporting Korean-English bilingual text generation, summarization, and instruction following.

The models use a hybrid attention mechanism combining local sliding-window attention, global attention, and the convolution-based ConvKV-Gated Mixer. This design provides lightweight-model processing speed and low memory usage without sacrificing the performance needed for complex long-context tasks.

ConvKV-Gated Mixer: Why Is It Fast?

The bottleneck in on-device inference is often memory access rather than computation. Auto-regressive decoding reads and writes the KV-cache for every generated token. Once this state exceeds on-chip SRAM, external DRAM traffic becomes a major source of latency and power consumption.

Sliding-window attention only bounds the cache to a window of size W: a larger W still increases the live decoding state. ConvKV-Gated Mixer changes this pattern. Instead of a token window, it maintains only a convolution buffer with a fixed kernel size :

Most local attention layers are replaced with ConvKV-Gated Mixer, while a small number of global GQA layers route long-range information. With only static linear, convolution, and element-wise operations, and no softmax or matrix multiplication, the mixer maps readily to CPUs, GPUs, and NPUs. This is central to its NPU-friendly design.

Pre-trained & SFT

Running foundation-model ablation studies and training on tens of trillions of tokens requires substantial time and resources with 8 H200 GPUs. To train efficiently within a short time, the 1B and 270M models collected approximately 2T candidate tokens from English, Korean, mathematics, and general web sources, then used filtering, deduplication, and mixture rebalancing to select 0.5T tokens . The final training mix was approximately 92% English, 6% Korean, and 2% mathematics.

The training recipe has three stages:

1. Pretraining : large-scale pretraining with a 4K context on the curated 0.5T tokens
2. Long-Context Extension : two additional training stages on the prolong-data-64k corpus, using 10B tokens each in English and Korean, progressively extending the context from 4K → 32K → 64K
3. Supervised Fine-Tuning (SFT): a two-stage design separating general instruction-following SFT from reasoning-oriented SFT for mathematical and analytical responses

The final checkpoints balance strong foundational capabilities with deeper contextual understanding and reasoning, demonstrating adaptability across downstream tasks.

Benchmark Performance

We evaluated the models on a broad collection of datasets and metrics covering text generation, using LM Evaluation Harness. The significance of Gear’s results lies in the combination of fewer training tokens, no distillation, and Korean performance rather than absolute superiority on every benchmark.

Korean benchmark	Opt.Gear-1B	Gemma-3-1B	Llama-3.2-1B	EXAONE4-1.2B	LFM2.5-1.2B
Trained Tokens	0.5T	2T	6T	12T	36T
KMMLU	36.0	30.7	29.9	32.6	29.2
KoBEST	60.3	59.5	51.8	50.7	59.1
CLIcK	39.7	37.8	30.6	32.9	38.3
HAERAE	44.2	35.3	32.6	30.2	33.9

Language Model Evaluation Harness, one shared configuration, July 2026. Full tables — including the English suite and the 270M class — are on the benchmarks page.

What stands out is data efficiency . While baselines such as Gemma3, Llama3.2, and EXAONE4 use 2T~12T tokens and knowledge distillation, Gear-270M and Gear-1B achieve comparable or better performance with just 0.5T tokens and no distillation, particularly across Korean benchmarks including KMMLU, KoBEST, CLIcK, and HAERAE.

Inference Performance

To measure text generation speed in real edge environments, we benchmarked across processors (CPU/GPU/NPU) and runtimes (Qualcomm AI Engine Direct, llama.cpp, CoreML). The configuration uses W4A16 quantization with 16-bit embedding and LM head.

Qualcomm Snapdragon NPU

On the Snapdragon 8 Elite GEN5 NPU, Opt.Gear-1B achieved prefill 7,042 tokens/s, substantially ahead of Llama3.2-1B (4,481 tok/s) and Gemma3-1B (3,226 tok/s). This demonstrates server-class prompt throughput in a mobile environment.

Apple iPhone (CoreML)

Build with Opt.Gear

Opt.Gear is available as open weights on Hugging Face and supports inference with Transformers, llama.cpp, ExecuTorch, and vLLM. Optimized execution binaries for Qualcomm Hexagon NPU and Apple ANE are also provided.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OptAI/Opt.Gear-1B"  # TODO: 실제 리포지토리 ID로 교체

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [{"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")

outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

# llama.cpp — TODO: GGUF 파일명 확정 후 교체
llama-cli -m opt-gear-1b-q4.gguf -p "안녕하세요" -n 128

Limitation and Future works

The limited 0.5T-token training budget also constrains Gear-270M and Gear-1B on complex reasoning and mathematics tasks. Beyond increasing the number of training tokens, future work will focus on improving capability gained per token through high-quality data selection, domain-aware mixture optimization, curriculum scheduling, and reasoning- and mathematics-focused data.

Summary

Opt.Gear shows that well-designed architectures and curated data can deliver meaningful on-device performance. Trained on 0.5T tokens without distillation, the 1B model outperforms comparable models trained on 2T~12T tokens across Korean benchmarks and achieves prefill speeds of up to 7,042 tok/s on Snapdragon NPUs. The next articles introduce Opt.Gear-QAT, covering quantization-aware training, and the ultra-small microcontroller model Opt.Gear-1M. We welcome community feedback and look forward to the edge applications built with Opt.Gear.

Citation

@misc{optgear2026,
    title  = {{Opt-Gear} Technical Report},
    author = {{OptAI Gear Team}},
    year   = {2026},
    url    = {https://huggingface.co/OptAI}
}

Correspondence: contact@opt-ai.kr · Hugging Face: huggingface.co/OptAI