Wait-Think-Answer Control

Learning When to Think While Listening in Large Audio-Language Models

This paper learns when to externalize intermediate reasoning during streaming audio by training a controller to decide online whether to wait, emit a thinking update, or answer. It optimizes full trajectory with rewards covering correctness, timing, reasoning quality, and latency—not just final answer.

• llm
• multimodal
• streaming
• dialogue
• reasoning
• rlhf

Authors: Zhiyuan Song, Weici Zhao, Yang Xiao, Suhao Yu, Cheng Zhu, Jiatao Gu

Categories: cs.CL, cs.AI, cs.LG, cs.SD

Comment: 19 pages, 4 figures, 6 tables

Published 2026-05-26 · Updated 2026-05-26

Links

Paper & demos

• arXiv
• Project page

Code & resources

• GitHub2 stars0 watchers0 forksNOASSERTION
• Hugging Face51 downloads1 likes

Impact

• Semantic Scholar0 citations

Abstract

Recent advances in Large Audio-Language Models (LALMs) have made real-time, streaming spoken interaction increasingly practical. In this setting, reasoning quality and responsiveness are tightly coupled: delaying reasoning until the speech endpoint can improve answer quality but moves deliberation into user-visible response delay, while answering too early risks committing before decisive evidence arrives. We introduce a learnable wait-think-answer control formulation for LALMs. Motivated by the incremental nature of human conversation, the controller decides under partial audio evidence when to wait, when to externalize a compact reasoning update, and when to answer. Using Qwen2.5-Omni-7B as the base model, we construct aligned wait-think-answer traces from spoken reasoning data, train the controller with supervised fine-tuning (SFT), and then apply Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO). The reward combines answer correctness, action validity, update timing, latency synchronization, reasoning quality, and chain consistency, optimizing the complete wait-think-answer trajectory and not the final answer alone. On a six-task synthetic spoken reasoning question answering (SRQA) benchmark, the six-reward DAPO controller improves the row-weighted accuracy from 67.6% to 70.3% while reducing post-endpoint final-think length by 14% under the same Qwen deployment harness. On a 186-item human-recorded Real Audio Bench, a transfer check beyond text-to-speech (TTS)-rendered speech, the controller family remains functional: SFT achieves the strongest accuracy, while the six-reward DAPO controller is the only learned variant whose final-think length falls below the base. These results suggest that a streaming model should learn when to make intermediate reasoning explicit during the audio stream.

---

Introduction and Motivation

Large Audio-Language Models (LALMs) are increasingly capable of participating in real-time spoken interaction. Systems like Moshi, Mini-Omni, Freeze-Omni, and Qwen2.5-Omni can process streaming audio and generate responses with low latency, making it practical to ask an audio model to reason while speech is still arriving. However, this creates a fundamental tension: answer quality and responsiveness are tightly coupled. Waiting until the full utterance has been heard allows more complete reasoning but pushes deliberation into user-visible response delay. Answering early is fast but risks committing to a wrong answer before decisive evidence arrives at the end of the utterance.

This temporal control problem is analogous to human conversation. Psycholinguistic and neuroscience research demonstrates that listeners frequently begin preparing responses before the current speaker's turn has ended — short response gaps in natural dialogue require overlap between comprehension and production planning. The paper draws on this intuition to motivate a learnable wait-think-answer controller for streaming LALMs: a model that, at each point along the audio stream, explicitly decides whether to wait for more evidence, emit a compact intermediate reasoning update, or commit to a final answer.

Most prior work on audio reasoning follows a complete-audio protocol: the model receives the full utterance, generates a reasoning chain, and then produces its response. This is useful for measuring final answer quality, but it sidesteps the online decision problem. Recent streaming systems have started to address this gap — for example, Shih et al. use an inference-time question-completeness signal to trigger a single post-utterance chain of thought, with accuracy–latency tradeoffs shaped by length-biased preference optimization. The present paper goes further: instead of a single terminal chain of thought triggered by a completeness signal, the model is trained end-to-end to emit multiple grounded intermediate state updates during the audio stream, each visible to and reusable by later controller decisions.

Problem Formulation: Wait-Think-Answer Control

The paper frames streaming speech reasoning as an online control problem over an explicit text memory. Let $x_{1:T}$ denote the full spoken input, let $0 < t_1 < \cdots < t_K \leq T$ be controller decision times, and let $z_{<k}$ be the visible reasoning states already emitted. The controller operates in full-prefix mode: at decision step $k$, it observes the complete audio prefix $x_{1:t_k}$ heard so far, together with the committed text memory $z_{<k}$.

The controller observation and action are:

where $\pi_\theta$ is the controller policy with parameters $\theta$, and $\mathcal{A}_k$ is the legal action set at step $k$. The action space contains exactly three primitives:

• <wait/> — advances the audio stream without changing the visible reasoning state.
• <think>...</think> — emits a short grounded intermediate state update $z_k$, appended to the memory so all future controller calls can condition on it.
• <answer>...</answer> — commits to the final response.

Before the speech endpoint, the legal actions are wait and think. After the endpoint, the controller emits one final think and then an answer. Answer timing is therefore not a learned pre-endpoint stopping decision in these experiments — the answer action is endpoint-gated. This design keeps timing trainable and inspectable while allowing open-ended text generation inside thinking and answering spans.

The key insight is that distributing reasoning across pre-endpoint think actions amortizes the total reasoning cost: each visible update emitted during listening absorbs deliberation that would otherwise accumulate into the final post-endpoint think, reducing residual deliberation as a proxy for user-visible response delay.

Base Model and Deployment Semantics

The controller is instantiated on Qwen2.5-Omni-7B, a streaming Thinker–Talker omni-model that perceives audio, vision, and text while generating text and speech. The paper's implementation uses full-prefix replay: every controller call replays the observed audio prefix from scratch, while preserving the evidence available in the intended deployment semantics. This approximation is used because the available Qwen2.5-Omni serving path does not expose an official controller-style cache interface for the wait-think-answer loop — building an efficient cache-native implementation would require lower-level runtime and kernel work.

The authors also built an experimental stateful KV-cache controller prototype that appends audio chunks into a single persistent Qwen2.5-Omni thinker KV cache and forks temporary states for each controller decision. Mechanism checks confirm that cache lengths match attention lengths and that wait actions do not enter the persistent cache, but the prototype is not yet a score-producing batch runner. The full-prefix replay approach is used for all reported benchmarks.

The paper notes that the same information pattern would appear in a true cache-native streaming deployment: audio chunks would enter one persistent context, the KV cache would not be discarded between chunks, intermediate think text would be appended back into the visible context, and the final-think turn would see the complete cached audio plus all earlier thoughts.

Training Data Construction

The data pipeline constructs aligned wait-think-answer traces from spoken reasoning examples in three stages:

Synthetic Controller Corpus

An initial candidate corpus of 80,000 records (40,000 verifiable and 40,000 open-ended) is generated by prompting GPT-4o. Generation is stratified by verifiability, difficulty, and topic. Each record stores:

• A spoken surface form (tts_text)
• A TTS style instruction (tts_instruct)
• Lexical anchors for answer-relevant state changes (anchor_words)
• A semantic wait-think-answer trace (logical_actions)
• The final answer (final_answer)

After validation, deduplication, and human spot checks, the aligned corpus contains 75,723 audio-text records: 38,213 verifiable and 37,510 open-ended, split into 73,675 training and 2,048 validation records. SFT uses the full aligned corpus. DAPO uses the verifiable training branch (37,180 scorable training records), augmented with a small set of training-split examples from ARC-Challenge, ARC-Easy, GSM8K, PIQA, and SocialIQA (kept below 2% of the DAPO mix; held-out evaluation examples are never included).

Speech Synthesis and Alignment

Each spoken input is synthesized as a single full utterance using Qwen3-TTS. Full-utterance synthesis preserves global prosody and avoids artificial prosodic breaks at action boundaries — segment-wise synthesis would inject unnatural pauses exactly where the model is supposed to learn natural timing. CTC-style forced alignment maps transcript words to audio timestamps, and controller boundaries are snapped upward to a 0.5-second decision grid. A tick sweep at 0.5s, 1.0s, and 1.5s grids showed tied validation token accuracy (0.6108, 0.6105, 0.6102 respectively), so 0.5s was chosen for finer timing resolution. Qwen2.5-Omni calls are padded to the 2.0s minimum audio window required by the model.

Real Audio Bench

For transfer evaluation beyond TTS-rendered speech, the authors collected Real Audio Bench: five speakers recorded 200 GPT-4o-generated candidate items designed for natural spoken delivery. Human screening removed ambiguous or unanswerable prompts and corrected answer keys, leaving 186 recordings. Each item is designed so that an early plausible answer becomes wrong or incomplete once a late decisive cue arrives. The benchmark spans categories such as mid-way reversal, adversarial pause, incremental accumulation, and noisy environment, across scenario families like travel logistics, workplace admin, and food/shopping. It is treated as a compact transfer benchmark for real human delivery, not a large-scale user study.

Supervised Fine-Tuning (SFT)

The controller is first trained with supervised fine-tuning on Qwen2.5-Omni-7B using Low-Rank Adaptation (LoRA; rank 8, alpha 32, dropout 0.05) via MS-Swift. The audio encoder and aligner are frozen; only the language model is trainable. Training runs for one epoch over the aligned controller export with a cosine learning rate schedule ($1 \times 10^{-5}$), effective batch size 32, and fused AdamW with weight decay 0.1.

The supervised objective teaches: (1) the action serialization format, (2) the short semantic state style for intermediate thoughts, (3) the distinction between ordinary waits, answer-relevant pre-endpoint thoughts, final-think compression, and the final answer. Training and validation curves show decreasing loss and increasing token accuracy, stabilizing during training. The SFT controller serves as the initialization for all DAPO runs.

DAPO Policy Optimization

After SFT, the controller is optimized using a controller-specific adaptation of Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO), which belongs to the Group Relative Policy Optimization (GRPO) family. DAPO requires live streaming rollouts, controller-action parsing, local reward computation, dynamic resampling, and adapter updates inside the controller runtime — this motivates a custom trainer rather than the generic MS-Swift GRPO path.

Group Relative Advantages

Each prompt is sampled into a group of $G$ rollouts scored with the trajectory reward. Group-relative advantages are computed as:

Token-Level Clipped Objective

Token-level credit assignment is applied because the policy mixes actions with different failure modes — bad waits, useful short thoughts, malformed final-think turns, and correct answers should not all receive identical pressure. With $r_{i,t}(\theta) = \exp(\log \pi_\theta(y_{i,t} \mid o_{i,t}) - \log \pi_\text{old}(y_{i,t} \mid o_{i,t}))$ denoting the per-token probability ratio:

where $M$ is the completion-token mask. The asymmetric clipping uses $\epsilon_l = 0.20$ and $\epsilon_h = 0.28$. KL regularization is applied with coefficient 0.01. The actor learning rate is $4 \times 10^{-7}$ with 50 warmup steps over 1,000 total steps. Maximum rollout context is 8,192 tokens, with 48-token caps on think and answer spans.

Before a DAPO update, sampled groups must contain enough format-valid rollouts, at least one valid final-think rollout, and at least one valid pre-endpoint thought. Groups that collapse to all-wait or malformed final-think behavior are resampled up to a fixed retry budget; persistent failures are skipped and logged.

Reward Design

The reward function is central to the paper's contribution. The authors argue that optimizing only the final answer encourages a wait-all policy that shifts deliberation into user-visible response delay, while optimizing only latency can produce premature answers, empty thoughts, or malformed action traces. The reward therefore scores the full wait-think-answer trajectory.

Six reward terms are combined, four rule-based and two judge-assisted:

The shaped reward for a valid trajectory is:

The final reward applies a protocol gate:

Reward weights are $\lambda_a = 1.0$, $\lambda_f = 1.0$, $\lambda_s = 1.0$, $\lambda_u = 3.0$, $\lambda_t = 1.0$, $\lambda_c = 0.45$. The protocol gate gives format validity priority over all other objectives: if the model answers before the utterance is complete, omits the required final-think/answer structure, or emits an ill-formed trace, answer correctness cannot rescue the rollout. The consistency bonus is gated by $R_a > 0$, so a fluent but wrong chain does not receive extra credit for being self-consistent.

Key implementation details for specific reward terms:

• $R_a$: Uses difficulty-aware effort calibration so a correct answer is not rewarded equally when the controller uses an implausibly empty or unnecessarily verbose reasoning state.
• $R_s$: Applies a six-token free budget then a linear penalty with coefficient 0.30, capped at 3.0. A small bonus applies for compact answer cues of 3–6 tokens; an answer-shape guardrail penalizes question-form answers, label-only answers when a semantic answer is expected, and yes/no type mismatches.
• $R_u$: Pre-endpoint thoughts are matched to answer-relevant evidence-update ticks with a tolerance of two 0.5s ticks; sparsity pressure penalizes both missing important updates and inserting thoughts at nearly every tick.
• $R_t$, $R_c$: Scored by a local Qwen3.6-35B-A3B judge, with outputs mapped to coarse scores in $\{0, 0.5, 1\}$.

The update-timing weight $\lambda_u = 3.0$ and latency weight $\lambda_s = 1.0$ had the largest qualitative effect on the accuracy–residual-latency operating point during validation sweeps, because they directly shape when and how often the controller emits visible reasoning state.

Evaluation Benchmarks and Protocols

Synthetic SRQA Benchmark

The paper adopts the Spoken Reasoning Question Answering (SRQA) evaluation construction introduced by Shih et al. Text reasoning problems are rewritten into spoken questions, rendered with TTS, and evaluated as audio inputs. The benchmark contains 8,959 items across six tasks:

A local Qwen3.6-35B-A3B judge is used for short-answer scoring, together with task-specific normalization for numeric and multiple-choice answers.

Evaluation Protocols

Two protocols are reported:

• Offline mode: Complete-audio setting. The model receives the full spoken question once, emits one final <think> state, then emits the answer. This is the standard upper-bound reference for models without a controller interface.
• Deployment mode: Streaming protocol. Audio arrives on a fixed 0.5s grid, each controller call sees the full audio prefix plus previous visible thoughts, and the policy chooses <wait/> or <think>...</think> before the endpoint. At the endpoint, the controller emits one final <think> and then an <answer>. Deployment mode is only available for Qwen-family rows.

The main latency metric is the length of the post-endpoint final-think state (in tokens), used as the controller-level measure of residual deliberation after the user stops speaking. Cross-family token-latency comparisons are treated as descriptive only.

Baselines

Within the Qwen family, the paper compares the base Qwen2.5-Omni-7B (in both offline and deployment modes), the SFT controller, and DAPO controllers trained with four, five, or six reward terms. External baselines evaluated in offline mode only include Audio Flamingo 3, Audio Flamingo 3 + AF-Think, and GLM-4-Voice-9B. Moshi rows from Shih et al. are included for context.

Results on Synthetic SRQA

The six-reward DAPO controller is the strongest accuracy row within the Qwen streaming-controller family and is second only to the complete-audio Qwen row in the full table. It improves the row-weighted synthetic average from 67.6% (base controller) and 66.1% (SFT controller) to 70.3%, with gains on ARC-C, ARC-E, SIQA, PIQA, and GSM8K. Crucially, it also reduces mean post-endpoint final-think length from 10.44 tokens (base) to 8.99 tokens — a 14% reduction — demonstrating that the controller has learned to move deliberation earlier in the stream.

The SFT controller provides the supervised initialization for DAPO and should be read as protocol learning under token-level supervision, not direct task-reward optimization. Its accuracy (66.1%) is slightly below the base controller, while its final-think length (9.82) is marginally shorter. The DAPO stage supplies the task-level reward that moves the controller back toward task-aligned behavior and beyond the base.

The ablation across reward stacks reveals a clear progression: adding reward terms progressively improves accuracy, but the key jump in both accuracy and latency reduction comes when the sixth reward ($R_c$, chain consistency) is added alongside the fifth ($R_t$, thought quality).

Results on Real Audio Bench

On Real Audio Bench, the controller family transfers beyond TTS-rendered speech, but the best accuracy and shortest final-think points come from different learned variants. The SFT controller achieves the strongest accuracy (+4.8 points over base), while the six-reward DAPO controller is the only learned variant whose final-think length falls below the base controller (6.33 vs. 6.52 tokens). The five-reward DAPO variant has the strongest real-audio accuracy among DAPO variants (67.7%).

The 95% bootstrap confidence intervals (computed over 10,000 resamples of the 186 items) overlap substantially across all controller lanes, so Real Audio Bench is treated as a compact transfer check rather than a fine-grained controller ranking benchmark:

Runtime Analysis

Because full-prefix replay incurs repeated prefix prefill cost, the paper reports replay-harness real-time factor (RTF) — total controller wall-clock time divided by source-audio duration — as an implementation diagnostic separate from the final-think token metric:

The six-reward DAPO controller's replay-harness RTF is below the SFT controller and nearly matches the base controller on average (1.134 vs. 1.137). This suggests that the DAPO controller's shorter final-think states also translate into modestly faster overall runtime in the replay harness, reinforcing the use of final-think length as a residual reasoning metric.

Controller Behavior Analysis

Policy optimization changes the trace-level operating point of the streaming system in interpretable ways. The SFT controller sits at 66.1% synthetic accuracy with 9.82 final-think tokens — protocol learning under supervised signal, not yet task-reward-optimized. The four-reward DAPO variant recovers and exceeds the base accuracy (68.5%) but at a longer final-think length (10.87 tokens), suggesting that the basic accuracy–latency–timing reward stack does not yet sufficiently incentivize moving deliberation earlier into the stream. Adding thought quality ($R_t$, five rewards) further improves accuracy (69.2%) with only marginal change in final-think length. Adding chain consistency ($R_c$, six rewards) simultaneously achieves the best accuracy (70.3%) and the shortest final-think length (8.99), suggesting that asking the controller to maintain a consistent and correct-supporting thought chain across the stream encourages more useful intermediate reasoning and less post-endpoint rework.

On Real Audio Bench, the divergence between the SFT controller (best accuracy) and the six-reward DAPO controller (shortest final-think) exposes different operating points along the accuracy–residual-latency frontier. The DAPO variants optimize primarily for the synthetic SRQA task distribution; the SFT controller, trained on a broader corpus including open-ended items, generalizes more uniformly to the real-audio domain. This suggests that the choice of training variant should be made with the target deployment accuracy–latency trade-off in mind.

Comparison with Related Work

The paper situates itself relative to three prior lines of work:

Complete-audio audio reasoning

Audio-CoT, Audio-Reasoner, Audio-Thinker, SARI, AudSemThinker, and related systems improve reasoning in LALMs through chain-of-thought prompting, structured data, curriculum learning, or reward-based post-training — but all operate on the full audio input. Audio-Thinker is the closest prior work in the LALM space, deciding whether to produce a pre-answer rationale and training for chain consistency, but it acts only after hearing the complete utterance. The wait-think-answer formulation differs by requiring the controller to act repeatedly under partial audio evidence.

Think-while-listening systems

Shih et al. use an inference-time question-completeness signal (entropy decrease over streaming ASR distribution) to trigger a single post-utterance chain of thought, with accuracy–latency shaped by length-biased DPO. STITCH, SHANKS, and StreamingThinker study simultaneous reasoning over speech or text streams. The key distinction from the present work is that these systems either trigger a single terminal chain-of-thought or use inference-time heuristics, whereas the wait-think-answer formulation trains the controller end-to-end with a multi-term trajectory reward to emit multiple grounded intermediate updates during listening that accumulate in visible memory.

Streaming omni models

Moshi, Mini-Omni, Freeze-Omni, and Qwen2.5-Omni target real-time speech-to-speech dialogue. The present work uses Qwen2.5-Omni as a substrate but studies a different problem: how the model should update an explicit reasoning state while streaming audio is still arriving.

Stated Contributions and Novelty

The paper claims the following contributions:

1. A training formulation for wait-think-answer control in LALMs — a learned policy over partial audio evidence, with a serialized three-action interface (<wait/>, <think>, <answer>) that is trainable, inspectable, and semantics-preserving.
2. An aligned spoken-controller data pipeline — GPT-4o-generated traces, Qwen3-TTS synthesis, CTC-based forced alignment, and conversion to chained streaming windows, enabling both SFT and DAPO training.
3. A six-term trajectory reward that scores correctness, protocol validity, response latency, update timing, thought quality, and chain consistency — optimizing the complete streaming trajectory rather than the final answer alone.
4. Empirical evidence that post-training moves the controller along the accuracy–residual-latency frontier: the six-reward DAPO controller improves row-weighted accuracy from 67.6% to 70.3% while reducing post-endpoint final-think length by 14% on synthetic SRQA, and transfers to human-recorded audio with the six-reward variant being the only learned controller whose final-think length falls below the base.
5. Real Audio Bench — a 186-item human-recorded transfer benchmark for evaluating streaming LALM controllers beyond TTS-rendered speech.

Limitations

The authors are explicit about several limitations:

• Real Audio Bench scale: Only 186 recordings from five speakers, making it a compact transfer check rather than a large-scale user study. Scaling to broader speaker diversity, accents, environments, and interaction styles is planned follow-up work.
• Full-prefix replay implementation: The current implementation replays the full audio prefix at each controller call rather than using a cache-native server. The RTF audit measures the paper implementation rather than an ideal cache-native server that would forward the audio KV cache and avoid repeated prefill. Cache-native serving remains future work.
• Endpoint-gated answer timing: Answer commitment is endpoint-gated in these experiments; learning pre-endpoint answer timing is not explored.
• Single base model: All controller experiments use Qwen2.5-Omni-7B; generalization to other model families is untested.
• No head-to-head comparison with question-completeness controllers under the same Qwen harness, which is identified as useful future work.

Broader Impacts

The paper discusses both positive and negative societal implications. On the beneficial side, low-latency spoken reasoning supports accessibility tools, real-time captioning, on-device translation, and conversational tutoring — applications where response delay disproportionately harms users who rely on these aids. On the risk side, a controller reasoning over partial speech is a step toward systems that monitor or pre-empt conversations without user awareness, lowers the cost of real-time social engineering, and can create asymmetric advantage. The paper's design choices constrain the misuse surface: the action space is restricted to a three-element set with a hard format gate, and the reward explicitly penalizes spurious thinking, aligning the controller with non-interruptive conversational norms.

Conclusion

The paper presents a principled formulation of streaming speech reasoning as wait-think-answer control for LALMs. By training a stateful controller that accumulates intermediate reasoning updates during the audio stream — rather than emitting a single post-utterance chain of thought — the system learns when to externalize reasoning and when to wait, shifting the accuracy–residual-latency operating point without changing the base architecture. The six-reward DAPO controller achieves both the best accuracy and the shortest residual deliberation among the learned controller variants on synthetic SRQA, and the six-reward variant is the only learned controller whose post-endpoint reasoning length falls below the base on human-recorded audio. These results support the central claim: a streaming audio-language model should learn when to make intermediate reasoning explicit during the audio stream, not only what to reason about after it ends.