Frontend · Backend · Artificial Intelligence

A 0.9B Model on a 5060 Ti Just Solved the Meeting-Minutes Problem

By 雪隐_上班了 · Jul 22, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Meeting audio contains sensitive business information that most teams cannot send to cloud APIs. This pipeline proves that a single consumer GPU can run a complete, private meeting-minutes system end-to-end, with speaker labels and structured output, for zero recurring cost.

Summary

A local pipeline built around the MOSS-Transcribe-Diarize 0.9B model handles transcription and speaker separation in a single inference pass, outputting timestamped, speaker-labeled text. The raw transcript is compressed into a script-like format and handed to a locally running Gemma 4 26B model, which organizes it into structured Markdown minutes with topics, discussion points, resolutions, and to-do items.

The entire stack runs on a single RTX 5060 Ti 16G with zero API costs. A lazy-loading singleton keeps the transcription model resident, while an LLM scheduler routes tasks to different local models based on context length and task type, falling back to a cloud API only if local backends fail. Subtitle generation and courseware organization reuse the same transcription backend.

A 36-minute recording exposed a critical VRAM bug: the default full-attention implementation requested 49.55 GiB. Switching to SDPA attention and adding automatic 10-minute chunking with timestamp offsets brought memory usage under control, though the serial execution of transcription and LLM organization means the total pipeline runs at roughly real-time speed.

Takeaways

— Switching from full attention to SDPA in the model loader dropped VRAM allocation from 49.55 GiB to a manageable level with a one-line change.

— Automatic 10-minute chunking with timestamp offsets acts as a second safety net when a single forward pass still exceeds GPU memory.

— Speaker IDs are independent per chunk, so stitching segments together can misattribute speakers unless the LLM is unloaded to free memory for a single-pass run.

— A lazy-loading singleton pattern avoids reloading the transcription model on every request, cutting response time after the first call.

— The LLM scheduler estimates input token length before choosing a backend, skipping local models entirely when the text exceeds their context window rather than timing out.

— Subtitle duplicate merging uses longest-common-subsequence similarity at an 0.85 threshold to clean up ASR stutter without losing content.

— The system prompt explicitly instructs the model to write 'None' for missing resolutions and to-dos, preventing hallucinated action items.

— Translation runs on a separate small model (hy-mt2-7b) in batches of five lines to prevent the model from merging consecutive sentences and breaking line counts.

Conclusions

End-to-end diarization models like MOSS invert the traditional ASR pipeline: instead of gluing Whisper and pyannote together and fighting timestamp alignment, a single 0.9B model handles both tasks, and on the Alimeeting benchmark it beat human reference annotations.

The real bottleneck is not model capability but GPU memory scheduling. MOSS and Gemma cannot coexist in 16G of VRAM, forcing a serial workflow that doubles total processing time compared to what a parallel setup could achieve.

The 49.55 GiB OOM error reveals how default attention implementations silently scale quadratically with sequence length, making long audio a hidden trap for developers who test only on short clips.

Assigning different local models to different tasks based on context length, rather than using one model for everything, is a practical optimization that keeps translation fast and reasoning high-capacity without wasting resources.

Concepts & terms

Speaker Diarization

The process of partitioning an audio stream into segments according to speaker identity, answering 'who spoke when.' Traditional pipelines run ASR and diarization as separate steps and then align their outputs.

SDPA (Scaled Dot-Product Attention)

PyTorch's memory-efficient attention implementation that reduces the quadratic O(n²) VRAM cost of standard attention to O(n), critical for long sequences like multi-minute audio transcripts.

LCS (Longest Common Subsequence)

An algorithm that finds the longest sequence of characters present in two strings in the same order. Used here to detect near-duplicate ASR output segments by measuring the ratio of shared characters.

Singleton Pattern

A software design pattern that restricts a class to a single instance, providing a global access point. Applied here to load the transcription model once and reuse it across all requests.

From the discussion

Featured comments

ivenshine

Enterprise meeting software nowadays all have corresponding AI meeting minutes.

雪隐_上班了

You're talking about large enterprises holding online meetings. For small businesses, just find an office with a screen and you can discuss requirements.

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗