[FEATURE]: Education AI Suite - System & Performance Metrics panel to display LLM/ASR configuration and performance KPIs for Smart Classroom sample app
Description Extend the System & Performance Metrics panel to include software-level metrics such as model names, inference latency, and token throughput. This helps ISVs evaluate the performance of their AI workloads on Intel platforms.
Assumptions:
- Models include Qwen2-7B, Qwenvl2-2B, and ASR models like Whisper or Conformer 1 2.
- Metrics include TTFT (Time to First Token), TPOT (Time Per Output Token), and total tokens processed.
- Metrics are collected per session and reset on new input.
Dependencies
- LLM and ASR inference engines (IPEX-LLM, OpenVINO)
- Token counters and latency timers
- GUI integration
Expected Output
- Display of:
- LLM: Qwen2-7B, Qwenvl2-2B
- ASR: [Model name]
- TTFT, TPOT, total tokens
- Time to generate summary
Acceptance Criteria
- Metrics are displayed after each inference session
- Values are accurate and updated per session
- Display is stable and readable