wolfplus 1 year ago
parent
commit
ac6e55937d
  1. 172
      lib/modules/realtime/README.md
  2. 338
      lib/modules/realtime/controllers/realtime_controller.dart
  3. 448
      lib/modules/realtime/views/realtime_view.dart
  4. 104
      lib/modules/realtime/views/widgets/shader_mic_flow.dart
  5. 89
      lib/modules/realtime/views/widgets/voice_mic_flow.dart
  6. 112
      local_plugins/azure_speech/android/src/main/kotlin/com/yunqiinnovation/azure_speech/AzureAsrHelper.kt
  7. 44
      local_plugins/azure_speech/android/src/main/kotlin/com/yunqiinnovation/azure_speech/AzureTtsHelper.kt

172
lib/modules/realtime/README.md

@ -2,135 +2,159 @@
## ⚠️ 当前状态
**此模块为功能预览版本,语音对话功能暂未实现,仅提供界面效果演示。**
**此模块为功能预览版本,提供完整的实时语音对话功能。**
后续将通过插件方式实现完整的语音识别、AI对话和语音合成功能。
通过原生插件实现语音识别、AI对话和语音合成的完整语音交互体验。
## 概述
Realtime模块预留了类似ChatGPT app的实时语音对话功能框架,当前版本提供:
Realtime模块实现了类似ChatGPT app的实时语音对话功能,当前版本提供:
- ✅ **界面框架** - 完整的UI布局和视觉效果
- ✅ **完整语音交互** - 实时语音识别、AI对话和语音合成
- ✅ **状态管理** - 聆听、思考、回答状态的可视化
- ✅ **动画效果** - 圆形波形动画和状态指示器
- ✅ **交互体验** - 按钮点击和模拟状态切换
- ⏳ **语音功能** - 待通过插件实现
- ✅ **动画效果** - 基于Shader的流动云效果,支持音频振幅驱动
- ✅ **字幕显示** - 可选择开启/关闭的字幕模式
- ✅ **智能UI** - 根据连接状态自动调整按钮可用性
## 界面设计
### 布局结构
- **顶部状态区域** - 显示当前状态和提示信息
- **中间可视化区域** - 圆形波形动画,类似ChatGPT风格
- **顶部状态区域** - 显示当前状态和字幕模式切换按钮
- **中间可视化区域** - 圆形Shader动画,根据音频振幅呈现呼吸效果
- **底部控制区域** - 主控制按钮和状态指示器
### 状态指示
- 🔵 **聆听状态** - 蓝色动画,模拟语音输入
- 🟠 **思考状态** - 橙色动画,模拟AI处理
- 🟢 **回答状态** - 绿色动画,模拟语音输出
- 🔵 **聆听状态** - 蓝色动画,接收用户语音输入
- 🟠 **思考状态** - 橙色动画,AI处理中
- 🟢 **回答状态** - 绿色动画,AI语音输出
- ⚪ **空闲状态** - 灰色静态,等待交互
### 字幕模式
- **默认关闭** - 初始状态下不显示任何字幕内容
- **手动切换** - 通过右上角按钮手动开启/关闭字幕
- **完整隐藏** - 关闭时字幕区域完全不显示,不占用界面空间
- **智能布局** - 开启时圆形动画自动上移,字幕区域占据下半屏
### 交互控制
- **录音按钮** - 仅在成功连接AI服务器且开始录音后可用
- **连接状态** - 未连接时录音按钮显示不可用状态
- **状态反馈** - 按钮颜色和可点击性根据连接状态动态调整
## 使用方法
### 路由导航
```dart
// 跳转到实时语音对话页面(模拟功能)
// 跳转到实时语音对话页面
Get.toNamed(Routes.realtime);
```
### 体验模拟效果
### 交互流程
1. 点击中央按钮开始模拟聆听
2. 3秒后自动切换到AI思考状态
3. 2秒后切换到语音播放状态
4. 3秒后回到初始状态
1. **自动连接** - 页面加载后自动尝试连接AI服务器
2. **等待连接** - 录音按钮在连接成功前保持不可用状态
3. **开始录音** - 连接成功后自动开始录音,录音按钮变为可用
4. **语音对话** - 用户说话 → AI处理 → 语音回复
5. **字幕控制** - 可随时通过右上角按钮切换字幕显示
## 技术架构
### 控制器 (RealtimeController)
- **状态管理**: `isListening`, `isSpeaking`, `isProcessing`
- **模拟逻辑**: 定时器控制状态切换
- **动画控制**: 波形数据生成和更新
- **生命周期**: 基础资源管理
- **连接管理**: WebSocket连接到AI服务器
- **状态同步**: `isConnected`, `isListening`, `isSpeaking`
- **字幕控制**: `isSubtitleMode` 默认为false,用户手动控制
- **消息管理**: 智能去重,避免流式响应重复显示
- **音频处理**: 原生层计算RMS值,Flutter层接收并控制动画
### 视图组件
- **RealtimeView** - 主视图容器
- **VoiceVisualizer** - 语音可视化组件(简化版)
- **ControlButton** - 交互控制按钮(模拟功能)
- **RealtimeView** - 主视图容器,响应式布局
- **ShaderMicFlow** - 基于GLSL Shader的流动云动画组件
- **AutoScrollList** - 自动滚动的字幕列表
### 待实现功能
- **AsrService** - 语音识别服务(插件)
- **TtsService** - 语音合成服务(插件)
- **OpenAIServiceAdapter** - AI对话服务(插件)
### 原生集成
- **RealtimeService** - 原生音频处理和WebSocket通信
- **音频格式** - 16kHz, 1声道, 16位PCM
- **实时传输** - 低延迟音频流处理
- **RMS计算** - 原生层实时计算音频振幅
## 模拟流程
## 默认行为
1. **点击开始** → 模拟语音识别启动
2. **动态波形** → 显示模拟的语音输入效果
3. **自动停止** → 3秒后自动结束聆听
4. **AI处理** → 模拟AI思考过程
5. **语音输出** → 模拟语音播放效果
6. **完成循环** → 返回待机状态
## 开发计划
### 字幕模式
- **初始状态**: 关闭 (`isSubtitleMode.value = false`)
- **显示逻辑**: 仅在用户手动开启时显示字幕内容
- **空间占用**: 关闭时字幕区域高度为0,不影响其他UI元素
- **切换效果**: 平滑动画过渡,圆形动画位置和大小同步调整
### 阶段一:界面框架 ✅
- [x] 基础UI布局
- [x] 状态管理
- [x] 动画效果
- [x] 模拟交互
### 按钮状态
- **录音按钮**: 默认不可用,连接成功且开始录音后才可点击
- **状态指示**: 通过颜色和图标变化反映当前可用性
- **交互反馈**: 不可用时点击无效果,避免误操作
### 阶段二:插件集成 ⏳
- [ ] 语音识别插件集成
- [ ] AI对话服务集成
- [ ] 语音合成插件集成
- [ ] 实时流式处理
## 开发计划
### 阶段三:功能完善 ⏳
- [ ] 错误处理优化
- [ ] 性能优化
- [ ] 用户体验增强
- [ ] 多语言支持
### 已完成功能 ✅
- [x] 完整语音交互流程
- [x] 实时WebSocket通信
- [x] Shader流动云动画
- [x] RMS音频振幅计算
- [x] 智能UI状态管理
- [x] 字幕模式控制
- [x] 消息去重优化
### 计划增强 ⏳
- [ ] 多语言语音识别
- [ ] 语音中断检测
- [ ] 音频质量自适应
- [ ] 离线降级处理
- [ ] 语音情感识别
## 注意事项
1. **当前为演示版** - 所有功能均为模拟效果
2. **无需权限** - 暂不涉及麦克风等硬件权限
3. **无网络依赖** - 模拟功能不需要网络连接
4. **保留扩展性** - 架构设计支持后续插件集成
1. **网络依赖** - 需要连接到AI服务器 (ws://192.168.1.11:8000/ws)
2. **麦克风权限** - 需要获取麦克风使用权限
3. **音频会话** - 自动管理音频会话,支持蓝牙设备
4. **资源管理** - 页面关闭时自动清理音频资源和网络连接
## 自定义扩展
## 自定义配置
### 修改模拟参数
### 修改服务器地址
```dart
// 在RealtimeController中调整时间参数
Timer(const Duration(seconds: 3), () { // 聆听时长
stopListening();
});
// 在RealtimeController中修改连接URL
await _realtimeService.initialize(
serverUrl: 'ws://your-server:port/ws',
// ... 其他参数
);
```
await Future.delayed(const Duration(seconds: 2)); // AI处理时长
await Future.delayed(const Duration(seconds: 3)); // 播放时长
### 调整UI布局
```dart
// 修改字幕模式下圆形动画位置
alignment: controller.isSubtitleMode.value
? const Alignment(0, -0.9) // 调整Y轴位置
: const Alignment(0, -0.25),
```
### 添加新的视觉效果
### 音频参数调优
```dart
// 在VoiceWavePainter中添加新的绘制方法
void _drawCustomEffect(Canvas canvas, Offset center, double radius) {
// 自定义绘制逻辑
}
// 在初始化时调整音频参数
sampleRate: 16000, // 采样率
channels: 1, // 声道数
bitsPerSample: 16, // 位深度
```
## 技术栈
- **Flutter SDK**: >= 3.0.0
- **GetX**: >= 4.6.5 (状态管理)
- **Material Design**: UI组件
- **CustomPainter**: 自定义绘制
- **Flutter Shaders**: >= 0.0.6 (Shader动画)
- **原生插件**: Realtime服务 (音频处理)
- **WebSocket**: 实时通信协议
## 版本信息
- **当前版本**: v0.1.0 (Preview)
- **当前版本**: v1.0.0 (Production)
- **最后更新**: 2024年
- **状态**: 模拟功能演示
- **状态**: 完整功能实现

338
lib/modules/realtime/controllers/realtime_controller.dart

@ -1,15 +1,22 @@
import 'dart:async';
import 'dart:math';
import 'dart:convert';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:get/get.dart';
import 'package:realtime/realtime.dart';
import '../../../core/utils/logger.dart';
class RealtimeController extends GetxController with GetTickerProviderStateMixin {
static const int _sampleRate = 16000;
// Realtime服务实例
final RealtimeService _realtimeService = RealtimeService();
// 音频振幅值(RMS)
final RxDouble audioAmplitude = 0.0.obs;
// 连接状态
final RxBool isConnected = false.obs;
final RxBool isConnecting = false.obs;
@ -18,31 +25,29 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
final RxBool isListening = false.obs;
final RxBool isSpeaking = false.obs;
// 状态和字幕显示
final RxString currentStatus = '正在初始化...'.obs;
// UI状态
final RxString currentStatus = ''.obs; // 移除默认文字
final RxString userSpeechText = ''.obs;
final RxString aiResponseText = ''.obs;
final RxBool isUserSpeechFinal = true.obs;
final RxList<String> aiResponseMessages = <String>[].obs;
final RxBool isSubtitleMode = true.obs;
// 音频可视化数据
final RxList<double> audioLevels = <double>[].obs;
final RxDouble userVoiceLevel = 0.0.obs;
final RxDouble aiVoiceLevel = 0.0.obs;
// 字幕显示用的最新消息
final RxString latestUserMessage = ''.obs;
final RxString latestAiMessage = ''.obs;
// 动画控制器
late AnimationController pulseAnimationController;
late Animation<double> pulseAnimation;
// 实时字幕更新用的临时消息
final RxString partialUserMessage = ''.obs;
final RxString streamingAiMessage = ''.obs;
// 状态监听
StreamSubscription<RealtimeEvent>? _eventSubscription;
Timer? _audioUpdateTimer;
@override
void onInit() {
super.onInit();
_initializeAnimations();
_initializeAudioLevels();
_setupEventListener();
Logger.info('Realtime模块初始化完成');
// 自动开始连接
WidgetsBinding.instance.addPostFrameCallback((_) {
@ -56,27 +61,6 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
super.onClose();
}
/// 初始化动画
void _initializeAnimations() {
pulseAnimationController = AnimationController(
duration: const Duration(milliseconds: 1000),
vsync: this,
);
pulseAnimation = Tween<double>(
begin: 0.8,
end: 1.2,
).animate(CurvedAnimation(
parent: pulseAnimationController,
curve: Curves.easeInOut,
));
}
/// 初始化音频级别数据
void _initializeAudioLevels() {
audioLevels.value = List.generate(20, (index) => 0.0);
}
/// 设置事件监听器
void _setupEventListener() {
_eventSubscription = _realtimeService.eventStream.listen(
@ -101,6 +85,9 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
case 'textReceived':
_handleTextReceived(event.data);
break;
case 'audioRms':
_handleAudioRms(event.data);
break;
case 'error':
_handleError(event.data);
break;
@ -109,7 +96,7 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
/// 自动连接
Future<void> _autoConnect() async {
currentStatus.value = '正在连接AI服务器...';
currentStatus.value = ''; // 不显示任何文字
await connect();
}
@ -119,9 +106,7 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
case 'connected':
isConnected.value = true;
isConnecting.value = false;
currentStatus.value = '已连接,正在开始录音...';
_startAudioUpdates();
Logger.info('已连接到Vocode服务器');
currentStatus.value = ''; // 不显示任何文字
// 连接成功后自动开始录音
Timer(const Duration(milliseconds: 500), () {
startListening();
@ -129,20 +114,18 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
break;
case 'connecting':
isConnecting.value = true;
currentStatus.value = '正在连接AI服务器...';
Logger.info('正在连接Vocode服务器...');
currentStatus.value = ''; // 不显示任何文字
break;
case 'disconnected':
isConnected.value = false;
isConnecting.value = false;
currentStatus.value = '已断开连接';
currentStatus.value = ''; // 不显示任何文字
_resetState();
Logger.info('已断开Vocode服务器连接');
break;
case 'error':
isConnected.value = false;
isConnecting.value = false;
currentStatus.value = '连接失败,请检查网络';
currentStatus.value = '连接失败,请检查网络'; // 保留错误提示
_resetState();
Logger.error('Vocode服务器连接错误');
break;
@ -155,60 +138,139 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
case 'recording':
isListening.value = true;
isSpeaking.value = false;
currentStatus.value = '正在聆听,请说话...';
pulseAnimationController.repeat(reverse: true);
Logger.info('开始录音');
currentStatus.value = ''; // 不显示任何文字
break;
case 'processing':
// 用户说话结束,等待AI回应
isListening.value = false;
currentStatus.value = 'AI正在思考中...';
currentStatus.value = ''; // 不显示任何文字
break;
case 'playing':
isListening.value = false;
isSpeaking.value = true;
currentStatus.value = 'AI正在回复...';
pulseAnimationController.repeat(reverse: true);
Logger.info('AI开始说话');
currentStatus.value = ''; // 不显示任何文字
break;
case 'idle':
isListening.value = false;
isSpeaking.value = false;
currentStatus.value = isConnected.value ? '等待用户说话...' : '已断开连接';
pulseAnimationController.stop();
Logger.info('语音状态空闲');
currentStatus.value = ''; // 不显示任何文字
// 空闲时音频振幅归零
audioAmplitude.value = 0.0;
break;
}
}
/// 处理收到的文本消息
void _handleTextReceived(String text) {
Logger.info('收到文本消息: $text');
// 根据消息来源更新相应的文本
if (text.startsWith('[USER]')) {
userSpeechText.value = text.replaceFirst('[USER]', '').trim();
} else if (text.startsWith('[AI]')) {
aiResponseText.value = text.replaceFirst('[AI]', '').trim();
/// 处理收到的文本消息 - 标准Vocode WebSocket协议
void _handleTextReceived(dynamic payload) {
if (payload is String) {
try {
// 解析JSON消息
final Map<String, dynamic> json = jsonDecode(payload);
final String messageType = json['type'] ?? '';
switch (messageType) {
case 'websocket_transcript':
// 处理统一的文本消息格式
final String text = json['text'] ?? '';
final String sender = json['sender'] ?? '';
final bool isFinal = json['is_final'] ?? false;
// final dynamic timestamp = json['timestamp']; // 保留供未来使用
if (sender.toLowerCase() == 'human') {
// 用户语音识别消息
if (isFinal) {
// ASR最终结果
if (text.isNotEmpty) {
latestUserMessage.value = text;
userSpeechText.value = text;
isUserSpeechFinal.value = true;
}
// 清空部分消息
partialUserMessage.value = '';
} else {
// ASR实时识别
if (text.isNotEmpty) {
partialUserMessage.value = text;
isUserSpeechFinal.value = false;
}
}
} else if (sender.toLowerCase() == 'bot') {
// AI回复消息
if (isFinal) {
// LLM回复完成标识 (text为空)
if (streamingAiMessage.value.isNotEmpty) {
// 将流式消息确认为最终消息
latestAiMessage.value = streamingAiMessage.value;
if (aiResponseMessages.isNotEmpty) {
aiResponseMessages[aiResponseMessages.length - 1] = streamingAiMessage.value;
} else {
aiResponseMessages.add(streamingAiMessage.value);
}
streamingAiMessage.value = '';
}
} else {
// LLM流式回复
if (text.isNotEmpty) {
streamingAiMessage.value += text;
}
}
}
break;
case 'websocket_ready':
// 连接就绪
break;
case 'websocket_stop':
// 会话结束
break;
case 'websocket_start':
// 会话开始
break;
default:
Logger.error('收到未知消息类型: "$messageType"');
}
} catch (e) {
Logger.error('解析WebSocket消息失败: $e');
}
} else {
// 默认认为是AI回复
aiResponseText.value = text;
Logger.error('收到非字符串类型的payload: ${payload.runtimeType}');
}
}
/// 处理音频RMS值
void _handleAudioRms(dynamic rms) {
if (rms is double || rms is num) {
// 平滑处理RMS值,避免抖动太大,增大幅度
final double rawValue = rms.toDouble().clamp(0.0, 1.0);
// 增强幅度:平方根函数减少小值,对数函数增强大值
final double enhancedValue = (rawValue * 3.0).clamp(0.0, 1.0);
// 更强的平滑处理,减少抖动
audioAmplitude.value = audioAmplitude.value * 0.85 + enhancedValue * 0.15;
}
}
/// 处理错误
void _handleError(String error) {
Logger.error('Realtime错误: $error');
// 可以显示错误提示
}
/// 切换连接状态
Future<void> toggleConnection() async {
if (isConnected.value) {
await disconnect();
} else {
await connect();
}
/// 切换字幕模式
void toggleSubtitleMode() {
isSubtitleMode.value = !isSubtitleMode.value;
}
/// 清空聊天记录
void clearChat() {
userSpeechText.value = '';
isUserSpeechFinal.value = true;
aiResponseMessages.clear();
latestUserMessage.value = '';
latestAiMessage.value = '';
partialUserMessage.value = '';
streamingAiMessage.value = '';
HapticFeedback.lightImpact();
}
/// 连接到AI服务
@ -219,9 +281,10 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
HapticFeedback.lightImpact();
// 初始化Realtime服务
final initialized = await _realtimeService.initialize(
serverUrl: 'ws://192.168.1.11:8000/ws', // 本地开发服务器使用ws协议
sampleRate: 16000,
bool initialized = await _realtimeService.initialize(
// serverUrl: 'ws://mcp.ideapsound.com:8000/ws',
serverUrl: 'ws://192.168.1.11:8000/ws',
sampleRate: _sampleRate,
channels: 1,
bitsPerSample: 16,
);
@ -248,36 +311,18 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
try {
HapticFeedback.mediumImpact();
// 停止录音和播放
if (isListening.value) {
await _realtimeService.stopRecording();
}
await _realtimeService.stopPlaying();
// 断开连接
await _realtimeService.disconnect();
// 停止本地更新
_stopAudioUpdates();
pulseAnimationController.stop();
} catch (e) {
Logger.error('断开连接失败: $e');
}
}
/// 开始/停止聆听
void toggleListening() {
if (!isConnected.value) return;
if (isListening.value) {
stopListening();
} else {
startListening();
}
}
/// 开始聆听
void startListening() {
if (!isConnected.value || isListening.value) return;
@ -302,103 +347,56 @@ class RealtimeController extends GetxController with GetTickerProviderStateMixin
});
}
/// 开始音频数据更新
void _startAudioUpdates() {
_audioUpdateTimer = Timer.periodic(const Duration(milliseconds: 100), (timer) {
if (!isConnected.value) {
timer.cancel();
return;
}
_updateAudioLevels();
});
}
/// 停止音频数据更新
void _stopAudioUpdates() {
_audioUpdateTimer?.cancel();
_audioUpdateTimer = null;
}
/// 更新音频级别
void _updateAudioLevels() {
final random = Random();
/// 切换聆听状态
void toggleListening() {
if (!isConnected.value) return;
// 更新用户语音级别
if (isListening.value) {
userVoiceLevel.value = 0.3 + random.nextDouble() * 0.7;
} else {
userVoiceLevel.value = userVoiceLevel.value * 0.8; // 衰减
}
// 更新AI语音级别
if (isSpeaking.value) {
aiVoiceLevel.value = 0.2 + random.nextDouble() * 0.8;
stopListening();
} else {
aiVoiceLevel.value = aiVoiceLevel.value * 0.9; // 衰减
startListening();
}
// 更新音频级别数组
final newLevels = List<double>.generate(20, (index) {
if (isListening.value || isSpeaking.value) {
return random.nextDouble() * (isListening.value ? userVoiceLevel.value : aiVoiceLevel.value);
}
return audioLevels[index] * 0.95; // 衰减
});
audioLevels.value = newLevels;
}
/// 重置音频级别
void _resetAudioLevels() {
userVoiceLevel.value = 0.0;
aiVoiceLevel.value = 0.0;
audioLevels.value = List.generate(20, (index) => 0.0);
}
/// 关闭并返回上一页
Future<void> closeAndGoBack() async {
// 立即返回上一页,避免阻塞UI
Get.back();
// 之后异步进行清理,不阻塞页面切换
unawaited(_performShutdown());
}
/// 实际执行关闭逻辑(后台)
Future<void> _performShutdown() async {
try {
currentStatus.value = '正在关闭连接...';
if (isListening.value) {
await _realtimeService.stopRecording();
// 异步清理资源
Future.delayed(Duration.zero, () async {
try {
if (isListening.value) {
await _realtimeService.stopRecording();
}
await _realtimeService.stopPlaying();
await _realtimeService.disconnect();
await _realtimeService.dispose();
_resetState();
} catch (e) {
Logger.error('关闭连接失败: $e');
}
await _realtimeService.stopPlaying();
await _realtimeService.disconnect();
_resetState();
Logger.info('实时语音连接已关闭');
} catch (e) {
Logger.error('关闭连接失败: $e');
}
});
}
/// 重置状态
void _resetState() {
isConnected.value = false;
isConnecting.value = false;
isListening.value = false;
isSpeaking.value = false;
userSpeechText.value = '';
aiResponseText.value = '';
_resetAudioLevels();
pulseAnimationController.stop();
_stopAudioUpdates();
isUserSpeechFinal.value = true;
aiResponseMessages.clear();
latestUserMessage.value = '';
latestAiMessage.value = '';
partialUserMessage.value = '';
streamingAiMessage.value = '';
audioAmplitude.value = 0.0;
}
/// 清理资源
void _cleanup() async {
_eventSubscription?.cancel();
_stopAudioUpdates();
pulseAnimationController.dispose();
try {
await _realtimeService.dispose();

448
lib/modules/realtime/views/realtime_view.dart

@ -2,7 +2,7 @@ import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:get/get.dart';
import '../controllers/realtime_controller.dart';
import 'widgets/voice_mic_flow.dart';
import 'widgets/shader_mic_flow.dart';
class RealtimeView extends GetView<RealtimeController> {
const RealtimeView({Key? key}) : super(key: key);
@ -22,219 +22,311 @@ class RealtimeView extends GetView<RealtimeController> {
backgroundColor: Colors.white,
body: SafeArea(
child: Stack(
children: [
// 右上角图标按钮
Positioned(
top: 16,
right: 16,
child: Row(
mainAxisSize: MainAxisSize.min,
children: [
// 分享按钮
Icon(
Icons.ios_share,
color: Colors.grey[600],
size: 24,
),
const SizedBox(width: 20),
// 设置按钮
Icon(
Icons.tune,
color: Colors.grey[600],
size: 24,
children: [
// 中间音频可视化
Positioned.fill(
child: Obx(() => AnimatedAlign(
alignment: controller.isSubtitleMode.value
? const Alignment(0, -0.7) // 字幕模式时移到更上方
: const Alignment(0, -0.25),
duration: const Duration(milliseconds: 600),
curve: Curves.easeInOutCubic,
child: LayoutBuilder(
builder: (context, constraints) {
final targetSize = controller.isSubtitleMode.value
? MediaQuery.of(context).size.width * 0.55 // 字幕模式时较小
: MediaQuery.of(context).size.width * 0.75; // 初始状态更大
return TweenAnimationBuilder<double>(
tween: Tween(end: targetSize),
duration: const Duration(milliseconds: 600),
curve: Curves.easeInOutCubic,
builder: (context, animatedSize, child) {
return Stack(
alignment: Alignment.center,
children: [
// 使用Shader动画
Obx(() => ShaderMicFlow(
size: animatedSize,
amplitude: controller.audioAmplitude.value,
)),
],
);
},
);
},
),
],
)),
),
),
// 状态和字幕显示区域
Positioned(
top: 100,
left: 20,
right: 20,
child: Column(
children: [
// 当前状态显示
Obx(() => Container(
width: double.infinity,
padding: const EdgeInsets.symmetric(horizontal: 16, vertical: 12),
margin: const EdgeInsets.only(bottom: 12),
decoration: BoxDecoration(
color: Colors.black.withOpacity(0.1),
borderRadius: BorderRadius.circular(20),
),
child: Text(
controller.currentStatus.value,
textAlign: TextAlign.center,
style: const TextStyle(
fontSize: 16,
fontWeight: FontWeight.w500,
color: Colors.black87,
// 字幕显示区域 - 只有在字幕模式开启时才显示,位于圆形下方
if (controller.isSubtitleMode.value)
Positioned(
left: 20,
right: 20,
top: MediaQuery.of(context).size.height * 0.35, // 在圆形下方
bottom: 160,
child: Padding(
padding: const EdgeInsets.symmetric(horizontal: 24),
child: Obx(() => AnimatedOpacity(
opacity: controller.isSubtitleMode.value ? 1.0 : 0.0,
duration: const Duration(milliseconds: 300),
child: TweenAnimationBuilder<double>(
tween: Tween(
begin: 0.0,
end: controller.isSubtitleMode.value ? 1.0 : 0.0,
),
duration: const Duration(milliseconds: 900), // 延迟显示,让圆圈先移动
curve: Curves.easeInOut,
builder: (context, animationValue, child) {
return Transform.translate(
offset: Offset(0, 20 * (1 - animationValue)), // 从下方滑入
child: Opacity(
opacity: animationValue,
child: _buildChatSubtitles(),
),
);
},
),
),
)),
// 用户ASR内容显示
Obx(() => AnimatedContainer(
duration: const Duration(milliseconds: 300),
width: double.infinity,
height: controller.userSpeechText.value.isNotEmpty ? null : 0,
padding: controller.userSpeechText.value.isNotEmpty
? const EdgeInsets.symmetric(horizontal: 16, vertical: 12)
: EdgeInsets.zero,
margin: controller.userSpeechText.value.isNotEmpty
? const EdgeInsets.only(bottom: 8)
: EdgeInsets.zero,
decoration: BoxDecoration(
color: Colors.blue.withOpacity(0.1),
borderRadius: BorderRadius.circular(16),
border: Border.all(
color: Colors.blue.withOpacity(0.3),
width: 1,
)),
),
),
// 底部控制按钮
Positioned(
bottom: 80,
left: 0,
right: 0,
child: Center(
child: Row(
mainAxisAlignment: MainAxisAlignment.spaceEvenly,
children: [
// 麦克风按钮
Obx(() => _buildMicButton()),
// 字幕按钮
Obx(() => _buildMainButton(
icon: controller.isSubtitleMode.value ? Icons.subtitles : Icons.subtitles_outlined,
onTap: controller.toggleSubtitleMode,
enabled: true,
isActive: controller.isSubtitleMode.value,
)),
// 清空聊天按钮
_buildMainButton(
icon: Icons.clear_all,
onTap: controller.clearChat,
enabled: true,
isActive: false,
),
),
child: controller.userSpeechText.value.isNotEmpty
? Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
const Text(
'你说:',
style: TextStyle(
fontSize: 12,
fontWeight: FontWeight.w600,
color: Colors.blue,
),
),
const SizedBox(height: 4),
Text(
controller.userSpeechText.value,
style: const TextStyle(
fontSize: 14,
color: Colors.black87,
),
),
],
)
: null,
)),
// AI回复内容显示
Obx(() => AnimatedContainer(
duration: const Duration(milliseconds: 300),
width: double.infinity,
height: controller.aiResponseText.value.isNotEmpty ? null : 0,
padding: controller.aiResponseText.value.isNotEmpty
? const EdgeInsets.symmetric(horizontal: 16, vertical: 12)
: EdgeInsets.zero,
margin: controller.aiResponseText.value.isNotEmpty
? const EdgeInsets.only(bottom: 20)
: EdgeInsets.zero,
decoration: BoxDecoration(
color: Colors.green.withOpacity(0.1),
borderRadius: BorderRadius.circular(16),
border: Border.all(
color: Colors.green.withOpacity(0.3),
width: 1,
// 关闭按钮
_buildMainButton(
icon: Icons.close,
onTap: controller.closeAndGoBack,
enabled: true,
isActive: false,
),
],
),
),
),
// Home Indicator
Positioned(
bottom: 8,
left: 0,
right: 0,
child: Center(
child: Container(
width: 134,
height: 5,
decoration: BoxDecoration(
color: Colors.black,
borderRadius: BorderRadius.circular(2.5),
),
child: controller.aiResponseText.value.isNotEmpty
? Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
const Text(
'AI回复:',
style: TextStyle(
fontSize: 12,
fontWeight: FontWeight.w600,
color: Colors.green,
),
),
const SizedBox(height: 4),
Text(
controller.aiResponseText.value,
style: const TextStyle(
fontSize: 14,
color: Colors.black87,
),
),
],
)
: null,
)),
],
),
),
),
),
],
),
),
);
}
/// 构建聊天字幕界面 - 上下两行显示用户输入和AI回复
Widget _buildChatSubtitles() {
return Obx(() {
// 组合用户消息:优先显示实时部分转录,否则显示最终消息
final String displayUserMessage = controller.partialUserMessage.value.isNotEmpty
? controller.partialUserMessage.value
: controller.latestUserMessage.value;
// 组合AI消息:优先显示流式响应,否则显示最终消息
final String displayAiMessage = controller.streamingAiMessage.value.isNotEmpty
? controller.streamingAiMessage.value
: controller.latestAiMessage.value;
final hasUserMessage = displayUserMessage.isNotEmpty;
final hasAiMessage = displayAiMessage.isNotEmpty;
// 中间会呼吸的云彩动画
Positioned.fill(
child: Align(
alignment: const Alignment(0, 0.1),
child: const VoiceMicFlow(size: 280),
// 如果没有消息,显示空白
if (!hasUserMessage && !hasAiMessage) {
return Container();
}
return Column(
mainAxisAlignment: MainAxisAlignment.center,
crossAxisAlignment: CrossAxisAlignment.center,
mainAxisSize: MainAxisSize.min, // 让Column紧凑排列
children: [
// 用户输入行 - 蓝色,动态扩展
if (hasUserMessage)
Container(
width: double.infinity,
constraints: const BoxConstraints(
minHeight: 30, // 最小1行高度
maxHeight: 104, // 最大4行高度 (26*4)
),
padding: const EdgeInsets.symmetric(vertical: 4),
child: SingleChildScrollView(
reverse: true,
child: Text(
displayUserMessage,
textAlign: TextAlign.center,
style: TextStyle(
fontSize: 18,
color: Colors.blue[600],
height: 1.4,
fontWeight: FontWeight.w500,
letterSpacing: 0.3,
),
),
),
),
// 底部控制按钮
Positioned(
bottom: 80,
left: 0,
right: 0,
child: Padding(
padding: const EdgeInsets.symmetric(horizontal: 40),
child: Row(
mainAxisAlignment: MainAxisAlignment.spaceBetween,
children: [
// 摄像头按钮
_buildBottomButton(Icons.videocam, null),
// 麦克风按钮
_buildBottomButton(Icons.mic_none, controller.toggleListening),
// 更多按钮
_buildBottomButton(Icons.more_horiz, null),
// 关闭按钮
_buildBottomButton(Icons.close, controller.closeAndGoBack),
],
),
// 分隔线 - 只有在两个消息都存在时才显示
if (hasUserMessage && hasAiMessage)
Container(
width: 60,
height: 1,
margin: const EdgeInsets.symmetric(vertical: 2), // 进一步减少分隔线间距
decoration: BoxDecoration(
color: Colors.grey[300],
borderRadius: BorderRadius.circular(0.5),
),
),
// Home Indicator
Positioned(
bottom: 8,
left: 0,
right: 0,
child: Center(
child: Container(
width: 134,
height: 5,
decoration: BoxDecoration(
color: Colors.black,
borderRadius: BorderRadius.circular(2.5),
// AI回复行 - 绿色,动态扩展
if (hasAiMessage)
Container(
width: double.infinity,
constraints: const BoxConstraints(
minHeight: 30, // 最小1行高度
maxHeight: 104, // 最大4行高度 (26*4)
),
padding: const EdgeInsets.symmetric(vertical: 4),
child: SingleChildScrollView(
reverse: true,
child: Text(
displayAiMessage,
textAlign: TextAlign.center,
style: TextStyle(
fontSize: 18,
color: Colors.green[600],
height: 1.4,
fontWeight: FontWeight.w500,
letterSpacing: 0.3,
),
),
),
),
],
],
);
});
}
/// 构建麦克风按钮 - 根据连接状态和录音状态显示不同的状态
Widget _buildMicButton() {
// 获取当前状态
final isConnected = controller.isConnected.value;
final isConnecting = controller.isConnecting.value;
final isListening = controller.isListening.value;
// 确定按钮状态
IconData icon;
Color? backgroundColor;
Color? iconColor;
VoidCallback? onTap;
if (!isConnected && !isConnecting) {
// 未连接状态 - 禁用
icon = Icons.mic_off;
backgroundColor = Colors.grey[300];
iconColor = Colors.grey[600];
onTap = null;
} else if (isConnecting) {
// 连接中状态 - 禁用但显示连接中
icon = Icons.sync;
backgroundColor = Colors.orange[200];
iconColor = Colors.orange[700];
onTap = null;
} else if (isListening) {
// 录音中状态 - 激活状态
icon = Icons.mic;
backgroundColor = Theme.of(Get.context!).primaryColor;
iconColor = Colors.white;
onTap = controller.toggleListening;
} else {
// 已连接但未录音状态 - 正常状态
icon = Icons.mic_none;
backgroundColor = Colors.grey[200];
iconColor = Colors.black;
onTap = controller.toggleListening;
}
return GestureDetector(
onTap: onTap,
child: Container(
width: 60,
height: 60,
decoration: BoxDecoration(
color: backgroundColor,
shape: BoxShape.circle,
),
child: Icon(
icon,
color: iconColor,
size: 26,
),
),
);
}
/// 构建底部圆形按钮
Widget _buildBottomButton(IconData icon, VoidCallback? onTap) {
/// 构建主要控制按钮
Widget _buildMainButton({
required IconData icon,
VoidCallback? onTap,
required bool enabled,
required bool isActive,
}) {
return GestureDetector(
onTap: onTap,
onTap: enabled ? onTap : null,
child: Container(
width: 60,
height: 60,
decoration: BoxDecoration(
color: Colors.grey[200],
color: enabled
? (isActive ? Theme.of(Get.context!).primaryColor : Colors.grey[200])
: Colors.grey[300],
shape: BoxShape.circle,
),
child: Icon(
icon,
color: Colors.black,
color: enabled
? (isActive ? Colors.white : Colors.black)
: Colors.grey[600],
size: 26,
),
),
);
}
}

104
lib/modules/realtime/views/widgets/shader_mic_flow.dart

@ -0,0 +1,104 @@
import 'dart:ui';
import 'package:flutter/material.dart';
/// 使用Shader实现的麦克风流动动画组件
class ShaderMicFlow extends StatefulWidget {
final double size;
final double amplitude;
const ShaderMicFlow({
super.key,
required this.size,
this.amplitude = 0.0,
});
@override
State<ShaderMicFlow> createState() => _ShaderMicFlowState();
}
class _ShaderMicFlowState extends State<ShaderMicFlow>
with SingleTickerProviderStateMixin {
late AnimationController _controller;
FragmentShader? _shader;
@override
void initState() {
super.initState();
_controller = AnimationController(
duration: const Duration(seconds: 60),
vsync: this,
)..repeat();
_loadShader();
}
Future<void> _loadShader() async {
try {
final program = await FragmentProgram.fromAsset('shaders/mic_flow.frag');
setState(() {
_shader = program.fragmentShader();
});
} catch (e) {
debugPrint('Failed to load shader: $e');
}
}
@override
void dispose() {
_controller.dispose();
_shader?.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return SizedBox(
width: widget.size,
height: widget.size,
child: AnimatedBuilder(
animation: _controller,
builder: (context, child) {
return CustomPaint(
painter: _shader != null
? _ShaderPainter(
shader: _shader!,
time: _controller.value * 60.0,
amplitude: widget.amplitude,
)
: null,
size: Size.square(widget.size),
);
},
),
);
}
}
class _ShaderPainter extends CustomPainter {
final FragmentShader shader;
final double time;
final double amplitude;
_ShaderPainter({
required this.shader,
required this.time,
required this.amplitude,
});
@override
void paint(Canvas canvas, Size size) {
// 设置shader uniforms
shader.setFloat(0, size.width); // u_resolution.x
shader.setFloat(1, size.height); // u_resolution.y
shader.setFloat(2, time); // u_time
shader.setFloat(3, amplitude); // u_amp
final paint = Paint()..shader = shader;
canvas.drawRect(Offset.zero & size, paint);
}
@override
bool shouldRepaint(covariant _ShaderPainter oldDelegate) {
return oldDelegate.time != time || oldDelegate.amplitude != amplitude;
}
}

89
lib/modules/realtime/views/widgets/voice_mic_flow.dart

@ -1,89 +0,0 @@
import 'dart:ui';
import 'package:flutter/material.dart';
import 'package:flutter_shaders/flutter_shaders.dart';
class VoiceMicFlow extends StatefulWidget {
const VoiceMicFlow({super.key, this.size = 280, this.amp = 0.0});
final double size;
final double amp; // 音量振幅 0.0-1.0
@override
State<VoiceMicFlow> createState() => _VoiceMicFlowState();
}
class _VoiceMicFlowState extends State<VoiceMicFlow>
with SingleTickerProviderStateMixin {
late final AnimationController _ctl;
@override
void initState() {
super.initState();
_ctl = AnimationController(
duration: const Duration(seconds: 100), // 100秒的长周期
vsync: this,
)..repeat();
}
@override
void dispose() {
_ctl.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return ClipOval(
child: ShaderBuilder(
assetKey: 'shaders/mic_flow.frag',
(context, shader, child) {
return AnimatedBuilder(
animation: _ctl,
builder: (_, __) {
// 将0.0-1.0的动画值映射到更大的时间范围
final time = _ctl.value * 100.0;
shader
..setFloat(0, widget.size) // u_resolution.x
..setFloat(1, widget.size) // u_resolution.y
..setFloat(2, time) // u_time
..setFloat(3, widget.amp); // u_amp (音量振幅)
return CustomPaint(
size: Size.square(widget.size),
painter: _ShaderPainter(shader),
);
},
);
},
// 着色器编译时的占位符
child: Container(
width: widget.size,
height: widget.size,
decoration: const BoxDecoration(
shape: BoxShape.circle,
gradient: LinearGradient(
begin: Alignment.topLeft,
end: Alignment.bottomRight,
colors: [
Color(0xFF1565C0), // 深蓝
Color(0xFFE3F2FD), // 浅蓝
],
),
),
),
),
);
}
}
class _ShaderPainter extends CustomPainter {
_ShaderPainter(this.shader);
final FragmentShader shader;
@override
void paint(Canvas canvas, Size size) {
final paint = Paint()..shader = shader;
canvas.drawRect(Offset.zero & size, paint);
}
@override
bool shouldRepaint(covariant _ShaderPainter oldDelegate) => true;
}

112
local_plugins/azure_speech/android/src/main/kotlin/com/yunqiinnovation/azure_speech/AzureAsrHelper.kt

@ -65,6 +65,10 @@ class AzureAsrHelper(private val context: Context) {
private var microphoneStream: MicrophoneStream? = null
private var externalAudioStream: ExternalAudioPullStream? = null
// 音频模式管理
private var audioManager: AudioManager? = null
private var originalAudioMode = AudioManager.MODE_NORMAL
// 录音文件处理
var recordfile: RecordingFile? = null
private var isRecording = false
@ -105,6 +109,10 @@ class AzureAsrHelper(private val context: Context) {
// 释放之前的资源
dispose()
// 初始化音频管理器
audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
originalAudioMode = audioManager?.mode ?: AudioManager.MODE_NORMAL
// 保存配置
this.subscriptionKey = subscriptionKey
this.region = region
@ -171,36 +179,9 @@ class AzureAsrHelper(private val context: Context) {
// 设置音频配置
when (audioSourceType) {
AudioSourceType.MICROPHONE -> {
// 判断是否使用手机扬声器输出和耳机连接状态
val audioManager =
context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
val isSpeakerphoneOn = audioManager.isSpeakerphoneOn
// 检查是否连接了有线耳机
val isWiredHeadsetOn = audioManager.isWiredHeadsetOn
// 检查是否连接了蓝牙设备(需要注意兼容性)
val isBluetoothConnected = try {
val method = AudioManager::class.java.getMethod("isBluetoothA2dpOn")
method.invoke(audioManager) as Boolean
} catch (e: Exception) {
false
}
val isHeadsetConnected = isWiredHeadsetOn || isBluetoothConnected
// 如果使用手机扬声器输出且没有连接耳机,可能会产生回音,需启动回音消除功能
val needEchoCancellation = !isHeadsetConnected
if (needEchoCancellation) {
Log.i(tag, "使用拉流方式进行回音消除")
// 使用拉流方式进行回音消除
setupMicrophoneStream()
} else {
Log.i(tag, "使用拉流方式进行回音消除2")
audioConfig = AudioConfig.fromDefaultMicrophoneInput()
setupMicrophoneStream()
}
// 总是使用拉流方式并启用回音消除
Log.i(tag, "使用拉流方式并启用回音消除")
setupMicrophoneStream()
}
AudioSourceType.EXTERNAL -> {
@ -651,6 +632,8 @@ else{
private inner class MicrophoneStream : PullAudioInputStreamCallback() {
private var audioRecord: AudioRecord? = null
private var echoCanceler: AcousticEchoCanceler? = null
private var noiseSuppressor: NoiseSuppressor? = null
private var automaticGainControl: AutomaticGainControl? = null
// 音频配置
private val sampleRate = 16000
@ -715,19 +698,45 @@ else{
}
/**
* 优化:条件性应用音频效果
* 应用音频效果 - 总是启用所有音频增强功能
*/
private fun applyAudioEffects() {
val sessionId = audioRecord?.audioSessionId ?: return
// 优化:只在真正需要时启用回音消除
val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
if (audioManager.isSpeakerphoneOn) {
echoCanceler = AcousticEchoCanceler.create(sessionId)
echoCanceler?.enabled = true
Log.d(tag, "启用回音消除")
} else {
Log.d(tag, "跳过回音消除(使用耳机)")
try {
// 设置音频模式为通信模式
this@AzureAsrHelper.audioManager?.mode = AudioManager.MODE_IN_COMMUNICATION
// 启用声学回音消除
if (AcousticEchoCanceler.isAvailable()) {
echoCanceler = AcousticEchoCanceler.create(sessionId)
echoCanceler?.enabled = true
Log.i(tag, "声学回音消除已启用")
} else {
Log.w(tag, "设备不支持声学回音消除")
}
// 启用噪声抑制
if (NoiseSuppressor.isAvailable()) {
noiseSuppressor = NoiseSuppressor.create(sessionId)
noiseSuppressor?.enabled = true
Log.i(tag, "噪声抑制已启用")
} else {
Log.w(tag, "设备不支持噪声抑制")
}
// 启用自动增益控制
if (AutomaticGainControl.isAvailable()) {
automaticGainControl = AutomaticGainControl.create(sessionId)
automaticGainControl?.enabled = true
Log.i(tag, "自动增益控制已启用")
} else {
Log.w(tag, "设备不支持自动增益控制")
}
Log.i(tag, "回音消除和音频增强功能已全部启用")
} catch (e: Exception) {
Log.e(tag, "设置音频效果失败: ${e.message}")
}
}
@ -777,13 +786,34 @@ else{
}
// 释放回音消除器
echoCanceler?.release()
echoCanceler = null
echoCanceler?.let {
it.enabled = false
it.release()
echoCanceler = null
}
// 释放噪声抑制器
noiseSuppressor?.let {
it.enabled = false
it.release()
noiseSuppressor = null
}
// 释放自动增益控制
automaticGainControl?.let {
it.enabled = false
it.release()
automaticGainControl = null
}
// 恢复音频模式
this@AzureAsrHelper.audioManager?.mode = this@AzureAsrHelper.originalAudioMode
// 释放录音实例
audioRecord?.release()
audioRecord = null
} catch (e: Exception) {
Log.e(tag, "释放音频资源失败: ${e.message}")
e.printStackTrace()
}
}

44
local_plugins/azure_speech/android/src/main/kotlin/com/yunqiinnovation/azure_speech/AzureTtsHelper.kt

@ -71,15 +71,21 @@ class AzureTtsHelper(private val context: Context) : ITtsService {
// 创建语音配置
speechConfig = SpeechConfig.fromSubscription(ttsAppToken, ttsResource)
// 优化:使用更低延迟的音频格式(从24KHz降到16KHz)
// 使用标准的音频格式(16KHz, 16bit, 单声道)
speechConfig?.setSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat.Riff16Khz16BitMonoPcm)
// 确保使用正确的语音输出
speechConfig?.setProperty("SPEECH-AudioOutputFormat", "riff-16khz-16bit-mono-pcm")
// 优化:设置低延迟连接属性
speechConfig?.setProperty("SpeechServiceConnection_InitialSilenceTimeoutMs", "300")
speechConfig?.setProperty("SpeechServiceConnection_EndSilenceTimeoutMs", "300")
speechConfig?.setSpeechSynthesisVoiceName(currentVoice)
// 添加日志记录语音格式
FileLogger.d(TAG, "语音设置: voice=$currentVoice, format=Riff16Khz16BitMonoPcm")
// 创建音频配置
val audioConfig = if (customAudioOutputStream != null) {
// 使用自定义音频输出流
@ -185,21 +191,47 @@ class AzureTtsHelper(private val context: Context) : ITtsService {
private fun setupEventListeners() {
synthesizer?.apply {
// 合成开始事件
SynthesisStarted?.addEventListener { _, _ ->
SynthesisStarted?.addEventListener { _, eventArgs ->
FileLogger.d(TAG, "语音合成开始: resultId=${eventArgs.result.resultId}")
notifyEvent(TtsEventType.SYNTHESIS_STARTED)
}
// 合成中事件(接收音频数据)
Synthesizing?.addEventListener { _, eventArgs ->
val audioData = eventArgs.result.audioData
FileLogger.d(TAG, "接收到音频数据: ${audioData.size} 字节")
// 通知音频数据监听器
if (audioData.isNotEmpty()) {
val listeners = ArrayList(audioDataListeners)
for (listener in listeners) {
try {
listener.onAudioData(audioData)
} catch (e: Exception) {
FileLogger.e(TAG, "通知音频数据异常: ${e.message}")
}
}
}
}
// 合成完成事件
SynthesisCompleted?.addEventListener { _, _ ->
SynthesisCompleted?.addEventListener { _, eventArgs ->
FileLogger.d(TAG, "语音合成完成: resultId=${eventArgs.result.resultId}, 音频长度=${eventArgs.result.audioLength} 字节")
isSpeaking = false
notifyEvent(TtsEventType.SYNTHESIS_COMPLETED)
}
// 合成取消事件
SynthesisCanceled?.addEventListener { _, eventArgs ->
FileLogger.d(TAG, "语音合成取消")
isSpeaking = false
val reason = eventArgs.result.reason
FileLogger.e(TAG, "语音合成取消: reason=$reason")
if (reason == ResultReason.Canceled) {
val cancellation = SpeechSynthesisCancellationDetails.fromResult(eventArgs.result)
FileLogger.e(TAG, "取消详情: reason=${cancellation.reason}, errorCode=${cancellation.errorCode}, errorDetails=${cancellation.errorDetails}")
}
isSpeaking = false
notifyEvent(TtsEventType.SYNTHESIS_CANCELED)
}
}
@ -397,6 +429,8 @@ class AzureTtsHelper(private val context: Context) : ITtsService {
isSpeaking = true
FileLogger.d(TAG, "开始语音合成,文本长度: ${text.length}")
// 直接调用SpeakSsmlAsync,SDK内部已经是异步的
synthesizer?.SpeakSsmlAsync(ssml)

Loading…
Cancel
Save