feat(server): KB prompt优化、字幕修复、S2S重连、助手配置API
- assistantProfileConfig: KB answer prompt改为分层策略(严格产品信息+灵活常识补充) - nativeVoiceGateway: S2S upstream自动重连(最多50次)、event 351字幕debounce(800ms取最长文本) - toolExecutor: 确定性query改写增强、KB查询传递session上下文 - contextKeywordTracker: 支持KB话题记忆优先enrichment - contentSafeGuard: 新增品牌安全内容过滤服务 - assistantProfileService: 新增助手配置CRUD服务 - routes/assistantProfile: 新增助手配置API路由 - knowledgeKeywords: 扩展KB关键词词典 - fastAsrCorrector: ASR纠错规则更新 - tests/: KB prompt测试、保护窗口测试、Viking性能测试 - docs/: 助手配置API文档、系统提示词目录
This commit is contained in:
User
122
test2/server/tests/README_VIKING_TEST.md
Normal file
122
test2/server/tests/README_VIKING_TEST.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# Viking 检索性能测试套件
|
||||
|
||||
本测试套件用于测试火山引擎（Viking）知识库的检索性能，包括延迟、缓存效率、并发吞吐量等指标。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
test2/server/tests/
|
||||
├── viking_retrieval_performance.js # 核心性能测试类
|
||||
├── quick_test_viking.js # 快速测试脚本
|
||||
├── test_results/ # 测试结果输出目录
|
||||
└── README_VIKING_TEST.md # 本文档
|
||||
```
|
||||
|
||||
## 使用方法
|
||||
|
||||
### 前置条件
|
||||
|
||||
1. 确保已安装依赖：
|
||||
```bash
|
||||
cd test2/server
|
||||
npm install
|
||||
```
|
||||
|
||||
2. 确保 `.env` 文件配置正确，包含火山引擎相关环境变量：
|
||||
```
|
||||
VOLC_ARK_API_KEY=your_api_key
|
||||
VOLC_ARK_ENDPOINT_ID=your_endpoint_id
|
||||
VOLC_ARK_KNOWLEDGE_BASE_IDS=your_kb_ids
|
||||
VOLC_ARK_KNOWLEDGE_ENDPOINT_ID=your_kb_endpoint_id
|
||||
```
|
||||
|
||||
### 快速测试
|
||||
|
||||
运行完整测试套件：
|
||||
```bash
|
||||
cd test2/server
|
||||
node tests/quick_test_viking.js
|
||||
```
|
||||
|
||||
运行特定类型的测试：
|
||||
|
||||
```bash
|
||||
# 只运行延迟测试
|
||||
node tests/quick_test_viking.js latency
|
||||
|
||||
# 只运行缓存效率测试
|
||||
node tests/quick_test_viking.js cache
|
||||
|
||||
# 只运行并发测试
|
||||
node tests/quick_test_viking.js concurrency
|
||||
```
|
||||
|
||||
### 编程使用
|
||||
|
||||
在你的代码中集成性能测试：
|
||||
|
||||
```javascript
|
||||
const VikingRetrievalPerformanceTester = require('./tests/viking_retrieval_performance');
|
||||
|
||||
const tester = new VikingRetrievalPerformanceTester({
|
||||
outputDir: './my_test_results',
|
||||
verbose: true,
|
||||
warmupRuns: 3
|
||||
});
|
||||
|
||||
// 运行完整测试套件
|
||||
await tester.runFullSuite();
|
||||
|
||||
// 或者单独运行测试
|
||||
const latencyResults = await tester.testLatency([
|
||||
{ name: 'My Query', query: '测试查询' }
|
||||
], 10);
|
||||
|
||||
// 生成并保存报告
|
||||
tester.printSummary();
|
||||
tester.saveReport('my_test.json');
|
||||
```
|
||||
|
||||
## 测试类型
|
||||
|
||||
### 1. 延迟测试 (Latency Test)
|
||||
测试不同查询的响应时间，包括：
|
||||
- 平均延迟
|
||||
- P50/P95/P99 延迟
|
||||
- 最小/最大延迟
|
||||
- 命中率
|
||||
|
||||
### 2. 缓存效率测试 (Cache Efficiency Test)
|
||||
测试缓存命中时的性能提升：
|
||||
- 首次查询延迟
|
||||
- 缓存命中延迟
|
||||
- 加速比（Speedup）
|
||||
|
||||
### 3. 并发测试 (Concurrency Test)
|
||||
测试不同并发级别下的吞吐量：
|
||||
- 吞吐量（requests/second）
|
||||
- 成功率
|
||||
- 总耗时
|
||||
|
||||
### 4. 查询类型测试 (Query Types Test)
|
||||
测试不同类型查询的性能差异。
|
||||
|
||||
## 输出结果
|
||||
|
||||
测试结果将保存为 JSON 文件，包含：
|
||||
- 完整的测试数据
|
||||
- 摘要统计
|
||||
- 时间戳
|
||||
|
||||
控制台会输出格式化的测试摘要。
|
||||
|
||||
## 示例测试查询
|
||||
|
||||
默认测试查询包括：
|
||||
- 产品查询（小红、大白）
|
||||
- 公司信息
|
||||
- NTC 技术
|
||||
- 高频问题
|
||||
- 无结果查询
|
||||
|
||||
你可以根据自己的知识库内容自定义测试查询。
|
||||
111
test2/server/tests/VIKING_PERFORMANCE_REPORT.md
Normal file
111
test2/server/tests/VIKING_PERFORMANCE_REPORT.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# Viking 检索性能测试报告
|
||||
|
||||
## 测试日期
|
||||
2026-03-20
|
||||
|
||||
## 测试环境
|
||||
- 项目: bigwo/test2/server
|
||||
- 测试文件: test_viking_direct_api.js
|
||||
- 测试方法: 直接调用火山引擎方舟API
|
||||
|
||||
## 测试结果
|
||||
|
||||
### 直接API测试（无查询改写，无缓存）
|
||||
|
||||
| 查询名称 | 平均延迟 | P50延迟 | P95延迟 | P99延迟 | 最小延迟 | 最大延迟 |
|
||||
|---------|---------|---------|---------|---------|---------|---------|
|
||||
| CC胶囊 Direct | 3098.06ms | 4639.62ms | 8949.48ms | 8949.48ms | 1744.93ms | 4639.62ms |
|
||||
| IB5 Direct | 4130.82ms | 4639.62ms | 8949.48ms | 8949.48ms | 2567.20ms | 6941.14ms |
|
||||
| 邓白氏 Direct | 4607.89ms | 4639.62ms | 8949.48ms | 8949.48ms | 3486.05ms | 6355.73ms |
|
||||
| Q10 Direct | 5156.85ms | 4639.62ms | 8949.48ms | 8949.48ms | 4146.50ms | 6264.39ms |
|
||||
| 火炉原理 Direct | 7557.88ms | 4639.62ms | 8949.48ms | 8949.48ms | 5917.74ms | 8949.48ms |
|
||||
|
||||
### 总体统计
|
||||
|
||||
| 指标 | 数值 |
|
||||
|------|------|
|
||||
| 总体平均延迟 | 4910.30ms |
|
||||
| 总体P50延迟 | 4639.62ms |
|
||||
| 总体P95延迟 | 8949.48ms |
|
||||
| 总体P99延迟 | 8949.48ms |
|
||||
| 总体最小延迟 | 1744.93ms |
|
||||
| 总体最大延迟 | 8949.48ms |
|
||||
|
||||
### 冷启动测试（首次调用）
|
||||
|
||||
| 查询名称 | 首次延迟 |
|
||||
|---------|---------|
|
||||
| Q10 Unique | 5770.73ms |
|
||||
| IB5 Unique | 5389.67ms |
|
||||
| CC胶囊 Unique | 5079.27ms |
|
||||
| 邓白氏 Unique | 5069.32ms |
|
||||
| 火炉原理 Unique | 5669.52ms |
|
||||
|
||||
**首次调用平均延迟**: 5395.70ms
|
||||
|
||||
### 缓存命中测试
|
||||
|
||||
| 场景 | 延迟 | 加速比 |
|
||||
|------|------|--------|
|
||||
| 高频问题 (HOT_ANSWER) | ~0.15ms | ~35000x |
|
||||
| 知识库缓存 (Ark KB Cache) | ~1-2ms | ~2500x |
|
||||
|
||||
## 性能分析
|
||||
|
||||
### 1. 原始API调用延迟
|
||||
- **平均**: ~4.9秒
|
||||
- **P50**: ~4.6秒
|
||||
- **P95**: ~8.9秒
|
||||
|
||||
### 2. 缓存优化效果
|
||||
项目中的多层缓存机制带来了显著的性能提升：
|
||||
|
||||
1. **高频问题缓存**: ~0.15ms，提升约35,000倍
|
||||
2. **知识库结果缓存**: ~1-2ms，提升约2,500倍
|
||||
3. **查询改写 + 缓存**: 进一步提升命中率
|
||||
|
||||
### 3. 各层延迟分布
|
||||
|
||||
```
|
||||
真实API调用: ~4.9秒
|
||||
↓
|
||||
知识库缓存: ~1-2ms (提升2500x)
|
||||
↓
|
||||
高频问题缓存: ~0.15ms (提升35000x)
|
||||
```
|
||||
|
||||
## 测试文件
|
||||
|
||||
本次测试使用的文件：
|
||||
1. `viking_retrieval_performance.js` - 完整测试套件
|
||||
2. `viking_retrieval_performance_with_mock.js` - 带模拟模式的测试
|
||||
3. `test_real_viking_kb.js` - 真实知识库测试
|
||||
4. `test_viking_cold_start.js` - 冷启动测试
|
||||
5. `test_viking_direct_api.js` - 直接API测试
|
||||
6. `quick_test_viking.js` - 快速测试脚本
|
||||
7. `run_real_test.js` - 自动检测配置测试
|
||||
|
||||
## 结论
|
||||
|
||||
1. **原始Viking API延迟**: 约4-9秒
|
||||
2. **缓存优化效果显著**: 多层缓存可将延迟降低到毫秒级
|
||||
3. **查询改写机制**: 有效提升缓存命中率
|
||||
4. **推荐配置**:
|
||||
- 保持当前的缓存策略
|
||||
- 考虑增加高频问题的覆盖范围
|
||||
- 监控P95延迟，优化长尾请求
|
||||
|
||||
## 使用方法
|
||||
|
||||
```bash
|
||||
cd test2/server
|
||||
|
||||
# 运行完整测试（模拟模式）
|
||||
node tests/viking_retrieval_performance_with_mock.js
|
||||
|
||||
# 运行真实测试
|
||||
node tests/test_viking_direct_api.js
|
||||
|
||||
# 快速测试
|
||||
node tests/quick_test_viking.js
|
||||
```
|
||||
95
test2/server/tests/_deploy_kb_prompt.cjs
Normal file
95
test2/server/tests/_deploy_kb_prompt.cjs
Normal file
@@ -0,0 +1,95 @@
|
||||
const { Client } = require('ssh2');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const SERVER = { host: '119.45.10.34', port: 22, username: 'root', password: '#xyzh%CS#2512@28' };
|
||||
const REMOTE_DIR = '/www/wwwroot/demo.tensorgrove.com.cn/server';
|
||||
|
||||
const FILES = [
|
||||
{
|
||||
local: path.join(__dirname, '..', 'services', 'assistantProfileConfig.js'),
|
||||
remote: REMOTE_DIR + '/services/assistantProfileConfig.js',
|
||||
},
|
||||
];
|
||||
|
||||
function sshExec(client, cmd) {
|
||||
return new Promise((resolve, reject) => {
|
||||
client.exec(cmd, (err, stream) => {
|
||||
if (err) return reject(err);
|
||||
let out = '', errOut = '';
|
||||
stream.on('data', (d) => (out += d.toString()));
|
||||
stream.stderr.on('data', (d) => (errOut += d.toString()));
|
||||
stream.on('close', (code) => resolve({ out: out.trim(), err: errOut.trim(), code }));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function sshUpload(client, localPath, remotePath) {
|
||||
return new Promise((resolve, reject) => {
|
||||
client.sftp((err, sftp) => {
|
||||
if (err) return reject(err);
|
||||
sftp.fastPut(localPath, remotePath, (err) => {
|
||||
if (err) return reject(err);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function deploy() {
|
||||
const client = new Client();
|
||||
client.on('ready', async () => {
|
||||
try {
|
||||
console.log('✅ SSH connected\n');
|
||||
const ts = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19);
|
||||
|
||||
// 1. 先查看服务器当前prompt
|
||||
console.log('📋 服务器当前prompt:');
|
||||
const before = await sshExec(client, `grep -n 'buildKnowledgeAnswerPrompt\\|知识库涵盖\\|产品用法' ${REMOTE_DIR}/services/assistantProfileConfig.js | head -5`);
|
||||
console.log(' ' + (before.out || '(not found)') + '\n');
|
||||
|
||||
// 2. 备份 + 上传
|
||||
for (const { local, remote } of FILES) {
|
||||
const name = path.basename(remote);
|
||||
await sshExec(client, `cp ${remote} ${remote}.bak_${ts}`);
|
||||
console.log('📦 Backup: ' + name);
|
||||
await sshUpload(client, local, remote);
|
||||
console.log('📤 Uploaded: ' + name);
|
||||
const syntax = await sshExec(client, `node -c ${remote}`);
|
||||
if (syntax.code !== 0) {
|
||||
console.error('❌ Syntax error in ' + name + '! Rolling back...');
|
||||
await sshExec(client, `cp ${remote}.bak_${ts} ${remote}`);
|
||||
client.end();
|
||||
return;
|
||||
}
|
||||
console.log('🔍 Syntax OK: ' + name + '\n');
|
||||
}
|
||||
|
||||
// 3. 重启PM2
|
||||
const pm2Result = await sshExec(client, `cd ${REMOTE_DIR} && pm2 restart all --update-env`);
|
||||
console.log('🔄 PM2 restarted');
|
||||
if (pm2Result.out) console.log(' ' + pm2Result.out.split('\n').slice(0, 3).join('\n '));
|
||||
await new Promise(r => setTimeout(r, 5000));
|
||||
|
||||
// 4. 验证新prompt已生效
|
||||
console.log('\n📋 服务器新prompt:');
|
||||
const after = await sshExec(client, `grep -n '知识库涵盖\\|产品常有别名\\|不得编造' ${REMOTE_DIR}/services/assistantProfileConfig.js | head -5`);
|
||||
console.log(' ' + (after.out || '(not found)'));
|
||||
|
||||
// 5. PM2 状态
|
||||
const status = await sshExec(client, 'pm2 status');
|
||||
console.log('\n📊 PM2 Status:');
|
||||
console.log(' ' + status.out.split('\n').slice(0, 6).join('\n '));
|
||||
|
||||
console.log('\n✅ 部署完成！KB answer prompt 已更新为优化版本');
|
||||
} catch (e) {
|
||||
console.error('❌ Error:', e.message);
|
||||
} finally {
|
||||
client.end();
|
||||
}
|
||||
});
|
||||
client.on('error', (err) => console.error('❌ SSH Error:', err.message));
|
||||
client.connect(SERVER);
|
||||
}
|
||||
|
||||
deploy();
|
||||
70
test2/server/tests/_test_single.js
Normal file
70
test2/server/tests/_test_single.js
Normal file
@@ -0,0 +1,70 @@
|
||||
const path = require('path');
|
||||
const https = require('https');
|
||||
const fs = require('fs');
|
||||
|
||||
// 加载 .env
|
||||
const envPath = path.join(__dirname, '../.env');
|
||||
if (fs.existsSync(envPath)) {
|
||||
fs.readFileSync(envPath, 'utf8').split('\n').forEach(line => {
|
||||
const t = line.trim();
|
||||
if (!t || t.startsWith('#')) return;
|
||||
const i = t.indexOf('=');
|
||||
if (i > 0) {
|
||||
const k = t.slice(0, i).trim();
|
||||
let v = t.slice(i + 1).trim();
|
||||
if ((v.startsWith('"') && v.endsWith('"')) || (v.startsWith("'") && v.endsWith("'")))
|
||||
v = v.slice(1, -1);
|
||||
if (!process.env[k]) process.env[k] = v;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
const PROMPT = '你是大沃的智能助手，负责回答与PM-FitLine德国产品相关的问题。知识库涵盖近50款PM-FitLine产品的完整资料（成分、用法、剂量、价格、规格、搭配方案、好转反应等）及117个常见问答。回答规则：产品相关具体信息必须严格依据知识库，不得猜测或自行补充；公司背景、健康常识可适当补充。产品常有别名（小红=艾特维、大白=倍适、小白=维适多等），请注意识别。不得编造产品名或数据。PM是营养品非药物，涉及疾病建议咨询医生。若知识库无相关内容，坦诚说明并建议咨询推荐人。回答口语化、简洁，1-3句给结论，150字内。';
|
||||
|
||||
const QUERY = process.argv[2] || '一成系统 赋能 德国PM';
|
||||
|
||||
const m = process.env.VOLC_ARK_KB_MODEL || process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID;
|
||||
const ak = process.env.VOLC_ARK_API_KEY || process.env.VOLC_ACCESS_KEY_ID;
|
||||
const ids = (process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS || '').split(',').map(s => s.trim()).filter(Boolean);
|
||||
const th = parseFloat(process.env.VOLC_ARK_KNOWLEDGE_THRESHOLD) || 0.3;
|
||||
|
||||
const body = JSON.stringify({
|
||||
model: m,
|
||||
messages: [
|
||||
{ role: 'system', content: PROMPT },
|
||||
{ role: 'user', content: QUERY },
|
||||
],
|
||||
metadata: { knowledge_base: { dataset_ids: ids, top_k: 3, threshold: th } },
|
||||
stream: false,
|
||||
max_tokens: 80,
|
||||
thinking: { type: 'disabled' },
|
||||
});
|
||||
|
||||
const t0 = Date.now();
|
||||
const req = https.request({
|
||||
hostname: 'ark.cn-beijing.volces.com',
|
||||
path: '/api/v3/chat/completions',
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${ak}`,
|
||||
'Content-Length': Buffer.byteLength(body),
|
||||
},
|
||||
timeout: 15000,
|
||||
}, (res) => {
|
||||
let chunks = [];
|
||||
res.on('data', c => chunks.push(c));
|
||||
res.on('end', () => {
|
||||
const j = JSON.parse(Buffer.concat(chunks).toString());
|
||||
const content = j.choices?.[0]?.message?.content || '(empty)';
|
||||
const u = j.usage || {};
|
||||
const ms = Date.now() - t0;
|
||||
const isNoHit = /未找到|未提及|暂无|没有相关|无法|不确定/.test(content);
|
||||
console.log(`\n检索词: "${QUERY}"`);
|
||||
console.log(`延迟: ${ms}ms | tokens: ${u.prompt_tokens}->${u.completion_tokens} | ${isNoHit ? '❌NO-HIT' : '✅HIT'}`);
|
||||
console.log(`\n回答:\n${content}\n`);
|
||||
});
|
||||
});
|
||||
req.on('error', e => console.error('ERROR:', e.message));
|
||||
req.write(body);
|
||||
req.end();
|
||||
59
test2/server/tests/quick_test_viking.js
Normal file
59
test2/server/tests/quick_test_viking.js
Normal file
@@ -0,0 +1,59 @@
|
||||
const path = require('path');
|
||||
const VikingRetrievalPerformanceTester = require('./viking_retrieval_performance');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const testType = args[0] || 'full';
|
||||
|
||||
const tester = new VikingRetrievalPerformanceTester();
|
||||
|
||||
const testQueries = [
|
||||
{ name: 'Product Query - Xiaohong', query: '小红产品有什么功效' },
|
||||
{ name: 'Product Query - Dabai', query: '大白产品怎么吃' },
|
||||
{ name: 'Company Info', query: '德国PM公司介绍' },
|
||||
{ name: 'NTC Technology', query: 'NTC营养保送系统原理' },
|
||||
{ name: 'Hot Answer', query: '基础三合一怎么吃' },
|
||||
{ name: 'No Hit Query', query: '今天天气怎么样' }
|
||||
];
|
||||
|
||||
switch (testType) {
|
||||
case 'latency':
|
||||
console.log('Running latency test only...');
|
||||
await tester.warmup(testQueries.map(q => q.query));
|
||||
await tester.testLatency(testQueries, 5);
|
||||
tester.printSummary();
|
||||
tester.saveReport('latency_test.json');
|
||||
break;
|
||||
|
||||
case 'cache':
|
||||
console.log('Running cache efficiency test only...');
|
||||
await tester.warmup(testQueries.map(q => q.query));
|
||||
await tester.testCacheEfficiency(testQueries.slice(0, 3), 5);
|
||||
tester.printSummary();
|
||||
tester.saveReport('cache_test.json');
|
||||
break;
|
||||
|
||||
case 'concurrency':
|
||||
console.log('Running concurrency test only...');
|
||||
await tester.warmup(testQueries.map(q => q.query));
|
||||
await tester.testConcurrency(testQueries.slice(0, 3), [1, 2, 3, 5]);
|
||||
tester.printSummary();
|
||||
tester.saveReport('concurrency_test.json');
|
||||
break;
|
||||
|
||||
case 'full':
|
||||
default:
|
||||
console.log('Running full performance test suite...');
|
||||
await tester.runFullSuite();
|
||||
break;
|
||||
}
|
||||
|
||||
console.log('\nTest completed!');
|
||||
}
|
||||
|
||||
main().catch(err => {
|
||||
console.error('Test failed:', err);
|
||||
process.exit(1);
|
||||
});
|
||||
68
test2/server/tests/run_real_test.js
Normal file
68
test2/server/tests/run_real_test.js
Normal file
@@ -0,0 +1,68 @@
|
||||
const path = require('path');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
console.log('Checking environment variables...');
|
||||
console.log('VOLC_ARK_API_KEY:', process.env.VOLC_ARK_API_KEY ? 'Set' : 'Not set');
|
||||
console.log('VOLC_ARK_ENDPOINT_ID:', process.env.VOLC_ARK_ENDPOINT_ID);
|
||||
console.log('VOLC_ARK_KNOWLEDGE_BASE_IDS:', process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS);
|
||||
console.log('VOLC_ARK_KNOWLEDGE_ENDPOINT_ID:', process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID);
|
||||
|
||||
const hasRequiredConfig = process.env.VOLC_ARK_API_KEY &&
|
||||
process.env.VOLC_ARK_ENDPOINT_ID &&
|
||||
process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS &&
|
||||
process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID;
|
||||
|
||||
if (hasRequiredConfig) {
|
||||
console.log('\nAll required configs found! Running real test...\n');
|
||||
|
||||
const VikingTester = require('./viking_retrieval_performance_with_mock');
|
||||
|
||||
(async () => {
|
||||
try {
|
||||
const tester = new VikingTester({ mockMode: false });
|
||||
await tester.runFullSuite();
|
||||
} catch (err) {
|
||||
console.error('Test failed:', err);
|
||||
}
|
||||
})();
|
||||
} else {
|
||||
console.log('\nMissing required environment variables. Checking parent directory...');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../../.env') });
|
||||
|
||||
console.log('VOLC_ARK_API_KEY (parent):', process.env.VOLC_ARK_API_KEY ? 'Set' : 'Not set');
|
||||
|
||||
const hasRequiredConfigParent = process.env.VOLC_ARK_API_KEY &&
|
||||
process.env.VOLC_ARK_ENDPOINT_ID &&
|
||||
process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS &&
|
||||
process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID;
|
||||
|
||||
if (hasRequiredConfigParent) {
|
||||
console.log('\nConfigs found in parent directory! Running real test...\n');
|
||||
|
||||
const VikingTester = require('./viking_retrieval_performance_with_mock');
|
||||
|
||||
(async () => {
|
||||
try {
|
||||
const tester = new VikingTester({ mockMode: false });
|
||||
await tester.runFullSuite();
|
||||
} catch (err) {
|
||||
console.error('Test failed:', err);
|
||||
}
|
||||
})();
|
||||
} else {
|
||||
console.log('\nNo configs found. Running mock test instead...\n');
|
||||
|
||||
const VikingTester = require('./viking_retrieval_performance_with_mock');
|
||||
|
||||
(async () => {
|
||||
try {
|
||||
const tester = new VikingTester({ mockMode: true });
|
||||
await tester.runFullSuite();
|
||||
} catch (err) {
|
||||
console.error('Test failed:', err);
|
||||
}
|
||||
})();
|
||||
}
|
||||
}
|
||||
520
test2/server/tests/test_asr_coverage.js
Normal file
520
test2/server/tests/test_asr_coverage.js
Normal file
@@ -0,0 +1,520 @@
|
||||
/**
|
||||
* 语音识别(ASR)纠错全覆盖测试
|
||||
* 覆盖：PHRASE_MAP、WORD_MAP、PRODUCT_ALIAS_MAP、激进正则、normalizeKnowledgeAlias、组合流水线
|
||||
*
|
||||
* 运行方式: node --test tests/test_asr_coverage.js
|
||||
*/
|
||||
const { describe, it } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const { correctAsrText, PHRASE_MAP, WORD_MAP, PRODUCT_ALIAS_MAP } = require('../services/fastAsrCorrector');
|
||||
const { normalizeKnowledgeAlias, shouldForceKnowledgeRoute } = require('../services/realtimeDialogRouting');
|
||||
const { hasKnowledgeRouteKeyword } = require('../services/knowledgeKeywords');
|
||||
|
||||
// 辅助：完整ASR流水线（模拟nativeVoiceGateway.extractUserText + routing）
|
||||
function fullAsrPipeline(rawText) {
|
||||
const corrected = correctAsrText(rawText);
|
||||
const normalized = normalizeKnowledgeAlias(corrected);
|
||||
return normalized;
|
||||
}
|
||||
|
||||
function assertPipelineRouteKb(rawText, msg) {
|
||||
const processed = fullAsrPipeline(rawText);
|
||||
const result = shouldForceKnowledgeRoute(processed);
|
||||
assert.equal(result, true, msg || `ASR "${rawText}" → processed "${processed}" should route to KB`);
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 1. PHRASE_MAP 全覆盖 —— 每条短语映射逐一验证
|
||||
// ================================================================
|
||||
describe('PHRASE_MAP —— 短语级ASR纠错全覆盖', () => {
|
||||
|
||||
describe('一成系统变体（36条）', () => {
|
||||
const yichengVariants = [
|
||||
'一城系统', '逸城系统', '一程系统', '易成系统', '一诚系统',
|
||||
'亦成系统', '艺成系统', '溢成系统', '义成系统', '毅成系统',
|
||||
'怡成系统', '以成系统', '已成系统', '亿成系统', '忆成系统',
|
||||
'益成系统', '一乘系统', '一承系统', '一丞系统', '一呈系统',
|
||||
'一澄系统', '一橙系统', '一层系统', '一趁系统', '一陈系统',
|
||||
'依成系统', '伊成系统', '益生系统', '易诚系统', '易乘系统',
|
||||
'一声系统', '亿生系统', '义诚系统', '忆诚系统', '以诚系统',
|
||||
];
|
||||
|
||||
for (const variant of yichengVariants) {
|
||||
it(`"${variant}" → "一成系统"`, () => {
|
||||
const result = correctAsrText(variant);
|
||||
assert.ok(result.includes('一成系统'), `"${variant}" should correct to 一成系统, got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('其他短语映射', () => {
|
||||
const otherPhrases = [
|
||||
['盛咖学院', '盛咖学愿'],
|
||||
['圣咖学愿', '盛咖学愿'],
|
||||
['盛卡学愿', '盛咖学愿'],
|
||||
['营养配送系统', 'NTC营养保送系统'],
|
||||
['营养输送系统', 'NTC营养保送系统'],
|
||||
['营养传送系统', 'NTC营养保送系统'],
|
||||
['营养传输系统', 'NTC营养保送系统'],
|
||||
['暖炉原理', '火炉原理'],
|
||||
['整应反应', '好转反应'],
|
||||
['整健反应', '好转反应'],
|
||||
['排毒反应', '好转反应'],
|
||||
['5加1', '5+1'],
|
||||
['五加一', '5+1'],
|
||||
['起步三观', '起步三关'],
|
||||
['起步三官', '起步三关'],
|
||||
['doublepm', '德国PM'],
|
||||
['double pm', '德国PM'],
|
||||
['DoublePM', '德国PM'],
|
||||
['Double PM', '德国PM'],
|
||||
['DOUBLEPM', '德国PM'],
|
||||
['DOUBLE PM', '德国PM'],
|
||||
['基础三合一', 'PM细胞营养素 基础套装'],
|
||||
['三合一基础套', 'PM细胞营养素 基础套装'],
|
||||
['大白小红小白', 'PM细胞营养素 基础套装'],
|
||||
];
|
||||
|
||||
for (const [input, expected] of otherPhrases) {
|
||||
it(`"${input}" → 含"${expected}"`, () => {
|
||||
const result = correctAsrText(input);
|
||||
assert.ok(result.includes(expected), `"${input}" should contain "${expected}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 2. WORD_MAP 全覆盖 —— 单词级ASR纠错
|
||||
// ================================================================
|
||||
describe('WORD_MAP —— 单词级ASR纠错全覆盖', () => {
|
||||
|
||||
describe('一成/一城等同音变体', () => {
|
||||
const wordVariants = [
|
||||
['一城', '一成'], ['逸城', '一成'], ['一程', '一成'], ['易成', '一成'],
|
||||
['一诚', '一成'], ['亦成', '一成'], ['艺成', '一成'], ['溢成', '一成'],
|
||||
['义成', '一成'], ['毅成', '一成'], ['怡成', '一成'], ['以成', '一成'],
|
||||
['已成', '一成'], ['亿成', '一成'], ['忆成', '一成'], ['益成', '一成'],
|
||||
['一乘', '一成'], ['一承', '一成'], ['一丞', '一成'], ['一呈', '一成'],
|
||||
['一澄', '一成'], ['一橙', '一成'], ['一层', '一成'], ['一陈', '一成'],
|
||||
['依成', '一成'], ['伊成', '一成'],
|
||||
['益生', '一成'], ['易诚', '一成'], ['义诚', '一成'], ['忆诚', '一成'], ['以诚', '一成'],
|
||||
['一声', '一成'], ['亿生', '一成'], ['易乘', '一成'],
|
||||
];
|
||||
|
||||
for (const [input, expected] of wordVariants) {
|
||||
it(`"${input}" → "${expected}"`, () => {
|
||||
const result = correctAsrText(`${input}的介绍`);
|
||||
assert.ok(result.includes(expected), `"${input}" should correct to "${expected}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('大沃同音变体', () => {
|
||||
const dawoVariants = ['大窝', '大握', '大我', '大卧'];
|
||||
for (const v of dawoVariants) {
|
||||
it(`"${v}" → "大沃"`, () => {
|
||||
const result = correctAsrText(v);
|
||||
assert.ok(result.includes('大沃'), `"${v}" should correct to 大沃, got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('Ai众享同音变体', () => {
|
||||
const aiVariants = ['爱众享', '艾众享', '哎众享'];
|
||||
for (const v of aiVariants) {
|
||||
it(`"${v}" → "Ai众享"`, () => {
|
||||
const result = correctAsrText(v);
|
||||
assert.ok(result.includes('Ai众享'), `"${v}" should correct to Ai众享, got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品名同音变体', () => {
|
||||
const productVariants = [
|
||||
['小洪', '小红'], ['小宏', '小红'], ['小鸿', '小红'],
|
||||
['大百', '大白'], ['大柏', '大白'],
|
||||
['小百', '小白'], ['小柏', '小白'], ['维适多', '小白'],
|
||||
];
|
||||
for (const [input, expected] of productVariants) {
|
||||
it(`"${input}" → "${expected}"`, () => {
|
||||
const result = correctAsrText(`${input}产品功效`);
|
||||
assert.ok(result.includes(expected), `"${input}" should correct to "${expected}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('其他同音变体', () => {
|
||||
const others = [
|
||||
['营养配送', '营养保送'],
|
||||
['营养输送', '营养保送'],
|
||||
['阿玉吠陀', '阿育吠陀'],
|
||||
['阿育费陀', '阿育吠陀'],
|
||||
];
|
||||
for (const [input, expected] of others) {
|
||||
it(`"${input}" → "${expected}"`, () => {
|
||||
const result = correctAsrText(input);
|
||||
assert.ok(result.includes(expected), `"${input}" should correct to "${expected}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 3. PRODUCT_ALIAS_MAP —— 产品别名扩展
|
||||
// ================================================================
|
||||
describe('PRODUCT_ALIAS_MAP —— 产品别名扩展', () => {
|
||||
const aliasCases = [
|
||||
['小红怎么吃', 'Activize'],
|
||||
['小红功效', 'Activize'],
|
||||
['Activize是什么', 'Activize Oxyplus'],
|
||||
['大白怎么吃', 'Basics'],
|
||||
['大白功效', 'Basics'],
|
||||
['Basics成分', 'Basics'],
|
||||
['小白怎么吃', 'Restorate'],
|
||||
['Restorate功效', 'Restorate'],
|
||||
['FitLine是什么', 'PM-FitLine'],
|
||||
['PM FitLine的功效', 'PM-FitLine'],
|
||||
['PM细胞营养', 'PM细胞营养素'],
|
||||
['PM营养素功效', 'PM细胞营养素'],
|
||||
['德国PM营养素', 'PM细胞营养素'],
|
||||
];
|
||||
|
||||
for (const [input, expectContain] of aliasCases) {
|
||||
it(`"${input}" → 扩展含"${expectContain}"`, () => {
|
||||
const result = correctAsrText(input);
|
||||
assert.ok(result.includes(expectContain), `"${input}" should expand to contain "${expectContain}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
|
||||
it('非追问位置不应触发产品扩展（如"小红帽"）', () => {
|
||||
const result = correctAsrText('小红帽故事');
|
||||
// 小红帽 doesn't match the expansion pattern because 帽 is not in the suffix list
|
||||
assert.ok(!result.includes('Activize'), `"小红帽故事" should NOT expand 小红, got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 4. 激进正则 —— X+成/城/程...+系统 统一纠正
|
||||
// ================================================================
|
||||
describe('激进正则 —— 未在字典中的"X成系统"变体', () => {
|
||||
const aggressiveCases = [
|
||||
'翼成系统', '奕成系统', '弈成系统', '颐成系统',
|
||||
'译成系统', '蚁成系统', '壹成系统',
|
||||
'一盛系统', '一胜系统', '一生系统',
|
||||
'一称系统', '一撑系统',
|
||||
'双成系统', '半成系统',
|
||||
];
|
||||
|
||||
for (const variant of aggressiveCases) {
|
||||
it(`"${variant}" → "一成系统"（激进正则兜底）`, () => {
|
||||
const result = correctAsrText(variant);
|
||||
assert.ok(result.includes('一成系统'), `"${variant}" should correct to 一成系统 via aggressive regex, got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 5. normalizeKnowledgeAlias —— 路由层额外归一化
|
||||
// ================================================================
|
||||
describe('normalizeKnowledgeAlias —— 路由层归一化', () => {
|
||||
it('一成，系统（带标点间隔）→ 一成系统', () => {
|
||||
const result = normalizeKnowledgeAlias('一成，系统');
|
||||
assert.ok(result.includes('一成系统'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('一成 系统（带空格间隔）→ 一成系统', () => {
|
||||
const result = normalizeKnowledgeAlias('一成 系统');
|
||||
assert.ok(result.includes('一成系统'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('XX系统 → 一成系统', () => {
|
||||
const result = normalizeKnowledgeAlias('XX系统');
|
||||
assert.ok(result.includes('一成系统'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('大窝 → 大沃', () => {
|
||||
const result = normalizeKnowledgeAlias('大窝');
|
||||
assert.ok(result.includes('大沃'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('暖炉原理 → 火炉原理', () => {
|
||||
const result = normalizeKnowledgeAlias('暖炉原理');
|
||||
assert.ok(result.includes('火炉原理'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('AI众享（大写）→ Ai众享', () => {
|
||||
const result = normalizeKnowledgeAlias('AI众享怎么用');
|
||||
assert.ok(result.includes('Ai众享'), `Got "${result}"`);
|
||||
});
|
||||
|
||||
it('圣咖学院 → 盛咖学愿', () => {
|
||||
const result = normalizeKnowledgeAlias('圣咖学院');
|
||||
assert.ok(result.includes('盛咖学愿'), `Got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 6. 完整ASR流水线 —— correctAsrText + normalizeKnowledgeAlias 组合
|
||||
// ================================================================
|
||||
describe('完整ASR流水线 —— 纠错+归一化组合', () => {
|
||||
|
||||
describe('一成系统变体经流水线后应路由到KB', () => {
|
||||
const pipelineCases = [
|
||||
'一城系统是什么',
|
||||
'逸城系统怎么用',
|
||||
'易成系统介绍',
|
||||
'益生系统怎么样',
|
||||
'义诚系统核心优势',
|
||||
'壹成系统三大平台',
|
||||
'一声系统有什么用',
|
||||
'翼成系统赋能团队',
|
||||
];
|
||||
for (const raw of pipelineCases) {
|
||||
it(`"${raw}" → 应路由到KB`, () => {
|
||||
assertPipelineRouteKb(raw);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品名ASR错误经流水线后应路由到KB', () => {
|
||||
const productAsrCases = [
|
||||
['小洪产品功效', '小红ASR错误'],
|
||||
['小宏怎么吃', '小红ASR错误'],
|
||||
['大百功效是什么', '大白ASR错误'],
|
||||
['大柏怎么吃', '大白ASR错误'],
|
||||
['小百怎么服用', '小白ASR错误'],
|
||||
['小柏功效', '小白ASR错误'],
|
||||
];
|
||||
for (const [raw, label] of productAsrCases) {
|
||||
it(`${label}: "${raw}" → 应路由到KB`, () => {
|
||||
assertPipelineRouteKb(raw);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('其他ASR错误经流水线后应路由到KB', () => {
|
||||
const otherAsrCases = [
|
||||
['暖炉原理是什么意思', '暖炉→火炉'],
|
||||
['营养配送系统原理', '配送→保送'],
|
||||
['整应反应是什么', '整应→好转'],
|
||||
['排毒反应正常吗', '排毒反应→好转反应'],
|
||||
['盛咖学院怎么用', '学院→学愿'],
|
||||
['起步三观是什么', '三观→三关'],
|
||||
['double pm介绍', 'double pm→德国PM'],
|
||||
['阿玉吠陀是什么', '阿玉→阿育'],
|
||||
];
|
||||
for (const [raw, label] of otherAsrCases) {
|
||||
it(`${label}: "${raw}" → 应路由到KB`, () => {
|
||||
assertPipelineRouteKb(raw);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 7. ASR识别失败/乱码/噪声场景
|
||||
// ================================================================
|
||||
describe('ASR识别失败/异常场景', () => {
|
||||
|
||||
it('空字符串 → 不崩溃，返回空', () => {
|
||||
assert.equal(correctAsrText(''), '');
|
||||
assert.equal(correctAsrText(null), '');
|
||||
assert.equal(correctAsrText(undefined), '');
|
||||
});
|
||||
|
||||
it('纯噪声标点 → 不崩溃', () => {
|
||||
const result = correctAsrText('，，，。。。！！');
|
||||
assert.equal(typeof result, 'string');
|
||||
});
|
||||
|
||||
it('语气词噪声 → 不应路由到KB', () => {
|
||||
const noises = ['嗯嗯嗯', '啊啊啊', '哦哦哦', '额额额'];
|
||||
for (const noise of noises) {
|
||||
const processed = fullAsrPipeline(noise);
|
||||
const result = shouldForceKnowledgeRoute(processed);
|
||||
assert.equal(result, false, `Noise "${noise}" should NOT route to KB`);
|
||||
}
|
||||
});
|
||||
|
||||
it('极短识别（单字/双字）→ 不应路由到KB', () => {
|
||||
const shorts = ['嗯', '好', '啊', '是', '对', '哦'];
|
||||
for (const s of shorts) {
|
||||
const processed = fullAsrPipeline(s);
|
||||
const result = shouldForceKnowledgeRoute(processed);
|
||||
assert.equal(result, false, `Short "${s}" should NOT route to KB`);
|
||||
}
|
||||
});
|
||||
|
||||
it('混合中英文乱码 → 不崩溃，不误触发KB', () => {
|
||||
const garbled = ['abc123你好', 'test test', '!!!???', '😊😊😊'];
|
||||
for (const g of garbled) {
|
||||
const processed = fullAsrPipeline(g);
|
||||
assert.equal(typeof processed, 'string', 'Should return string');
|
||||
}
|
||||
});
|
||||
|
||||
it('超长ASR文本 → 不崩溃', () => {
|
||||
const longText = '我想问一下关于'.repeat(50) + '基础三合一怎么吃';
|
||||
const result = correctAsrText(longText);
|
||||
assert.equal(typeof result, 'string');
|
||||
assert.ok(result.length > 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 8. ASR部分识别 —— 语音被截断的情况
|
||||
// ================================================================
|
||||
describe('ASR部分识别 —— 语音截断/不完整', () => {
|
||||
|
||||
it('"一成系" → 不完整但不崩溃', () => {
|
||||
const result = correctAsrText('一成系');
|
||||
assert.equal(typeof result, 'string');
|
||||
});
|
||||
|
||||
it('"基础三合" → 不完整，不应匹配PHRASE_MAP', () => {
|
||||
const result = correctAsrText('基础三合');
|
||||
assert.ok(!result.includes('基础套装'), `Incomplete "基础三合" should not trigger full phrase mapping, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"小红怎" → 不完整但产品名应被扩展', () => {
|
||||
const result = correctAsrText('小红怎');
|
||||
// 小红 后面是 怎，在suffix list里有 怎么 但没有单独的 怎
|
||||
assert.equal(typeof result, 'string');
|
||||
});
|
||||
|
||||
it('"德国P" → 不完整，不应触发', () => {
|
||||
const result = correctAsrText('德国P');
|
||||
assert.ok(!result.includes('德国PM公司'), `Incomplete should not over-expand, got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 9. 复合ASR错误 —— 一句话里有多个ASR错误
|
||||
// ================================================================
|
||||
describe('复合ASR错误 —— 一句话中包含多个识别错误', () => {
|
||||
|
||||
it('"一城系统的小洪产品功效" → 一成系统 + 小红', () => {
|
||||
const result = correctAsrText('一城系统的小洪产品功效');
|
||||
assert.ok(result.includes('一成系统'), `Should correct 一城→一成, got "${result}"`);
|
||||
assert.ok(result.includes('小红'), `Should correct 小洪→小红, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"大百和小柏的区别" → 大白 + 小白', () => {
|
||||
const result = correctAsrText('大百和小柏的区别');
|
||||
assert.ok(result.includes('大白'), `Should correct 大百→大白, got "${result}"`);
|
||||
assert.ok(result.includes('小白'), `Should correct 小柏→小白, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"爱众享和盛咖学院" → Ai众享 + 盛咖学愿', () => {
|
||||
const result = correctAsrText('爱众享和盛咖学院');
|
||||
assert.ok(result.includes('Ai众享'), `Should correct 爱众享→Ai众享, got "${result}"`);
|
||||
assert.ok(result.includes('盛咖学愿'), `Should correct 盛咖学院→盛咖学愿, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"大窝的暖炉原理" → 大沃 + 火炉原理', () => {
|
||||
const result = correctAsrText('大窝的暖炉原理');
|
||||
assert.ok(result.includes('大沃'), `Should correct 大窝→大沃, got "${result}"`);
|
||||
assert.ok(result.includes('火炉原理'), `Should correct 暖炉→火炉, got "${result}"`);
|
||||
});
|
||||
|
||||
it('复合错误经完整流水线后应路由到KB', () => {
|
||||
assertPipelineRouteKb('一城系统的小洪怎么吃');
|
||||
assertPipelineRouteKb('大百和小柏的区别是什么');
|
||||
assertPipelineRouteKb('爱众享和盛咖学院介绍');
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 10. 真实语音场景模拟 —— 模拟用户真实说话方式的ASR输出
|
||||
// ================================================================
|
||||
describe('真实语音场景 —— 模拟实际用户说话的ASR识别结果', () => {
|
||||
|
||||
it('"那个一城系统是干嘛的呀" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('那个一城系统是干嘛的呀');
|
||||
});
|
||||
|
||||
it('"小洪和大百一起吃吗" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('小洪和大百一起吃吗');
|
||||
});
|
||||
|
||||
it('"我想问一下那个暖炉原理是什么意思" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('我想问一下那个暖炉原理是什么意思');
|
||||
});
|
||||
|
||||
it('"你们那个double pm是什么公司" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('你们那个double pm是什么公司');
|
||||
});
|
||||
|
||||
it('"吃了以后有整应反应怎么办" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('吃了以后有整应反应怎么办');
|
||||
});
|
||||
|
||||
it('"盛咖学院里面的课程怎么看" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('盛咖学院里面的课程怎么看');
|
||||
});
|
||||
|
||||
it('"那个营养配送系统是怎么回事" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('那个营养配送系统是怎么回事');
|
||||
});
|
||||
|
||||
it('"新人起步三观是什么" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('新人起步三观是什么');
|
||||
});
|
||||
|
||||
it('"维适多怎么服用" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('维适多怎么服用');
|
||||
});
|
||||
|
||||
it('"大窝能帮我介绍一下大百吗" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('大窝能帮我介绍一下大百吗');
|
||||
});
|
||||
|
||||
it('"五加一活动是什么" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('五加一活动是什么');
|
||||
});
|
||||
|
||||
it('"阿育费陀跟PM产品有什么关系" → 应路由到KB', () => {
|
||||
assertPipelineRouteKb('阿育费陀跟PM产品有什么关系');
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 11. 负面用例 —— 正常文本不应被ASR纠错误改
|
||||
// ================================================================
|
||||
describe('负面用例 —— 正常文本不应被误纠', () => {
|
||||
|
||||
it('"一成不变" → 不应被改为"一成系统不变"', () => {
|
||||
const result = correctAsrText('一成不变');
|
||||
assert.ok(!result.includes('一成系统'), `"一成不变" should NOT be corrected, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"今天天气好" → 保持不变', () => {
|
||||
const result = correctAsrText('今天天气好');
|
||||
assert.equal(result, '今天天气好');
|
||||
});
|
||||
|
||||
it('"你好" → 保持不变', () => {
|
||||
const result = correctAsrText('你好');
|
||||
assert.equal(result, '你好');
|
||||
});
|
||||
|
||||
it('"谢谢你帮忙" → 保持不变', () => {
|
||||
const result = correctAsrText('谢谢你帮忙');
|
||||
assert.equal(result, '谢谢你帮忙');
|
||||
});
|
||||
|
||||
it('"大白天出去" → 不应被纠正为PM产品', () => {
|
||||
// "大白" 后面跟 "天" 不在suffix list，不应触发产品扩展
|
||||
const result = correctAsrText('大白天出去');
|
||||
assert.ok(!result.includes('Basics'), `"大白天出去" should NOT trigger product alias, got "${result}"`);
|
||||
});
|
||||
|
||||
it('"小白兔" → 不应被纠正为PM产品', () => {
|
||||
const result = correctAsrText('小白兔');
|
||||
assert.ok(!result.includes('Restorate'), `"小白兔" should NOT trigger product alias, got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
console.log('\n=== ASR覆盖测试加载完成 ===\n');
|
||||
241
test2/server/tests/test_kb_prompt_compare.js
Normal file
241
test2/server/tests/test_kb_prompt_compare.js
Normal file
@@ -0,0 +1,241 @@
|
||||
/**
|
||||
* KB Prompt A/B 对比测试
|
||||
* 用失败用例验证优化后的prompt效果
|
||||
* 零依赖：仅用Node内置模块
|
||||
*/
|
||||
const path = require('path');
|
||||
const https = require('https');
|
||||
const fs = require('fs');
|
||||
|
||||
// 手动加载 .env
|
||||
const envPath = path.join(__dirname, '../.env');
|
||||
if (fs.existsSync(envPath)) {
|
||||
fs.readFileSync(envPath, 'utf8').split('\n').forEach(line => {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith('#')) return;
|
||||
const idx = trimmed.indexOf('=');
|
||||
if (idx > 0) {
|
||||
const key = trimmed.slice(0, idx).trim();
|
||||
let val = trimmed.slice(idx + 1).trim();
|
||||
if ((val.startsWith('"') && val.endsWith('"')) || (val.startsWith("'") && val.endsWith("'"))) {
|
||||
val = val.slice(1, -1);
|
||||
}
|
||||
if (!process.env[key]) process.env[key] = val;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
const kbHttpAgent = new https.Agent({
|
||||
keepAlive: true,
|
||||
keepAliveMsecs: 30000,
|
||||
maxSockets: 6,
|
||||
timeout: 15000,
|
||||
});
|
||||
|
||||
// ===== 旧prompt =====
|
||||
const OLD_PROMPT = '你是大沃的智能助手，负责回答与PM-FitLine德国产品相关的问题。产品用法、成分、剂量、价格等专业信息必须依据知识库，不得自行补充或猜测；公司背景、理念等常识性内容可适当补充。不得编造不存在的产品名称或数据。若知识库无相关内容，坦诚说明并建议查看产品说明书或咨询推荐人。回答口语化、简洁、专业，1-3句给结论，150字内。';
|
||||
|
||||
// ===== 新prompt =====
|
||||
const NEW_PROMPT = '你是大沃的智能助手，负责回答与PM-FitLine德国产品相关的问题。知识库涵盖近50款PM-FitLine产品的完整资料（成分、用法、剂量、价格、规格、搭配方案、好转反应等）及117个常见问答。回答规则：产品相关具体信息必须严格依据知识库，不得猜测或自行补充；公司背景、健康常识可适当补充。产品常有别名（小红=艾特维、大白=倍适、小白=维适多等），请注意识别。不得编造产品名或数据。PM是营养品非药物，涉及疾病建议咨询医生。若知识库无相关内容，坦诚说明并建议咨询推荐人。回答口语化、简洁，1-3句给结论，150字内。';
|
||||
|
||||
// ===== 全面测试用例：10个维度 =====
|
||||
const TEST_CASES = [
|
||||
// ── 1. 产品用法（营养品） ──
|
||||
{ cat: '用法', name: '细胞抗氧素怎么吃(原失败)', query: 'Apple Antioxy Zellschutz 细胞抗氧素 怎么吃', original: '细胞抗氧素怎么吃', expectHit: true },
|
||||
{ cat: '用法', name: '小红怎么喝', query: 'Activize Oxyplus 艾特维 小红 怎么喝', original: '小红怎么喝', expectHit: true },
|
||||
{ cat: '用法', name: '小白什么时候喝', query: 'Restorate 维适多 小白 什么时候喝 服用方法', original: '小白什么时候喝', expectHit: true },
|
||||
{ cat: '用法', name: 'Q10怎么用', query: 'Q10辅酵素 怎么用 用法', original: 'Q10怎么用', expectHit: true },
|
||||
{ cat: '用法', name: '心脏衰竭者小红用量', query: 'PM-Fitline 心脏衰竭 小红 用量 怎么喝', original: '心脏衰竭的人小红怎么喝', expectHit: true },
|
||||
|
||||
// ── 2. 产品价格 ──
|
||||
{ cat: '价格', name: '大白多少钱', query: 'Basics 倍适 大白 价格 多少钱', original: '大白多少钱', expectHit: true },
|
||||
{ cat: '价格', name: '小绿排毒饮价格', query: 'D-Drink 小绿 排毒饮 价格 多少钱', original: '小绿多少钱', expectHit: true },
|
||||
{ cat: '价格', name: '祛皱凝胶价格', query: 'Ultimate Young 三分钟瞬间祛皱凝胶 价格', original: '祛皱凝胶多少钱', expectHit: true },
|
||||
|
||||
// ── 3. 产品成分 ──
|
||||
{ cat: '成分', name: '小红里有什么成分', query: 'Activize Oxyplus 艾特维 小红 成分 配方', original: '小红里面有什么成分', expectHit: true },
|
||||
{ cat: '成分', name: '美容饮成分', query: 'Beauty 肽美 美容饮 成分', original: '美容饮有什么成分', expectHit: true },
|
||||
|
||||
// ── 4. 产品搭配 ──
|
||||
{ cat: '搭配', name: '基础三合一是什么', query: '德国PM细胞营养素 基础套装 大白 小红 小白', original: '基础三合一是什么', expectHit: true },
|
||||
{ cat: '搭配', name: '减肥搭配方案', query: 'TopShape ProShape 减肥 搭配 方案', original: '想减肥应该吃什么搭配', expectHit: true },
|
||||
|
||||
// ── 5. 好转反应 ──
|
||||
{ cat: '好转反应', name: '为什么会痒', query: 'PM-Fitline 好转反应 痒 发红', original: '喝了PM为什么会痒', expectHit: true },
|
||||
{ cat: '好转反应', name: '喝了头晕怎么回事', query: 'PM-Fitline 好转反应 头晕 目眩', original: '喝完营养素头晕怎么回事', expectHit: true },
|
||||
{ cat: '好转反应', name: '为什么喝小红胃痛', query: 'PM-Fitline 小红 艾特维 胃痛 好转反应', original: '为什么喝小红胃会痛', expectHit: true },
|
||||
|
||||
// ── 6. 别名识别 ──
|
||||
{ cat: '别名', name: '小黑是什么产品', query: 'MEN+ 倍力健 小黑 产品介绍', original: '小黑是什么', expectHit: true },
|
||||
{ cat: '别名', name: '乐活50+适合谁', query: 'Generation 50+ 乐活50+ 适合谁 适用人群', original: '乐活50+适合谁吃', expectHit: true },
|
||||
|
||||
// ── 7. 护肤品 ──
|
||||
{ cat: '护肤', name: '眼霜怎么用', query: 'Eye Care 全效眼霜 用法 怎么用', original: '眼霜怎么用', expectHit: true },
|
||||
{ cat: '护肤', name: '面膜多久敷一次', query: 'Hydrating-Shot Mask 靓白保湿面膜 多久 频率', original: '面膜多久敷一次', expectHit: true },
|
||||
|
||||
// ── 8. 117问答/疾病相关 ──
|
||||
{ cat: 'Q&A', name: '高血压能吃PM吗', query: 'PM-Fitline 高血压 能吃吗 影响', original: '高血压能吃PM吗', expectHit: true },
|
||||
{ cat: 'Q&A', name: '糖尿病能减药吗', query: 'PM-Fitline 糖尿病 减药 停药', original: '吃PM后糖尿病的药能减吗', expectHit: true },
|
||||
{ cat: 'Q&A', name: '小红咖啡因和咖啡一样吗', query: 'PM-Fitline 小红 艾特维 咖啡因 瓜拉纳 区别', original: '小红里的咖啡因跟咖啡一样吗', expectHit: true },
|
||||
|
||||
// ── 9. 边界测试（KB中不存在的信息） ──
|
||||
{ cat: '边界', name: '不存在的产品', query: 'PM-Fitline 超级修复胶囊', original: 'PM有超级修复胶囊这个产品吗', expectHit: false },
|
||||
{ cat: '边界', name: '跟主题无关的问题', query: '今天天气怎么样', original: '今天天气怎么样', expectHit: false },
|
||||
|
||||
// ── 10. 公司/系统 ──
|
||||
{ cat: '公司', name: 'PM是什么公司', query: 'PM-International 德国PM公司 介绍', original: 'PM是什么公司', expectHit: true },
|
||||
|
||||
// ── 11. 一成系统 ──
|
||||
{ cat: '一成系统', name: '一成系统如何赋能德国PM', query: '一成系统 赋能 德国PM', original: '一成系统如何赋能德国PM', expectHit: true },
|
||||
];
|
||||
|
||||
function httpsPost(url, body, headers) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const data = JSON.stringify(body);
|
||||
const parsed = new URL(url);
|
||||
const req = https.request({
|
||||
hostname: parsed.hostname,
|
||||
path: parsed.pathname,
|
||||
method: 'POST',
|
||||
headers: { ...headers, 'Content-Length': Buffer.byteLength(data) },
|
||||
agent: kbHttpAgent,
|
||||
timeout: 15000,
|
||||
}, (res) => {
|
||||
let chunks = [];
|
||||
res.on('data', c => chunks.push(c));
|
||||
res.on('end', () => {
|
||||
try {
|
||||
resolve(JSON.parse(Buffer.concat(chunks).toString()));
|
||||
} catch (e) {
|
||||
reject(new Error('JSON parse error: ' + Buffer.concat(chunks).toString().slice(0, 200)));
|
||||
}
|
||||
});
|
||||
});
|
||||
req.on('error', reject);
|
||||
req.on('timeout', () => { req.destroy(); reject(new Error('timeout')); });
|
||||
req.write(data);
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
async function callKB(systemPrompt, query, label) {
|
||||
const endpointId = process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID || process.env.VOLC_ARK_ENDPOINT_ID;
|
||||
const kbModel = process.env.VOLC_ARK_KB_MODEL || endpointId;
|
||||
const authKey = process.env.VOLC_ARK_API_KEY || process.env.VOLC_ACCESS_KEY_ID;
|
||||
const kbIds = (process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS || '').split(',').map(id => id.trim()).filter(Boolean);
|
||||
const topK = parseInt(process.env.VOLC_ARK_KNOWLEDGE_TOP_K) || 3;
|
||||
const threshold = parseFloat(process.env.VOLC_ARK_KNOWLEDGE_THRESHOLD) || 0.3;
|
||||
|
||||
const body = {
|
||||
model: kbModel,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: query },
|
||||
],
|
||||
metadata: {
|
||||
knowledge_base: {
|
||||
dataset_ids: kbIds,
|
||||
top_k: topK,
|
||||
threshold: threshold,
|
||||
},
|
||||
},
|
||||
stream: false,
|
||||
max_tokens: 80,
|
||||
thinking: { type: 'disabled' },
|
||||
};
|
||||
|
||||
const start = Date.now();
|
||||
try {
|
||||
const res = await httpsPost(
|
||||
'https://ark.cn-beijing.volces.com/api/v3/chat/completions',
|
||||
body,
|
||||
{
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${authKey}`,
|
||||
}
|
||||
);
|
||||
const elapsed = Date.now() - start;
|
||||
const content = res?.choices?.[0]?.message?.content || '(empty)';
|
||||
const usage = res?.usage || {};
|
||||
const isNoHit = /未找到|未提及|暂无|没有相关|无法|不确定/.test(content);
|
||||
return { label, content, elapsed, usage, hit: !isNoHit };
|
||||
} catch (err) {
|
||||
return { label, content: `ERROR: ${err.message}`, elapsed: Date.now() - start, usage: {}, hit: false };
|
||||
}
|
||||
}
|
||||
|
||||
async function runTest() {
|
||||
console.log('============================================================');
|
||||
console.log(' KB 新Prompt 全面对话测试（10维度 x 25用例）');
|
||||
console.log('============================================================\n');
|
||||
|
||||
const kbModel = process.env.VOLC_ARK_KB_MODEL || process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID || '(not set)';
|
||||
const threshold = process.env.VOLC_ARK_KNOWLEDGE_THRESHOLD || '0.3';
|
||||
console.log(`模型: ${kbModel} | 阈值: ${threshold} | 用例数: ${TEST_CASES.length}\n`);
|
||||
|
||||
const results = [];
|
||||
const catStats = {};
|
||||
|
||||
for (let i = 0; i < TEST_CASES.length; i++) {
|
||||
const tc = TEST_CASES[i];
|
||||
console.log(`[${i + 1}/${TEST_CASES.length}] 【${tc.cat}】${tc.name}`);
|
||||
console.log(` 原始问题: "${tc.original}"`);
|
||||
|
||||
const r = await callKB(NEW_PROMPT, tc.query, tc.name);
|
||||
const matchExpect = r.hit === tc.expectHit;
|
||||
|
||||
// 对边界测试特殊处理：no-hit是期望结果
|
||||
const icon = matchExpect ? '✅' : '⚠️';
|
||||
const hitLabel = tc.expectHit
|
||||
? (r.hit ? 'HIT' : 'MISS(应命中)')
|
||||
: (r.hit ? 'HIT(应未命中)' : 'NO-HIT(正确)');
|
||||
|
||||
console.log(` ${icon} ${hitLabel} | ${r.elapsed}ms | tokens: ${r.usage.prompt_tokens || '?'}→${r.usage.completion_tokens || '?'}`);
|
||||
console.log(` 回答: ${r.content.slice(0, 150)}${r.content.length > 150 ? '...' : ''}\n`);
|
||||
|
||||
results.push({ ...tc, ...r, matchExpect });
|
||||
|
||||
if (!catStats[tc.cat]) catStats[tc.cat] = { total: 0, correct: 0, totalMs: 0 };
|
||||
catStats[tc.cat].total++;
|
||||
if (matchExpect) catStats[tc.cat].correct++;
|
||||
catStats[tc.cat].totalMs += r.elapsed;
|
||||
}
|
||||
|
||||
// ── 分类汇总 ──
|
||||
console.log('============================================================');
|
||||
console.log(' 分类汇总');
|
||||
console.log('============================================================');
|
||||
let totalCorrect = 0, totalMs = 0;
|
||||
for (const [cat, s] of Object.entries(catStats)) {
|
||||
const pct = Math.round(s.correct / s.total * 100);
|
||||
const avgMs = Math.round(s.totalMs / s.total);
|
||||
console.log(` 【${cat}】${s.correct}/${s.total} 正确 (${pct}%) | 平均 ${avgMs}ms`);
|
||||
totalCorrect += s.correct;
|
||||
totalMs += s.totalMs;
|
||||
}
|
||||
|
||||
// ── 总汇总 ──
|
||||
console.log('\n============================================================');
|
||||
console.log(' 总汇总');
|
||||
console.log('============================================================');
|
||||
const totalPct = Math.round(totalCorrect / TEST_CASES.length * 100);
|
||||
const avgMs = Math.round(totalMs / TEST_CASES.length);
|
||||
console.log(` 总正确率: ${totalCorrect}/${TEST_CASES.length} (${totalPct}%)`);
|
||||
console.log(` 平均延迟: ${avgMs}ms`);
|
||||
|
||||
// ── 异常用例列表 ──
|
||||
const issues = results.filter(r => !r.matchExpect);
|
||||
if (issues.length > 0) {
|
||||
console.log(`\n ⚠️ 异常用例 (${issues.length}个):`);
|
||||
issues.forEach(r => {
|
||||
console.log(` - 【${r.cat}】${r.name}: ${r.hit ? 'HIT' : 'NO-HIT'} (期望${r.expectHit ? 'HIT' : 'NO-HIT'})`);
|
||||
});
|
||||
} else {
|
||||
console.log('\n 🎉 全部用例符合预期！');
|
||||
}
|
||||
}
|
||||
|
||||
runTest().catch(err => {
|
||||
console.error('测试失败:', err.message);
|
||||
process.exit(1);
|
||||
});
|
||||
541
test2/server/tests/test_kb_protection.js
Normal file
541
test2/server/tests/test_kb_protection.js
Normal file
@@ -0,0 +1,541 @@
|
||||
/**
|
||||
* KB保护窗口 + 质疑关键词 + query去噪 + 话题记忆 功能性测试
|
||||
* 针对"粉末"幻觉问题暴露的所有修复点进行回归测试
|
||||
*
|
||||
* 运行方式: node --test tests/test_kb_protection.js
|
||||
*/
|
||||
const { describe, it } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
// ===== 被测模块加载 =====
|
||||
const { shouldForceKnowledgeRoute } = require('../services/realtimeDialogRouting');
|
||||
const { hasKnowledgeRouteKeyword } = require('../services/knowledgeKeywords');
|
||||
const contextKeywordTracker = require('../services/contextKeywordTracker');
|
||||
|
||||
// toolExecutor 需要特殊处理（有外部依赖），直接require静态方法
|
||||
let ToolExecutor;
|
||||
try {
|
||||
ToolExecutor = require('../services/toolExecutor');
|
||||
} catch (e) {
|
||||
// toolExecutor可能依赖env，提供fallback
|
||||
ToolExecutor = null;
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 1. isKnowledgeFollowUp / shouldForceKnowledgeRoute 质疑场景测试
|
||||
// ================================================================
|
||||
describe('shouldForceKnowledgeRoute — 质疑/纠正类话术', () => {
|
||||
// 为所有需要context的测试准备KB上下文
|
||||
const kbContext = [
|
||||
{ role: 'user', content: '骨关节产品有哪些' },
|
||||
{ role: 'assistant', content: '德国PM的健骨至尊氨糖软骨素胶囊...' },
|
||||
];
|
||||
|
||||
// ---- 1. 直接否定 ----
|
||||
describe('直接否定', () => {
|
||||
const cases = [
|
||||
'不是的',
|
||||
'不是啊',
|
||||
'不是不是',
|
||||
'才不是',
|
||||
'没有啊',
|
||||
'没有吧',
|
||||
'哪有',
|
||||
'不是这么回事',
|
||||
'不是这么说的',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true,
|
||||
`"${text}" should be recognized as KB follow-up with KB context`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 2. 指出错误 ----
|
||||
describe('指出错误', () => {
|
||||
const cases = [
|
||||
'你搞错了吧',
|
||||
'说错了',
|
||||
'弄错了',
|
||||
'记错了',
|
||||
'搞混了吧',
|
||||
'你说反了',
|
||||
'记岔了',
|
||||
'张冠李戴',
|
||||
'答非所问',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true,
|
||||
`"${text}" should be recognized as KB follow-up`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 3. 说AI不对 ----
|
||||
describe('说AI不对', () => {
|
||||
const cases = [
|
||||
'不对不对',
|
||||
'你说的不对',
|
||||
'不准确',
|
||||
'说得不准',
|
||||
'回答有误',
|
||||
'不太对吧',
|
||||
'说的有问题',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 4. 与自己认知矛盾 ----
|
||||
describe('与认知矛盾', () => {
|
||||
const cases = [
|
||||
'跟我了解的不一样',
|
||||
'我记得不是这样',
|
||||
'我听说不是这样的',
|
||||
'跟之前说的不一样',
|
||||
'前后矛盾',
|
||||
'你刚才不是说胶囊吗',
|
||||
'别人告诉我是粉末',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 5. 怀疑/不信 ----
|
||||
describe('怀疑不信', () => {
|
||||
const cases = [
|
||||
'我不信',
|
||||
'骗人的吧',
|
||||
'忽悠人呢',
|
||||
'吹牛吧',
|
||||
'太夸张了',
|
||||
'离谱',
|
||||
'扯淡',
|
||||
'有依据吗',
|
||||
'有证据吗',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 6. 要求复查 ----
|
||||
describe('要求复查', () => {
|
||||
const cases = [
|
||||
'你再查查',
|
||||
'再确认一下',
|
||||
'重新查一下',
|
||||
'核实一下',
|
||||
'帮我再查一下',
|
||||
'你查一下',
|
||||
'搞清楚再说',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 7. 委婉质疑 ----
|
||||
describe('委婉质疑', () => {
|
||||
const cases = [
|
||||
'好像不是这样吧',
|
||||
'我觉得不太对',
|
||||
'恐怕不是吧',
|
||||
'感觉不对',
|
||||
'我怎么记得不一样',
|
||||
'印象中不是这样',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 8. 质问来源 ----
|
||||
describe('质问来源', () => {
|
||||
const cases = [
|
||||
'谁说的',
|
||||
'谁告诉你的',
|
||||
'你从哪知道的',
|
||||
'有什么根据',
|
||||
'你确定吗',
|
||||
'确定吗',
|
||||
'真的吗',
|
||||
'真的假的',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 9. 不可能/反问 ----
|
||||
describe('不可能/反问', () => {
|
||||
const cases = [
|
||||
'怎么可能',
|
||||
'不可能',
|
||||
'不会吧',
|
||||
'不是吧',
|
||||
'开玩笑吧',
|
||||
'别逗了',
|
||||
'胡说',
|
||||
'瞎说',
|
||||
'别瞎说',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 10. 产品形态纠正 ----
|
||||
describe('产品形态纠正', () => {
|
||||
const cases = [
|
||||
'粉末来的呀',
|
||||
'是胶囊不是粉末',
|
||||
'这个是冲剂',
|
||||
'直接吞的',
|
||||
'冲着喝的',
|
||||
'是片剂',
|
||||
'口服液来着',
|
||||
'泡着喝',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 11. 纠正句式 ----
|
||||
describe('纠正句式', () => {
|
||||
const cases = [
|
||||
'到底是粉末还是胶囊',
|
||||
'究竟是什么形状',
|
||||
'应该是冲剂吧',
|
||||
'明明是粉末',
|
||||
'其实是胶囊',
|
||||
'怎么变成粉末了',
|
||||
'不应该是这样',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbContext), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---- 复现"粉末"原始场景 ----
|
||||
describe('复现原始"粉末"幻觉场景', () => {
|
||||
it('"粉末来的呀，你是搞错了吧？" → 必须走KB路由', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('粉末来的呀，你是搞错了吧？', kbContext), true);
|
||||
});
|
||||
|
||||
it('"粉末来的呀，你是搞错了吧？" → 无context也应命中（关键词直接匹配）', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('粉末来的呀，你是搞错了吧？'), true);
|
||||
});
|
||||
|
||||
it('"不对，是粉末不是胶囊" → 走KB路由', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('不对，是粉末不是胶囊', kbContext), true);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- 负面用例：纯闲聊不应走KB ----
|
||||
describe('负面用例：纯闲聊不应触发KB', () => {
|
||||
const cases = [
|
||||
'你好',
|
||||
'谢谢',
|
||||
'再见',
|
||||
'今天天气好',
|
||||
'哈哈哈',
|
||||
'嗯嗯',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 不应走KB路由`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text), false,
|
||||
`"${text}" should NOT route to KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 2. hasKnowledgeRouteKeyword — 新增质疑关键词命中测试
|
||||
// ================================================================
|
||||
describe('hasKnowledgeRouteKeyword — 新增质疑/产品形态关键词', () => {
|
||||
describe('产品剂型直接命中', () => {
|
||||
const keywords = ['粉末', '胶囊', '片剂', '冲剂', '口服液', '软胶囊', '颗粒', '膏状'];
|
||||
for (const kw of keywords) {
|
||||
it(`"${kw}" → 应直接命中KB关键词`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('质疑词直接命中', () => {
|
||||
const keywords = ['搞错了', '说错了', '不对', '确定吗', '真的吗', '不可能', '胡说', '离谱', '核实一下'];
|
||||
for (const kw of keywords) {
|
||||
it(`"${kw}" → 应直接命中KB关键词`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('复合句子中的关键词命中', () => {
|
||||
const cases = [
|
||||
'粉末来的呀你搞错了',
|
||||
'这个产品不对，应该是胶囊',
|
||||
'你确定吗这个是冲剂',
|
||||
'谁说的这是片剂',
|
||||
];
|
||||
for (const text of cases) {
|
||||
it(`"${text}" → 应命中KB关键词`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(text), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 3. sanitizeRewrittenQuery — 去噪截断测试
|
||||
// ================================================================
|
||||
describe('sanitizeRewrittenQuery — query后处理', () => {
|
||||
// 只有ToolExecutor加载成功才测试
|
||||
const skip = !ToolExecutor || !ToolExecutor.sanitizeRewrittenQuery;
|
||||
|
||||
it('清理口语填充词', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('骨关节啊怎么吃呢');
|
||||
assert.ok(!result.includes('啊'), `Should remove filler: got "${result}"`);
|
||||
assert.ok(!result.includes('呢'), `Should remove filler: got "${result}"`);
|
||||
assert.ok(result.includes('骨关节'), 'Should keep core content');
|
||||
assert.ok(result.includes('怎么吃'), 'Should keep core content');
|
||||
});
|
||||
|
||||
it('去除重复片段', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('骨关节 骨关节 怎么吃');
|
||||
const count = (result.match(/骨关节/g) || []).length;
|
||||
assert.equal(count, 1, `Should dedupe: got "${result}"`);
|
||||
});
|
||||
|
||||
it('截断超长query到80字符', { skip }, () => {
|
||||
const longQuery = '德国PM细胞营养素基础套装大白小红小白一成系统NTC营养保送系统的功效作用成分配方怎么吃怎么用服用方法适合谁适用人群区别搭配原理价格多少钱哪里买';
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery(longQuery);
|
||||
assert.ok(result.length <= 80, `Should truncate to <=80: got len=${result.length}`);
|
||||
});
|
||||
|
||||
it('空输入返回空', { skip }, () => {
|
||||
assert.equal(ToolExecutor.sanitizeRewrittenQuery(''), '');
|
||||
assert.equal(ToolExecutor.sanitizeRewrittenQuery(null), '');
|
||||
});
|
||||
|
||||
it('正常query不应被破坏', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('德国PM骨关节产品怎么吃');
|
||||
assert.ok(result.includes('德国PM'), 'Should preserve content');
|
||||
assert.ok(result.includes('骨关节'), 'Should preserve content');
|
||||
assert.ok(result.includes('怎么吃'), 'Should preserve content');
|
||||
});
|
||||
|
||||
it('清理连续标点', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('骨关节？？！！怎么吃');
|
||||
assert.ok(!/[？！]{2,}/.test(result), `Should clean punctuation: got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 4. enrichQueryWithContext — KB话题记忆优先级测试
|
||||
// ================================================================
|
||||
describe('enrichQueryWithContext — KB话题记忆优先', () => {
|
||||
const sessionId = 'test_session_' + Date.now();
|
||||
|
||||
it('有KB话题记忆时，追问应关联KB话题而非历史keyword', () => {
|
||||
// 模拟session有KB话题记忆
|
||||
const mockSession = {
|
||||
_lastKbTopic: '骨关节产品有哪些',
|
||||
_lastKbHitAt: Date.now(),
|
||||
};
|
||||
|
||||
// 往keyword tracker中注入旧关键词（模拟上一个话题是"一成系统"）
|
||||
contextKeywordTracker.updateSession(sessionId, '一成系统详细介绍');
|
||||
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(sessionId, '怎么吃', mockSession);
|
||||
|
||||
assert.ok(result.includes('骨关节'), `Should use KB topic memory: got "${result}"`);
|
||||
assert.ok(!result.includes('一成系统'), `Should NOT use old keyword: got "${result}"`);
|
||||
});
|
||||
|
||||
it('KB话题记忆过期后，回退到keyword tracker', () => {
|
||||
const mockSession = {
|
||||
_lastKbTopic: '骨关节产品有哪些',
|
||||
_lastKbHitAt: Date.now() - 120000, // 2分钟前，已过期
|
||||
};
|
||||
|
||||
contextKeywordTracker.updateSession(sessionId, '一成系统详细介绍');
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(sessionId, '怎么吃', mockSession);
|
||||
|
||||
assert.ok(!result.includes('骨关节'), `Should NOT use expired KB topic: got "${result}"`);
|
||||
});
|
||||
|
||||
it('无session时使用keyword tracker', () => {
|
||||
contextKeywordTracker.updateSession(sessionId, '基础套装功效');
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(sessionId, '怎么吃', null);
|
||||
|
||||
assert.ok(result.includes('怎么吃'), `Should include query: got "${result}"`);
|
||||
});
|
||||
|
||||
it('非追问类query不做enrichment', () => {
|
||||
const mockSession = {
|
||||
_lastKbTopic: '骨关节产品有哪些',
|
||||
_lastKbHitAt: Date.now(),
|
||||
};
|
||||
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(sessionId, '德国PM公司介绍', mockSession);
|
||||
assert.equal(result, '德国PM公司介绍', 'Non-follow-up should not be enriched');
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 5. KB保护窗口 — 集成场景模拟测试
|
||||
// ================================================================
|
||||
describe('KB保护窗口 — 场景模拟', () => {
|
||||
// 模拟nativeVoiceGateway中的保护窗口逻辑
|
||||
function simulateProtectionWindow(cleanText, session) {
|
||||
let isKnowledgeCandidate = shouldForceKnowledgeRoute(cleanText);
|
||||
const KB_PROTECTION_WINDOW_MS = 60000;
|
||||
if (!isKnowledgeCandidate && session._lastKbHitAt && (Date.now() - session._lastKbHitAt < KB_PROTECTION_WINDOW_MS)) {
|
||||
const isPureChitchat = /^(喂|你好|嗨|谢谢|再见|拜拜|好的|嗯|哦|行|没事了|不用了|可以了)[，,。！？\s]*$/.test(cleanText);
|
||||
if (!isPureChitchat) {
|
||||
isKnowledgeCandidate = true;
|
||||
}
|
||||
}
|
||||
return isKnowledgeCandidate;
|
||||
}
|
||||
|
||||
it('复现场景：KB hit后用户说"粉末来的呀你搞错了吧" → 应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 5000, _lastKbTopic: '骨关节' };
|
||||
assert.equal(simulateProtectionWindow('粉末来的呀，你是搞错了吧？', session), true);
|
||||
});
|
||||
|
||||
it('保护窗口内："你说的不对" → 应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtectionWindow('你说的不对', session), true);
|
||||
});
|
||||
|
||||
it('保护窗口内："哦这样啊" → 非闲聊，应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtectionWindow('哦这样啊', session), true);
|
||||
});
|
||||
|
||||
it('保护窗口内：纯闲聊"谢谢" → 不应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtectionWindow('谢谢', session), false);
|
||||
});
|
||||
|
||||
it('保护窗口内：纯闲聊"好的" → 不应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtectionWindow('好的', session), false);
|
||||
});
|
||||
|
||||
it('保护窗口内：纯闲聊"再见" → 不应走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtectionWindow('再见', session), false);
|
||||
});
|
||||
|
||||
it('保护窗口过期后："你搞错了" → 仍走KB（因为关键词直接命中）', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 120000 };
|
||||
assert.equal(simulateProtectionWindow('你搞错了', session), true);
|
||||
});
|
||||
|
||||
it('保护窗口过期后："天气怎么样" → 不走KB', () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 120000 };
|
||||
assert.equal(simulateProtectionWindow('今天心情好', session), false);
|
||||
});
|
||||
|
||||
it('无KB历史时："随便聊聊" → 不走KB', () => {
|
||||
const session = { _lastKbHitAt: 0 };
|
||||
assert.equal(simulateProtectionWindow('随便聊聊', session), false);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 6. 端到端场景回归测试（模拟完整对话链路）
|
||||
// ================================================================
|
||||
describe('端到端场景回归 — 模拟"粉末"幻觉完整链路', () => {
|
||||
it('场景1: 问骨关节→追问怎么吃→质疑粉末，三轮都应走KB', () => {
|
||||
// 第1轮：问骨关节
|
||||
const r1 = shouldForceKnowledgeRoute('骨关节产品有哪些');
|
||||
assert.equal(r1, true, '第1轮"骨关节产品有哪些"应走KB');
|
||||
|
||||
// 第2轮：追问怎么吃（有上下文）
|
||||
const ctx = [
|
||||
{ role: 'user', content: '骨关节产品有哪些' },
|
||||
{ role: 'assistant', content: '德国PM的健骨至尊氨糖软骨素胶囊...' },
|
||||
];
|
||||
const r2 = shouldForceKnowledgeRoute('怎么吃', ctx);
|
||||
assert.equal(r2, true, '第2轮"怎么吃"有KB上下文应走KB');
|
||||
|
||||
// 第3轮：质疑（关键词直接命中）
|
||||
const r3 = shouldForceKnowledgeRoute('粉末来的呀，你是搞错了吧？');
|
||||
assert.equal(r3, true, '第3轮"粉末来的呀搞错了吧"应走KB');
|
||||
});
|
||||
|
||||
it('场景2: 用户用各种方式质疑，全部应走KB', () => {
|
||||
const challengeTexts = [
|
||||
'粉末来的呀，你是搞错了吧？',
|
||||
'不对不对，是粉末',
|
||||
'你说的不对，这个是冲剂',
|
||||
'我记得不是胶囊啊',
|
||||
'你再查查，应该是粉末',
|
||||
'好像不是这样吧',
|
||||
'谁告诉你是胶囊的',
|
||||
'怎么可能是胶囊',
|
||||
'明明是粉末状的',
|
||||
'冲着喝的不是吞的',
|
||||
];
|
||||
const ctx = [
|
||||
{ role: 'user', content: '骨关节产品' },
|
||||
{ role: 'assistant', content: '氨糖软骨素胶囊' },
|
||||
];
|
||||
|
||||
for (const text of challengeTexts) {
|
||||
const result = shouldForceKnowledgeRoute(text, ctx);
|
||||
assert.equal(result, true, `质疑"${text}"应走KB路由`);
|
||||
}
|
||||
});
|
||||
|
||||
it('场景3: 正常追问也应走KB', () => {
|
||||
const followUps = [
|
||||
'详细说说',
|
||||
'怎么吃',
|
||||
'多少钱',
|
||||
'适合谁',
|
||||
'功效是什么',
|
||||
'成分是什么',
|
||||
];
|
||||
const ctx = [
|
||||
{ role: 'user', content: '一成系统' },
|
||||
{ role: 'assistant', content: '一成系统是德国PM的细胞营养素体系...' },
|
||||
];
|
||||
|
||||
for (const text of followUps) {
|
||||
const result = shouldForceKnowledgeRoute(text, ctx);
|
||||
assert.equal(result, true, `追问"${text}"有KB上下文应走KB`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
console.log('\n=== 测试文件加载完成，开始执行 ===\n');
|
||||
576
test2/server/tests/test_kb_protection_extended.js
Normal file
576
test2/server/tests/test_kb_protection_extended.js
Normal file
@@ -0,0 +1,576 @@
|
||||
/**
|
||||
* KB保护窗口 + 质疑检测 + query去噪 + 话题记忆 深度扩展测试
|
||||
* 与 test_kb_protection.js 互补，覆盖更多边界、组合、时序场景
|
||||
*
|
||||
* 运行方式: node --test tests/test_kb_protection_extended.js
|
||||
*/
|
||||
const { describe, it } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const { shouldForceKnowledgeRoute, normalizeKnowledgeAlias } = require('../services/realtimeDialogRouting');
|
||||
const { hasKnowledgeRouteKeyword } = require('../services/knowledgeKeywords');
|
||||
const contextKeywordTracker = require('../services/contextKeywordTracker');
|
||||
|
||||
let ToolExecutor;
|
||||
try { ToolExecutor = require('../services/toolExecutor'); } catch (e) { ToolExecutor = null; }
|
||||
|
||||
const kbCtx = [
|
||||
{ role: 'user', content: '基础三合一怎么吃' },
|
||||
{ role: 'assistant', content: '大白早上空腹1平勺温水冲服，小红中午1平勺，小白睡前1平勺...' },
|
||||
];
|
||||
|
||||
// ================================================================
|
||||
// 1. shouldForceKnowledgeRoute — 组合质疑+产品名
|
||||
// ================================================================
|
||||
describe('组合质疑+产品名 —— 质疑词嵌入具体产品场景', () => {
|
||||
const combos = [
|
||||
'大白不是这样吃的',
|
||||
'小红功效你搞错了吧',
|
||||
'CC套装明明是乳霜',
|
||||
'基础三合一不是冲剂',
|
||||
'Q10你说的不对',
|
||||
'D-Drink不是这么用的',
|
||||
'一成系统跟我了解的不一样',
|
||||
'火炉原理好像不是这么说的',
|
||||
'IB5不可能是这个功效吧',
|
||||
'小白Restorate我记得不是这样',
|
||||
'儿童倍适应该是胶囊不是粉末',
|
||||
'Hair+你再查查',
|
||||
'NTC你确定是这个原理吗',
|
||||
'邓白氏谁说的AAA+',
|
||||
'关节套装真的有这个功效吗',
|
||||
'TopShape太夸张了吧',
|
||||
'ProShape氨基酸骗人的吧',
|
||||
'叶黄素我不信有这个作用',
|
||||
'乳清蛋白说的有问题',
|
||||
'运动饮料不是这个成分',
|
||||
];
|
||||
|
||||
for (const text of combos) {
|
||||
it(`"${text}" → 应走KB`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbCtx), true, `"${text}" should route KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 2. shouldForceKnowledgeRoute — 带标点/语气词的质疑变体
|
||||
// ================================================================
|
||||
describe('带标点/语气词的质疑变体', () => {
|
||||
const variants = [
|
||||
'不对吧？',
|
||||
'不对不对不对！',
|
||||
'你搞错了吧！！',
|
||||
'说错了，，，',
|
||||
'我不信！真的假的？',
|
||||
'骗人的吧……',
|
||||
'太夸张了～',
|
||||
'离谱啊！',
|
||||
'扯淡吧？？',
|
||||
'怎么可能？？？',
|
||||
'不可能！不是吧！',
|
||||
'好像不对哦～',
|
||||
'你再查查？',
|
||||
'核实一下嘛。',
|
||||
'真的吗？真的吗？',
|
||||
'谁说的啊？',
|
||||
'有什么根据呢？',
|
||||
'到底是什么啊！',
|
||||
'应该是胶囊呀～',
|
||||
'明明是粉末嘛！',
|
||||
];
|
||||
|
||||
for (const text of variants) {
|
||||
it(`"${text}" → 应走KB`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbCtx), true, `"${text}" should route KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 3. shouldForceKnowledgeRoute — 前缀剥离后的质疑
|
||||
// ================================================================
|
||||
describe('前缀剥离 —— 带"那/那你/你再/再"前缀的质疑', () => {
|
||||
const prefixed = [
|
||||
'你再看看这个对不对',
|
||||
'帮我再确认一下',
|
||||
'你再看看吧',
|
||||
'再来说说',
|
||||
'麻烦你核实一下',
|
||||
'帮我确认一下',
|
||||
'那你确定吗',
|
||||
'那再确认一下',
|
||||
'那不对吧',
|
||||
'那你搞错了',
|
||||
'那我记得不是这样',
|
||||
'再帮我查查',
|
||||
'那再给我介绍一下',
|
||||
'那详细说说',
|
||||
'你再展开说说',
|
||||
'那怎么吃',
|
||||
'再讲讲功效是什么',
|
||||
];
|
||||
|
||||
for (const text of prefixed) {
|
||||
it(`"${text}" → 应识别为KB追问`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbCtx), true, `"${text}" should route KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 4. shouldForceKnowledgeRoute — 长句中嵌入质疑
|
||||
// ================================================================
|
||||
describe('长句中嵌入质疑 —— 质疑词不在句首', () => {
|
||||
const longSentences = [
|
||||
'我刚才听你说的跟我了解的不一样',
|
||||
'你之前的回答好像有误吧',
|
||||
'按照我之前看到的资料应该是胶囊',
|
||||
'怎么跟我之前在网上搜的不一致',
|
||||
'别人告诉我是粉末的来着',
|
||||
'但是我觉得你说的不太对',
|
||||
'我看了很多资料你确定吗',
|
||||
'感觉你说的和我了解的有出入',
|
||||
'以前有人跟我说是冲着喝的',
|
||||
'但是网上说法跟你不一样',
|
||||
'我一直以为不是这样的',
|
||||
'到底是什么意思啊',
|
||||
'这些信息可靠吗有根据吗',
|
||||
'我朋友说你讲的不对',
|
||||
'这跟官方说的不一致吧',
|
||||
];
|
||||
|
||||
for (const text of longSentences) {
|
||||
it(`"${text}" → 应走KB`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbCtx), true, `"${text}" should route KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 5. shouldForceKnowledgeRoute — 纯追问模式（subject+action）
|
||||
// ================================================================
|
||||
describe('subject+action追问模式 —— 带上下文', () => {
|
||||
const subjectActions = [
|
||||
'这个怎么吃',
|
||||
'那个功效是什么',
|
||||
'它适合谁',
|
||||
'这个产品多少钱',
|
||||
'那个产品哪里买',
|
||||
'这个怎么用',
|
||||
'那个怎么操作',
|
||||
'这个系统怎么配置',
|
||||
'这个产品成分是什么',
|
||||
'那个产品有什么功效',
|
||||
'它怎么服用',
|
||||
'这个有什么好处',
|
||||
'那个配方',
|
||||
'这个原理是什么',
|
||||
'这个产品适合什么人',
|
||||
'那个产品怎么买',
|
||||
];
|
||||
|
||||
for (const text of subjectActions) {
|
||||
it(`"${text}" → 有上下文应走KB`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, kbCtx), true, `"${text}" should route KB with context`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 6. shouldForceKnowledgeRoute — 否定用例扩展
|
||||
// ================================================================
|
||||
describe('否定用例扩展 —— 不应走KB的各类闲聊', () => {
|
||||
const chitchat = [
|
||||
'你好',
|
||||
'嗨',
|
||||
'谢谢',
|
||||
'再见',
|
||||
'好的',
|
||||
'嗯嗯',
|
||||
'哈哈哈',
|
||||
'拜拜',
|
||||
'没事了',
|
||||
'不用了',
|
||||
'可以了',
|
||||
'行',
|
||||
'知道了',
|
||||
'明白了',
|
||||
'了解了',
|
||||
'好吧',
|
||||
'算了',
|
||||
'今天天气好',
|
||||
'你是谁',
|
||||
'你叫什么名字',
|
||||
'你是机器人吗',
|
||||
'讲个笑话',
|
||||
'唱首歌',
|
||||
'几点了',
|
||||
'我饿了',
|
||||
'晚安',
|
||||
'早上好',
|
||||
'下午好',
|
||||
'辛苦了',
|
||||
'厉害',
|
||||
];
|
||||
|
||||
for (const text of chitchat) {
|
||||
it(`"${text}" → 不应走KB`, () => {
|
||||
assert.equal(shouldForceKnowledgeRoute(text), false, `"${text}" should NOT route KB`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 7. hasKnowledgeRouteKeyword — 系统性全类别关键词覆盖
|
||||
// ================================================================
|
||||
describe('hasKnowledgeRouteKeyword — 产品名关键词系统覆盖', () => {
|
||||
const productKeywords = [
|
||||
'大白', '小红', '小白', '基础三合一', 'Basics', 'Activize', 'Restorate',
|
||||
'儿童倍适', 'CC套装', 'CC-Cell', 'Q10', 'IB5', 'D-Drink',
|
||||
'Hair+', 'ProShape氨基酸', 'Herbal Tea', 'TopShape', 'Men Face',
|
||||
'MEN+', '乐活', '草本茶', '叶黄素', '葡萄籽', '益生菌',
|
||||
'胶原蛋白', '关节套装', '乳清蛋白', '运动饮料', '苹果细胞抗氧素',
|
||||
];
|
||||
|
||||
for (const kw of productKeywords) {
|
||||
it(`产品"${kw}" → 应命中`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true, `"${kw}" should match`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('hasKnowledgeRouteKeyword — FAQ/科学关键词系统覆盖', () => {
|
||||
const faqKeywords = [
|
||||
'怎么吃', '功效', '成分', '多少钱', '价格', '适合谁',
|
||||
'副作用', '多久见效', '见效', '好转反应', '是不是传销',
|
||||
'传销', '是不是传销', '保质期', '哪里买', '怎么买',
|
||||
'NTC', '火炉原理', '阿育吠陀', '细胞营养素',
|
||||
'正规吗', '合法吗', '贵不贵', '不舒服',
|
||||
];
|
||||
|
||||
for (const kw of faqKeywords) {
|
||||
it(`FAQ"${kw}" → 应命中`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true, `"${kw}" should match`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('hasKnowledgeRouteKeyword — 事业/培训关键词系统覆盖', () => {
|
||||
const bizKeywords = [
|
||||
'招商', '代理', '加盟', '事业机会', '创业', '起步三关',
|
||||
'精品会议', '成长上总裁', '做PM', '加入PM', 'PM事业',
|
||||
];
|
||||
|
||||
for (const kw of bizKeywords) {
|
||||
it(`事业"${kw}" → 应命中`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true, `"${kw}" should match`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('hasKnowledgeRouteKeyword — 质疑类关键词系统覆盖', () => {
|
||||
const challengeKeywords = [
|
||||
'搞错了', '说错了', '弄错了', '不对', '不准确', '有误',
|
||||
'确定吗', '真的吗', '不可能', '胡说', '骗人', '离谱',
|
||||
'核实一下', '再查查', '粉末', '胶囊', '片剂', '冲剂',
|
||||
'口服液', '软胶囊', '颗粒', '膏状', '到底是', '应该是',
|
||||
'明明是', '不信', '吹牛', '扯淡', '有依据吗', '谁说的',
|
||||
];
|
||||
|
||||
for (const kw of challengeKeywords) {
|
||||
it(`质疑"${kw}" → 应命中`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), true, `"${kw}" should match`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('hasKnowledgeRouteKeyword — 不应命中的普通词汇', () => {
|
||||
const noMatch = [
|
||||
'你好', '天气', '笑话', '唱歌', '吃饭', '睡觉',
|
||||
'电影', '音乐', '游戏', '旅游', '工作', '学习',
|
||||
'开心', '难过', '累', '饿', '渴', '无聊',
|
||||
];
|
||||
|
||||
for (const kw of noMatch) {
|
||||
it(`闲聊"${kw}" → 不应命中`, () => {
|
||||
assert.equal(hasKnowledgeRouteKeyword(kw), false, `"${kw}" should NOT match`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 8. sanitizeRewrittenQuery — 深度去噪测试
|
||||
// ================================================================
|
||||
describe('sanitizeRewrittenQuery — 深度去噪截断', () => {
|
||||
const skip = !ToolExecutor || !ToolExecutor.sanitizeRewrittenQuery;
|
||||
|
||||
const fillerCases = [
|
||||
['骨关节啊嗯呢产品', '骨关节', '去除嗯啊呢'],
|
||||
['那个就是说这个呢功效是什么', '功效', '去除口语填充'],
|
||||
['骨关节哦嗯额功效', '骨关节', '去除多个语气词'],
|
||||
['基础三合一呀怎么吃呀', '基础三合一', '去除呀'],
|
||||
['嗯嗯那个小红功效', '小红', '去除嗯嗯那个'],
|
||||
];
|
||||
|
||||
for (const [input, expectContain, label] of fillerCases) {
|
||||
it(`${label}: "${input}" → 含"${expectContain}"`, { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery(input);
|
||||
assert.ok(result.includes(expectContain), `Got "${result}"`);
|
||||
});
|
||||
}
|
||||
|
||||
it('多次重复去重: "小红 小红 小红 功效"', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('小红 小红 小红 功效');
|
||||
const count = (result.match(/小红/g) || []).length;
|
||||
assert.ok(count <= 2, `Should dedupe, got "${result}" (${count} occurrences)`);
|
||||
});
|
||||
|
||||
it('去除连续空格', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('骨关节 产品 功效');
|
||||
assert.ok(!/ /.test(result), `Should remove multi-spaces, got "${result}"`);
|
||||
});
|
||||
|
||||
const truncCases = [
|
||||
''.padEnd(100, '德国PM细胞营养素基础套装大白小红小白'),
|
||||
'这是一段超长的查询' + '关于产品的详细信息'.repeat(10),
|
||||
];
|
||||
|
||||
for (let i = 0; i < truncCases.length; i++) {
|
||||
it(`超长截断 case ${i + 1}`, { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery(truncCases[i]);
|
||||
assert.ok(result.length <= 80, `Should truncate, got len=${result.length}`);
|
||||
});
|
||||
}
|
||||
|
||||
it('特殊字符不崩溃', { skip }, () => {
|
||||
const specials = ['骨关节\n产品', '基础三合一\t怎么吃', '小红\r\n功效'];
|
||||
for (const s of specials) {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery(s);
|
||||
assert.equal(typeof result, 'string');
|
||||
}
|
||||
});
|
||||
|
||||
it('已干净的query不被破坏', { skip }, () => {
|
||||
const clean = '德国PM基础三合一 大白 小红 小白 怎么吃';
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery(clean);
|
||||
assert.ok(result.includes('基础三合一'), `Core preserved: got "${result}"`);
|
||||
assert.ok(result.includes('怎么吃'), `Action preserved: got "${result}"`);
|
||||
});
|
||||
|
||||
it('全标点输入', { skip }, () => {
|
||||
const result = ToolExecutor.sanitizeRewrittenQuery('，，，。。。！！！');
|
||||
assert.equal(typeof result, 'string');
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 9. enrichQueryWithContext — 多场景深度测试
|
||||
// ================================================================
|
||||
describe('enrichQueryWithContext — 多场景深度', () => {
|
||||
const sid = 'test_enrich_ext_' + Date.now();
|
||||
|
||||
it('新session空关键词 → 返回原始query', () => {
|
||||
const result = contextKeywordTracker.enrichQueryWithContext('empty_sid_' + Date.now(), '怎么吃', null);
|
||||
assert.ok(result.includes('怎么吃'), `Should return original: got "${result}"`);
|
||||
});
|
||||
|
||||
it('有关键词+追问 → 关键词注入', () => {
|
||||
const s = 'enrich_inject_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '大白产品功效详细介绍');
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, '怎么吃', null);
|
||||
assert.ok(result.includes('怎么吃'), `Should include query: got "${result}"`);
|
||||
});
|
||||
|
||||
it('非追问query → 不注入关键词', () => {
|
||||
const s = 'enrich_noinject_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '大白产品功效');
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, '德国PM公司在哪里', null);
|
||||
assert.equal(result, '德国PM公司在哪里', `Non-follow-up should not inject: got "${result}"`);
|
||||
});
|
||||
|
||||
it('KB话题记忆优先于keyword tracker', () => {
|
||||
const s = 'enrich_priority_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '一成系统三大平台');
|
||||
const session = { _lastKbTopic: 'CC套装功效', _lastKbHitAt: Date.now() };
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, '怎么吃', session);
|
||||
assert.ok(result.includes('CC'), `Should use KB topic: got "${result}"`);
|
||||
});
|
||||
|
||||
it('KB话题过期(>60s) → 降级到keyword tracker', () => {
|
||||
const s = 'enrich_expired_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '一成系统详细');
|
||||
const session = { _lastKbTopic: 'CC套装功效', _lastKbHitAt: Date.now() - 90000 };
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, '怎么吃', session);
|
||||
assert.ok(!result.includes('CC'), `Should NOT use expired KB topic: got "${result}"`);
|
||||
});
|
||||
|
||||
it('多轮更新后取最近关键词', () => {
|
||||
const s = 'enrich_multi_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '大白产品功效');
|
||||
contextKeywordTracker.updateSession(s, '小红Activize怎么吃');
|
||||
contextKeywordTracker.updateSession(s, 'Q10辅酵素作用');
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, '多少钱', null);
|
||||
assert.ok(result.includes('多少钱'), `Should include query: got "${result}"`);
|
||||
});
|
||||
|
||||
it('各种追问前缀都能触发enrichment', () => {
|
||||
const s = 'enrich_prefixes_' + Date.now();
|
||||
contextKeywordTracker.updateSession(s, '大白产品功效');
|
||||
const followUps = ['怎么吃', '功效是什么', '多少钱', '适合谁', '成分是什么', '哪里买', '副作用', '什么意思', '怎么用', '他的规格是什么', '它的包装是什么', '这款是什么剂型', '那个是什么形态', '一天几次', '每天几次', '每日几次'];
|
||||
for (const fup of followUps) {
|
||||
const result = contextKeywordTracker.enrichQueryWithContext(s, fup, null);
|
||||
assert.ok(result.includes(fup), `"${fup}" should be in result: got "${result}"`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 10. KB保护窗口 — 精细时序测试
|
||||
// ================================================================
|
||||
describe('KB保护窗口 — 精细时序与边界', () => {
|
||||
function simulateProtection(text, session) {
|
||||
let isKb = shouldForceKnowledgeRoute(text);
|
||||
const WINDOW = 60000;
|
||||
if (!isKb && session._lastKbHitAt && (Date.now() - session._lastKbHitAt < WINDOW)) {
|
||||
const isPureChitchat = /^(喂|你好|嗨|谢谢|再见|拜拜|好的|嗯|哦|行|没事了|不用了|可以了)[，,。！？\s]*$/.test(text);
|
||||
if (!isPureChitchat) isKb = true;
|
||||
}
|
||||
return isKb;
|
||||
}
|
||||
|
||||
describe('窗口内（5s-55s）非闲聊提升', () => {
|
||||
const timings = [5000, 10000, 20000, 30000, 45000, 55000, 59000, 59999];
|
||||
for (const t of timings) {
|
||||
it(`${t}ms前KB hit + "哦这样啊" → 应走KB`, () => {
|
||||
const session = { _lastKbHitAt: Date.now() - t };
|
||||
assert.equal(simulateProtection('哦这样啊', session), true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('窗口外（61s+）不提升', () => {
|
||||
const timings = [60001, 65000, 120000, 300000];
|
||||
for (const t of timings) {
|
||||
it(`${t}ms前KB hit + "哦这样啊" → 不走KB`, () => {
|
||||
const session = { _lastKbHitAt: Date.now() - t };
|
||||
assert.equal(simulateProtection('哦这样啊', session), false);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('窗口内各类纯闲聊不提升', () => {
|
||||
const chitchat = ['你好', '嗨', '谢谢', '再见', '拜拜', '好的', '嗯', '哦', '行', '没事了', '不用了', '可以了', '喂'];
|
||||
for (const c of chitchat) {
|
||||
it(`窗口内"${c}" → 不走KB`, () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 5000 };
|
||||
assert.equal(simulateProtection(c, session), false);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('窗口内各类非闲聊提升', () => {
|
||||
const nonChat = [
|
||||
'然后呢', '还有吗', '继续', '还有什么', '那怎么办',
|
||||
'这样可以吗', '有什么注意事项', '跟别的有什么区别',
|
||||
'会不会有副作用', '我能吃吗', '孕妇可以吗', '小孩能吃吗',
|
||||
'老人适合吗', '饭前还是饭后', '要吃多久', '一天几次',
|
||||
];
|
||||
for (const text of nonChat) {
|
||||
it(`窗口内"${text}" → 应走KB`, () => {
|
||||
const session = { _lastKbHitAt: Date.now() - 10000 };
|
||||
assert.equal(simulateProtection(text, session), true, `"${text}" should be elevated`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('无KB历史(_lastKbHitAt=0) → 不提升', () => {
|
||||
assert.equal(simulateProtection('然后呢', { _lastKbHitAt: 0 }), false);
|
||||
});
|
||||
|
||||
it('无KB历史(undefined) → 不提升', () => {
|
||||
assert.equal(simulateProtection('然后呢', {}), false);
|
||||
});
|
||||
|
||||
it('_lastKbHitAt=null → 不提升', () => {
|
||||
assert.equal(simulateProtection('然后呢', { _lastKbHitAt: null }), false);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 11. 端到端多轮模拟 — 更多变体场景
|
||||
// ================================================================
|
||||
describe('端到端多轮模拟 — 更多变体', () => {
|
||||
|
||||
it('3轮：产品→追问→质疑价格', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('Q10辅酵素功效'), true);
|
||||
const ctx = [{ role: 'user', content: 'Q10辅酵素功效' }, { role: 'assistant', content: 'Q10...' }];
|
||||
assert.equal(shouldForceKnowledgeRoute('多少钱', ctx), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('太贵了吧，你确定吗'), true);
|
||||
});
|
||||
|
||||
it('3轮：公司→认证→怀疑合法性', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('德国PM公司介绍'), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('邓白氏AAA+认证'), true);
|
||||
const ctx = [{ role: 'user', content: '邓白氏' }, { role: 'assistant', content: '邓白氏是...' }];
|
||||
assert.equal(shouldForceKnowledgeRoute('我不信，网上说是传销', ctx), true);
|
||||
});
|
||||
|
||||
it('4轮：系统→功能→质疑→再查', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('一成系统介绍'), true);
|
||||
const ctx1 = [{ role: 'user', content: '一成系统' }, { role: 'assistant', content: '一成系统...' }];
|
||||
assert.equal(shouldForceKnowledgeRoute('行动圈是什么', ctx1), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('跟我了解的不一样'), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('你再查查一成系统'), true);
|
||||
});
|
||||
|
||||
it('5轮：产品A→产品B→对比→质疑→纠正', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('大白怎么吃'), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('小红怎么吃'), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('大白和小红有什么区别'), true);
|
||||
const ctx = [{ role: 'user', content: '区别' }, { role: 'assistant', content: '大白是基础...' }];
|
||||
assert.equal(shouldForceKnowledgeRoute('你搞混了吧', ctx), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('应该是小红提供能量大白补充矿物质'), true);
|
||||
});
|
||||
|
||||
it('连续4次质疑不同方式', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('小白功效'), true);
|
||||
const ctx = [{ role: 'user', content: '小白功效' }, { role: 'assistant', content: '小白...' }];
|
||||
assert.equal(shouldForceKnowledgeRoute('不对吧', ctx), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('你再查查', ctx), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('我不信', ctx), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('有什么根据', ctx), true);
|
||||
});
|
||||
|
||||
it('KB话题→闲聊打断→再回到KB话题', () => {
|
||||
assert.equal(shouldForceKnowledgeRoute('CC套装怎么用'), true);
|
||||
assert.equal(shouldForceKnowledgeRoute('谢谢'), false);
|
||||
assert.equal(shouldForceKnowledgeRoute('CC套装适合谁'), true);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 12. normalizeKnowledgeAlias — 更多归一化场景
|
||||
// ================================================================
|
||||
describe('normalizeKnowledgeAlias — 更多归一化场景', () => {
|
||||
const cases = [
|
||||
['一成，，系统', '一成系统', '多标点分隔'],
|
||||
['一成、系统', '一成系统', '顿号分隔'],
|
||||
['一成 系统', '一成系统', '多空格分隔'],
|
||||
['大我产品', '大沃', '大我→大沃'],
|
||||
['大卧介绍', '大沃', '大卧→大沃'],
|
||||
['哎众享怎么用', 'Ai众享', '哎众享→Ai众享'],
|
||||
['艾众享是什么', 'Ai众享', '艾众享→Ai众享'],
|
||||
['盛卡学愿介绍', '盛咖学愿', '盛卡→盛咖'],
|
||||
['圣咖学院怎么用', '盛咖学愿', '圣咖学院→盛咖学愿'],
|
||||
];
|
||||
|
||||
for (const [input, expectContain, label] of cases) {
|
||||
it(`${label}: "${input}" → 含"${expectContain}"`, () => {
|
||||
const result = normalizeKnowledgeAlias(input);
|
||||
assert.ok(result.includes(expectContain), `Got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
console.log('\n=== KB保护扩展测试加载完成 ===\n');
|
||||
593
test2/server/tests/test_kb_scenarios.js
Normal file
593
test2/server/tests/test_kb_scenarios.js
Normal file
@@ -0,0 +1,593 @@
|
||||
/**
|
||||
* 基于知识库实际内容的功能性测试
|
||||
* 覆盖：单KB查询、多KB查询（话题切换）、追问+质疑混合、确定性改写、热答案匹配
|
||||
*
|
||||
* 运行方式: node --test tests/test_kb_scenarios.js
|
||||
*/
|
||||
const { describe, it } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const { shouldForceKnowledgeRoute } = require('../services/realtimeDialogRouting');
|
||||
const { hasKnowledgeRouteKeyword } = require('../services/knowledgeKeywords');
|
||||
const ToolExecutor = require('../services/toolExecutor');
|
||||
|
||||
// ================================================================
|
||||
// 辅助函数
|
||||
// ================================================================
|
||||
function assertKbRoute(text, ctx, msg) {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, ctx), true, msg || `"${text}" should route to KB`);
|
||||
}
|
||||
function assertNotKbRoute(text, ctx, msg) {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, ctx), false, msg || `"${text}" should NOT route to KB`);
|
||||
}
|
||||
function buildCtx(pairs) {
|
||||
return pairs.map(([role, content]) => ({ role, content }));
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 1. 单知识库查询 —— 每个产品/话题独立首轮查询
|
||||
// ================================================================
|
||||
describe('单KB查询 —— 产品类', () => {
|
||||
const productQueries = [
|
||||
['基础三合一怎么吃', '基础三合一'],
|
||||
['大白产品有什么功效', '大白Basics'],
|
||||
['小红Activize的作用是什么', '小红Activize'],
|
||||
['小白Restorate怎么服用', '小白Restorate'],
|
||||
['儿童倍适适合几岁的孩子', '儿童倍适'],
|
||||
['CC套装怎么用', 'CC套装'],
|
||||
['Q10辅酵素有什么功效', 'Q10辅酵素'],
|
||||
['IB5口腔喷雾怎么用', 'IB5口腔喷雾'],
|
||||
['D-Drink小绿排毒饮怎么用', 'D-Drink'],
|
||||
['Hair+发宝怎么用', 'Hair+发宝'],
|
||||
['运动饮料Fitness-Drink是什么', 'Fitness-Drink'],
|
||||
['TopShape纤萃减肥产品', 'TopShape'],
|
||||
['Generation 50+乐活产品', 'Generation 50+'],
|
||||
['Apple Antioxy细胞抗氧素功效', 'Apple Antioxy'],
|
||||
['ProShape氨基酸BCAA是什么', 'ProShape'],
|
||||
['Herbal Tea草本茶功效', 'Herbal Tea'],
|
||||
['Med Dental+草本护理牙膏', 'Med Dental+'],
|
||||
['Men Face男士护肤乳霜', 'Men Face'],
|
||||
['叶黄素产品怎么吃', '叶黄素'],
|
||||
['关节套装关节舒缓怎么用', '关节套装'],
|
||||
['乳清蛋白粉适合谁', '乳清蛋白'],
|
||||
['乐活奶昔怎么喝', '乐活奶昔'],
|
||||
];
|
||||
|
||||
for (const [query, label] of productQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('单KB查询 —— 系统/平台类', () => {
|
||||
const systemQueries = [
|
||||
['一成系统是什么', '一成系统'],
|
||||
['三大平台介绍一下', '三大平台'],
|
||||
['四大AI生态是什么', '四大AI生态'],
|
||||
['行动圈怎么用', '行动圈'],
|
||||
['盟主社区是什么', '盟主社区'],
|
||||
['AI众享是什么', 'AI众享'],
|
||||
['数字化工作室怎么用', '数字化工作室'],
|
||||
['盛咖学愿培训平台', '盛咖学愿'],
|
||||
];
|
||||
|
||||
for (const [query, label] of systemQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('单KB查询 —— 科学原理类', () => {
|
||||
const scienceQueries = [
|
||||
['NTC营养保送系统是什么原理', 'NTC'],
|
||||
['火炉原理是什么意思', '火炉原理'],
|
||||
['阿育吠陀是什么', '阿育吠陀'],
|
||||
];
|
||||
|
||||
for (const [query, label] of scienceQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('单KB查询 —— 公司/认证类', () => {
|
||||
const companyQueries = [
|
||||
['德国PM公司介绍', '德国PM'],
|
||||
['PM公司地址和电话', '地址电话'],
|
||||
['邓白氏认证是什么', '邓白氏'],
|
||||
['DSN全球100强', 'DSN'],
|
||||
['ELAB科隆名单认证', 'ELAB'],
|
||||
['PM是不是传销', '合法性'],
|
||||
['Rolf Sorg是谁', '创始人'],
|
||||
['宣明会慈善合作', '宣明会'],
|
||||
['培安烟台工厂', '培安'],
|
||||
];
|
||||
|
||||
for (const [query, label] of companyQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('单KB查询 —— FAQ常见问题类', () => {
|
||||
const faqQueries = [
|
||||
['PM产品多久见效', '见效时间'],
|
||||
['好转反应是什么', '好转反应'],
|
||||
['为什么要全套搭配使用', '全套搭配'],
|
||||
['和其他保健品有什么区别', '保健品区别'],
|
||||
['孕妇能吃PM产品吗', '特殊人群'],
|
||||
['产品多少钱', '价格'],
|
||||
['怎么加入PM', '加入方式'],
|
||||
['PM产品能治病吗', '治病声明'],
|
||||
];
|
||||
|
||||
for (const [query, label] of faqQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('单KB查询 —— 事业发展类', () => {
|
||||
const businessQueries = [
|
||||
['如何发展PM事业', '事业发展'],
|
||||
['线上拓客怎么做', '线上拓客'],
|
||||
['招商代理政策', '招商'],
|
||||
['新人起步三关是什么', '新人培训'],
|
||||
['为什么选择德国PM', '选择理由'],
|
||||
['陌生客户怎么沟通PM事业', '陌生沟通'],
|
||||
];
|
||||
|
||||
for (const [query, label] of businessQueries) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 2. 多KB查询 —— 话题切换场景
|
||||
// ================================================================
|
||||
describe('多KB查询 —— 话题切换（同一会话中切换不同产品/话题）', () => {
|
||||
|
||||
it('场景1: 大白→小红→小白 三个产品连续查询', () => {
|
||||
assertKbRoute('大白产品功效是什么');
|
||||
|
||||
const ctx1 = buildCtx([['user', '大白产品功效是什么'], ['assistant', '德国PM大白Basics是基础营养素...']]);
|
||||
assertKbRoute('那小红呢', ctx1);
|
||||
|
||||
const ctx2 = buildCtx([
|
||||
['user', '大白产品功效是什么'], ['assistant', '德国PM大白Basics...'],
|
||||
['user', '那小红呢'], ['assistant', 'FitLine小红Activize...'],
|
||||
]);
|
||||
assertKbRoute('小白怎么吃', ctx2);
|
||||
});
|
||||
|
||||
it('场景2: 产品→公司→再回到产品', () => {
|
||||
assertKbRoute('基础三合一介绍一下');
|
||||
|
||||
assertKbRoute('德国PM公司是什么时候成立的');
|
||||
|
||||
const ctx = buildCtx([
|
||||
['user', '德国PM公司介绍'], ['assistant', '德国PM-International是1993年创立的...'],
|
||||
]);
|
||||
assertKbRoute('那他们的产品有哪些', ctx);
|
||||
});
|
||||
|
||||
it('场景3: 产品→一成系统→培训', () => {
|
||||
assertKbRoute('CC套装怎么用');
|
||||
assertKbRoute('一成系统是什么');
|
||||
assertKbRoute('新人起步三关怎么做');
|
||||
});
|
||||
|
||||
it('场景4: 科学原理→产品→FAQ', () => {
|
||||
assertKbRoute('NTC营养保送系统原理');
|
||||
assertKbRoute('大白Basics功效');
|
||||
assertKbRoute('多久能见效');
|
||||
});
|
||||
|
||||
it('场景5: 合法性→公司→产品→事业', () => {
|
||||
assertKbRoute('PM是不是传销');
|
||||
assertKbRoute('邓白氏AAA+是什么');
|
||||
const ctx5 = buildCtx([['user', '邓白氏AAA+'], ['assistant', '邓白氏是全球最权威的商业信用评估机构...']]);
|
||||
assertKbRoute('那产品有哪些', ctx5);
|
||||
assertKbRoute('怎么加入PM事业');
|
||||
});
|
||||
|
||||
it('场景6: 快速切换5个不同产品', () => {
|
||||
const products = [
|
||||
'D-Drink排毒饮怎么用',
|
||||
'Q10辅酵素功效',
|
||||
'IB5口腔喷雾是什么',
|
||||
'叶黄素怎么吃',
|
||||
'关节套装适合谁',
|
||||
];
|
||||
for (const q of products) {
|
||||
assertKbRoute(q);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 3. 追问场景（有上下文）
|
||||
// ================================================================
|
||||
describe('追问场景 —— 基于上下文的知识库追问', () => {
|
||||
|
||||
it('聊基础三合一后追问"怎么吃"', () => {
|
||||
const ctx = buildCtx([['user', '基础三合一介绍'], ['assistant', '基础三合一包含大白小红小白...']]);
|
||||
assertKbRoute('怎么吃', ctx);
|
||||
});
|
||||
|
||||
it('聊小红后追问"功效是什么"', () => {
|
||||
const ctx = buildCtx([['user', '小红产品'], ['assistant', 'FitLine小红Activize Oxyplus...']]);
|
||||
assertKbRoute('功效是什么', ctx);
|
||||
});
|
||||
|
||||
it('聊一成系统后追问"怎么用"', () => {
|
||||
const ctx = buildCtx([['user', '一成系统介绍'], ['assistant', '一成系统是德国PM事业...']]);
|
||||
assertKbRoute('怎么用', ctx);
|
||||
});
|
||||
|
||||
it('聊CC套装后追问"适合谁"', () => {
|
||||
const ctx = buildCtx([['user', 'CC套装功效'], ['assistant', 'CC套装含有葡萄籽提取物...']]);
|
||||
assertKbRoute('适合谁', ctx);
|
||||
});
|
||||
|
||||
it('聊火炉原理后追问"什么意思"', () => {
|
||||
const ctx = buildCtx([['user', '火炉原理'], ['assistant', '火炉原理是PM产品的核心理念...']]);
|
||||
assertKbRoute('什么意思', ctx);
|
||||
});
|
||||
|
||||
it('聊D-Drink后追问"多少钱"', () => {
|
||||
const ctx = buildCtx([['user', 'D-Drink小绿怎么用'], ['assistant', 'D-Drink小绿是14天排毒饮料...']]);
|
||||
assertKbRoute('多少钱', ctx);
|
||||
});
|
||||
|
||||
it('聊邓白氏后追问"什么意思"', () => {
|
||||
const ctx = buildCtx([['user', '邓白氏AAA+认证'], ['assistant', '邓白氏是全球最权威的...']]);
|
||||
assertKbRoute('什么意思', ctx);
|
||||
});
|
||||
|
||||
it('聊NTC后追问"有什么好处"', () => {
|
||||
const ctx = buildCtx([['user', 'NTC营养保送系统'], ['assistant', 'NTC营养保送系统是PM的核心技术...']]);
|
||||
assertKbRoute('有什么好处', ctx);
|
||||
});
|
||||
|
||||
it('聊Hair+后追问"成分是什么"', () => {
|
||||
const ctx = buildCtx([['user', 'Hair+发宝怎么用'], ['assistant', 'Hair+包含口服发宝和外用发健...']]);
|
||||
assertKbRoute('成分是什么', ctx);
|
||||
});
|
||||
|
||||
it('用代词追问"这个产品怎么吃"', () => {
|
||||
const ctx = buildCtx([['user', '小白Restorate功效'], ['assistant', '德国PM小白Restorate的核心功效是夜间修复...']]);
|
||||
assertKbRoute('这个产品怎么吃', ctx);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 4. KB查询 + 质疑混合场景
|
||||
// ================================================================
|
||||
describe('KB查询+质疑混合 —— 用户查询后对回答产生质疑', () => {
|
||||
|
||||
it('场景1: 问基础三合一→AI说冲剂→用户纠正"不对，是胶囊"', () => {
|
||||
assertKbRoute('基础三合一怎么吃');
|
||||
const ctx = buildCtx([['user', '基础三合一怎么吃'], ['assistant', '基础三合一这样吃...']]);
|
||||
assertKbRoute('不对，是胶囊不是冲剂', ctx);
|
||||
});
|
||||
|
||||
it('场景2: 问小红功效→用户质疑"你搞错了吧"', () => {
|
||||
assertKbRoute('小红产品功效');
|
||||
const ctx = buildCtx([['user', '小红产品功效'], ['assistant', '小红Activize...']]);
|
||||
assertKbRoute('你搞错了吧', ctx);
|
||||
});
|
||||
|
||||
it('场景3: 问NTC原理→用户说"我听说不是这样的"', () => {
|
||||
assertKbRoute('NTC营养保送系统原理');
|
||||
const ctx = buildCtx([['user', 'NTC原理'], ['assistant', 'NTC营养保送系统...']]);
|
||||
assertKbRoute('我听说不是这样的', ctx);
|
||||
});
|
||||
|
||||
it('场景4: 问传销→用户要求"再查一下"', () => {
|
||||
assertKbRoute('PM是不是传销');
|
||||
const ctx = buildCtx([['user', 'PM是不是传销'], ['assistant', '德国PM不是传销...']]);
|
||||
assertKbRoute('你再查查，我看网上说法不一样', ctx);
|
||||
});
|
||||
|
||||
it('场景5: 问价格→用户质疑"太贵了，你确定吗"', () => {
|
||||
assertKbRoute('产品多少钱');
|
||||
const ctx = buildCtx([['user', '产品多少钱'], ['assistant', '产品价格因国家和地区有所不同...']]);
|
||||
assertKbRoute('你确定吗，怎么那么贵', ctx);
|
||||
});
|
||||
|
||||
it('场景6: 问CC套装→用户说"明明是乳霜不是胶囊"', () => {
|
||||
assertKbRoute('CC套装是什么');
|
||||
const ctx = buildCtx([['user', 'CC套装'], ['assistant', 'CC套装包含CC-Cell胶囊和乳霜...']]);
|
||||
assertKbRoute('明明是乳霜不是胶囊', ctx);
|
||||
});
|
||||
|
||||
it('场景7: 问好转反应→用户说"骗人的吧"', () => {
|
||||
assertKbRoute('好转反应是怎么回事');
|
||||
const ctx = buildCtx([['user', '好转反应'], ['assistant', '这是正常的好转反应...']]);
|
||||
assertKbRoute('骗人的吧，有科学依据吗', ctx);
|
||||
});
|
||||
|
||||
it('场景8: 问一成系统→用户说"跟我了解的不一样"', () => {
|
||||
assertKbRoute('一成系统介绍');
|
||||
const ctx = buildCtx([['user', '一成系统'], ['assistant', '一成系统是德国PM事业发展的智能赋能工具...']]);
|
||||
assertKbRoute('跟我了解的不一样啊', ctx);
|
||||
});
|
||||
|
||||
it('场景9: 问D-Drink→用户说"这个是泡着喝的吧"', () => {
|
||||
assertKbRoute('D-Drink怎么用');
|
||||
const ctx = buildCtx([['user', 'D-Drink怎么用'], ['assistant', 'D-Drink小绿是14天排毒饮料...']]);
|
||||
assertKbRoute('这个是泡着喝的吧', ctx);
|
||||
});
|
||||
|
||||
it('场景10: 问邓白氏→用户说"谁说的？有什么根据"', () => {
|
||||
assertKbRoute('邓白氏评级是什么');
|
||||
const ctx = buildCtx([['user', '邓白氏'], ['assistant', '邓白氏是全球最权威的商业信用评估机构...']]);
|
||||
assertKbRoute('谁说的？有什么根据', ctx);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 5. 多轮复杂对话场景（查询→追问→质疑→再查→切换话题）
|
||||
// ================================================================
|
||||
describe('多轮复杂对话 —— 模拟真实用户完整会话', () => {
|
||||
|
||||
it('5轮对话: 产品查询→追问→质疑→纠正→切换话题', () => {
|
||||
// 轮1: 直接问产品
|
||||
assertKbRoute('基础三合一是什么');
|
||||
|
||||
// 轮2: 追问怎么吃
|
||||
const ctx2 = buildCtx([['user', '基础三合一是什么'], ['assistant', '基础三合一包含大白小红小白...']]);
|
||||
assertKbRoute('怎么吃', ctx2);
|
||||
|
||||
// 轮3: 质疑回答
|
||||
const ctx3 = buildCtx([
|
||||
['user', '基础三合一是什么'], ['assistant', '基础三合一包含大白小红小白...'],
|
||||
['user', '怎么吃'], ['assistant', '大白早上空腹1平勺...'],
|
||||
]);
|
||||
assertKbRoute('你说的温度不对吧', ctx3);
|
||||
|
||||
// 轮4: 用户纠正
|
||||
assertKbRoute('应该是40度以下的水', ctx3);
|
||||
|
||||
// 轮5: 切换到完全不同的话题
|
||||
assertKbRoute('PM是不是传销');
|
||||
});
|
||||
|
||||
it('6轮对话: 系统→子功能→质疑→公司→产品→FAQ', () => {
|
||||
assertKbRoute('一成系统介绍');
|
||||
|
||||
const ctx2 = buildCtx([['user', '一成系统'], ['assistant', '一成系统是德国PM事业发展的智能赋能工具...']]);
|
||||
assertKbRoute('行动圈是什么', ctx2);
|
||||
|
||||
const ctx3 = buildCtx([['user', '一成系统'], ['assistant', '一成系统是德国PM事业发展的智能赋能工具...'], ['user', '行动圈是什么'], ['assistant', '行动圈是数字化工作室里的团队管理功能...']]);
|
||||
assertKbRoute('好像不是这样吧', ctx3);
|
||||
|
||||
assertKbRoute('德国PM公司背景');
|
||||
assertKbRoute('大白产品功效');
|
||||
assertKbRoute('孕妇能吃吗');
|
||||
});
|
||||
|
||||
it('4轮对话: 连续质疑同一个话题', () => {
|
||||
assertKbRoute('火炉原理是什么');
|
||||
|
||||
const ctx1 = buildCtx([['user', '火炉原理'], ['assistant', '火炉原理是PM产品的核心理念比喻...']]);
|
||||
assertKbRoute('不对，我记得不是这么说的', ctx1);
|
||||
assertKbRoute('你再查查，应该是另一种说法', ctx1);
|
||||
assertKbRoute('算了，到底是什么意思', ctx1);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 6. 确定性改写验证
|
||||
// ================================================================
|
||||
describe('确定性改写 —— buildDeterministicKnowledgeQuery', () => {
|
||||
|
||||
describe('产品直接改写', () => {
|
||||
const cases = [
|
||||
['基础三合一怎么吃', '基础套装'],
|
||||
['大白产品', 'Basics'],
|
||||
['小红Activize功效', 'Activize'],
|
||||
['小白Restorate成分', 'Restorate'],
|
||||
['儿童倍适怎么吃', '儿童倍适'],
|
||||
['CC套装功效', 'CC'],
|
||||
['Q10辅酵素作用', 'Q10'],
|
||||
['IB5口腔喷雾', 'IB5'],
|
||||
['D-Drink排毒', 'D-Drink'],
|
||||
];
|
||||
|
||||
for (const [query, expectContain] of cases) {
|
||||
it(`"${query}" → 改写应含"${expectContain}"`, () => {
|
||||
const result = ToolExecutor.buildDeterministicKnowledgeQuery(query, []);
|
||||
assert.ok(result, `Should have deterministic rewrite for "${query}"`);
|
||||
assert.ok(result.includes(expectContain),
|
||||
`Rewrite should contain "${expectContain}", got "${result}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('上下文追问改写', () => {
|
||||
it('上下文有"大白"时追问"怎么吃" → 改写含Basics', () => {
|
||||
const ctx = [{ role: 'assistant', content: '大白Basics是基础营养素...' }];
|
||||
const result = ToolExecutor.buildDeterministicKnowledgeQuery('怎么吃', ctx);
|
||||
assert.ok(result, 'Should rewrite');
|
||||
assert.ok(result.includes('Basics'), `Should contain Basics, got "${result}"`);
|
||||
});
|
||||
|
||||
it('上下文有"一成系统"时追问"怎么用" → 改写含一成系统', () => {
|
||||
const ctx = [{ role: 'assistant', content: '一成系统是德国PM事业发展...' }];
|
||||
const result = ToolExecutor.buildDeterministicKnowledgeQuery('怎么用', ctx);
|
||||
assert.ok(result, 'Should rewrite');
|
||||
assert.ok(result.includes('一成系统'), `Should contain 一成系统, got "${result}"`);
|
||||
});
|
||||
|
||||
it('上下文有"火炉原理"时追问"什么意思" → 改写含火炉原理', () => {
|
||||
const ctx = [{ role: 'assistant', content: '火炉原理是PM产品的核心理念...' }];
|
||||
const result = ToolExecutor.buildDeterministicKnowledgeQuery('什么意思', ctx);
|
||||
assert.ok(result === '火炉原理', `Should rewrite to 火炉原理, got "${result}"`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('无匹配时返回空', () => {
|
||||
it('"今天天气好" → 无确定性改写', () => {
|
||||
const result = ToolExecutor.buildDeterministicKnowledgeQuery('今天天气好', []);
|
||||
assert.equal(result, '', 'Chitchat should not have deterministic rewrite');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 7. 热答案匹配测试
|
||||
// ================================================================
|
||||
describe('热答案匹配 —— matchHotAnswer 验证', () => {
|
||||
// matchHotAnswer是模块内部函数，通过ToolExecutor.execute间接调用
|
||||
// 这里直接测试确定性改写+KB路由来验证热答案可达性
|
||||
|
||||
const hotTopics = [
|
||||
['基础三合一怎么吃', '基础三合一吃法'],
|
||||
['PM是不是传销', '合法性'],
|
||||
['NTC核心优势是什么', 'NTC核心优势'],
|
||||
['多久见效', '见效时间'],
|
||||
['为什么要全套搭配', '全套搭配原因'],
|
||||
['好转反应是什么', '好转反应'],
|
||||
['德国PM公司介绍', '公司介绍'],
|
||||
['小红功效', '小红功效'],
|
||||
['大白功效', '大白功效'],
|
||||
['小白功效', '小白功效'],
|
||||
['和其他保健品有什么区别', '保健品区别'],
|
||||
['CC套装怎么用', 'CC套装'],
|
||||
['Q10功效', 'Q10功效'],
|
||||
['IB5怎么用', 'IB5'],
|
||||
['邓白氏AAA+是什么', '邓白氏'],
|
||||
['一成系统是什么', '一成系统'],
|
||||
['火炉原理是什么', '火炉原理'],
|
||||
['D-Drink排毒饮怎么用', 'D-Drink'],
|
||||
['怎么加入PM', '加入方式'],
|
||||
['孕妇能吃PM产品吗', '特殊人群'],
|
||||
['多少钱', '价格'],
|
||||
];
|
||||
|
||||
for (const [query, label] of hotTopics) {
|
||||
it(`${label}: "${query}" → 应路由到KB（热答案可达）`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 8. 问题维度交叉测试（同一产品不同问法）
|
||||
// ================================================================
|
||||
describe('问题维度交叉 —— 同一产品的不同问法', () => {
|
||||
|
||||
describe('基础三合一 × 多维度', () => {
|
||||
const dimensions = [
|
||||
'基础三合一怎么吃',
|
||||
'基础三合一功效',
|
||||
'基础三合一成分',
|
||||
'基础三合一多少钱',
|
||||
'基础三合一适合谁',
|
||||
'为什么要全套搭配三合一',
|
||||
];
|
||||
for (const q of dimensions) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('小红 × 多维度', () => {
|
||||
const dimensions = [
|
||||
'小红怎么吃',
|
||||
'小红功效是什么',
|
||||
'小红成分有哪些',
|
||||
'小红副作用',
|
||||
'小红多少钱',
|
||||
'小红适合什么人',
|
||||
];
|
||||
for (const q of dimensions) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('一成系统 × 多维度', () => {
|
||||
const dimensions = [
|
||||
'一成系统是什么',
|
||||
'一成系统核心竞争力',
|
||||
'一成系统怎么用',
|
||||
'一成系统三大平台',
|
||||
'一成系统AI智能生产力',
|
||||
'一成系统线上拓客',
|
||||
'一成系统邀约话术',
|
||||
'一成系统文化',
|
||||
];
|
||||
for (const q of dimensions) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 9. 口语/ASR变体测试（语音识别的常见错误变体）
|
||||
// ================================================================
|
||||
describe('口语/ASR变体 —— 语音识别常见变体', () => {
|
||||
const asrVariants = [
|
||||
['移程系统', '一成系统ASR变体'],
|
||||
['PM细胞营养素', 'PM产品'],
|
||||
['暖炉原理', '火炉原理ASR变体'],
|
||||
['产品有哪些', '产品列表'],
|
||||
['你们公司产品', '口语化'],
|
||||
['这个东西怎么吃', '口语化追问'],
|
||||
['咱们公司介绍一下', '口语化公司'],
|
||||
];
|
||||
|
||||
for (const [query, label] of asrVariants) {
|
||||
it(`${label}: "${query}" → 应走KB路由`, () => {
|
||||
assertKbRoute(query);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 10. 边界测试
|
||||
// ================================================================
|
||||
describe('边界测试', () => {
|
||||
it('空字符串 → 不走KB', () => {
|
||||
assertNotKbRoute('');
|
||||
});
|
||||
|
||||
it('纯标点 → 不走KB', () => {
|
||||
assertNotKbRoute('？？？');
|
||||
});
|
||||
|
||||
it('单字"好" → 不走KB', () => {
|
||||
assertNotKbRoute('好');
|
||||
});
|
||||
|
||||
it('纯数字 → 不走KB', () => {
|
||||
assertNotKbRoute('123456');
|
||||
});
|
||||
|
||||
it('超长无意义文本 → 不走KB', () => {
|
||||
assertNotKbRoute('啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊');
|
||||
});
|
||||
|
||||
it('纯英文闲聊 → 不走KB', () => {
|
||||
assertNotKbRoute('hello how are you');
|
||||
});
|
||||
|
||||
it('含KB关键词但实际是闲聊的边界', () => {
|
||||
// "不对"是质疑词，会被检测为KB follow-up
|
||||
// 需要context才会真正路由到KB
|
||||
// 无context时: hasKnowledgeRouteKeyword('不对') → true (因为加了质疑词)
|
||||
// 这是预期行为：宁可多查一次KB，也不要漏掉用户质疑
|
||||
const result = shouldForceKnowledgeRoute('不对');
|
||||
assert.equal(typeof result, 'boolean', 'Should return boolean');
|
||||
});
|
||||
});
|
||||
|
||||
console.log('\n=== KB场景测试加载完成 ===\n');
|
||||
491
test2/server/tests/test_kb_scenarios_extended.js
Normal file
491
test2/server/tests/test_kb_scenarios_extended.js
Normal file
@@ -0,0 +1,491 @@
|
||||
/**
|
||||
* KB场景深度扩展测试
|
||||
* 覆盖：更多产品×维度交叉、更多确定性改写、更多追问变体、更多多轮切换、更多热答案、更多边界
|
||||
*
|
||||
* 运行方式: node --test tests/test_kb_scenarios_extended.js
|
||||
*/
|
||||
const { describe, it } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const { shouldForceKnowledgeRoute } = require('../services/realtimeDialogRouting');
|
||||
const { hasKnowledgeRouteKeyword } = require('../services/knowledgeKeywords');
|
||||
const ToolExecutor = require('../services/toolExecutor');
|
||||
|
||||
function assertKbRoute(text, ctx, msg) {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, ctx), true, msg || `"${text}" should route to KB`);
|
||||
}
|
||||
function assertNotKbRoute(text, ctx, msg) {
|
||||
assert.equal(shouldForceKnowledgeRoute(text, ctx), false, msg || `"${text}" should NOT route to KB`);
|
||||
}
|
||||
function buildCtx(pairs) {
|
||||
return pairs.map(([role, content]) => ({ role, content }));
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 1. 产品×维度全交叉 —— 每个产品 × 每个问题维度
|
||||
// ================================================================
|
||||
describe('产品×维度全交叉 —— 大白', () => {
|
||||
const dims = ['怎么吃', '功效是什么', '成分有哪些', '多少钱', '适合谁', '副作用', '哪里买', '保质期'];
|
||||
for (const d of dims) {
|
||||
it(`"大白${d}" → 应走KB`, () => { assertKbRoute(`大白${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— 小白', () => {
|
||||
const dims = ['怎么吃', '功效是什么', '成分有哪些', '多少钱', '适合谁', '副作用', '哪里买', '怎么服用'];
|
||||
for (const d of dims) {
|
||||
it(`"小白${d}" → 应走KB`, () => { assertKbRoute(`小白${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— CC套装', () => {
|
||||
const dims = ['怎么用', '功效是什么', '成分有哪些', '多少钱', '适合谁', '副作用', '区别', '包含什么'];
|
||||
for (const d of dims) {
|
||||
it(`"CC套装${d}" → 应走KB`, () => { assertKbRoute(`CC套装${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— Q10', () => {
|
||||
const dims = ['怎么吃', '功效是什么', '成分', '多少钱', '适合谁', '副作用', '怎么买', '适合什么人'];
|
||||
for (const d of dims) {
|
||||
it(`"Q10${d}" → 应走KB`, () => { assertKbRoute(`Q10${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— IB5', () => {
|
||||
const dims = ['怎么用', '功效是什么', '成分', '多少钱', '适合谁', '副作用', '什么时候用', '适合什么人'];
|
||||
for (const d of dims) {
|
||||
it(`"IB5${d}" → 应走KB`, () => { assertKbRoute(`IB5${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— D-Drink', () => {
|
||||
const dims = ['怎么用', '功效是什么', '成分', '多少钱', '适合谁', '副作用', '排毒原理', '喝法'];
|
||||
for (const d of dims) {
|
||||
it(`"D-Drink${d}" → 应走KB`, () => { assertKbRoute(`D-Drink${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— Hair+', () => {
|
||||
const dims = ['怎么用', '功效是什么', '成分', '多少钱', '适合谁', '副作用'];
|
||||
for (const d of dims) {
|
||||
it(`"Hair+${d}" → 应走KB`, () => { assertKbRoute(`Hair+${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— 儿童倍适', () => {
|
||||
const dims = ['怎么吃', '功效是什么', '成分', '多少钱', '适合几岁', '副作用', '适合什么人', '口味'];
|
||||
for (const d of dims) {
|
||||
it(`"儿童倍适${d}" → 应走KB`, () => { assertKbRoute(`儿童倍适${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('产品×维度全交叉 —— 关节套装', () => {
|
||||
const dims = ['怎么用', '功效是什么', '成分', '多少钱', '适合谁', '副作用'];
|
||||
for (const d of dims) {
|
||||
it(`"关节套装${d}" → 应走KB`, () => { assertKbRoute(`关节套装${d}`); });
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 2. 确定性改写深度覆盖 —— 更多产品规则
|
||||
// ================================================================
|
||||
describe('确定性改写深度 —— 更多产品规则', () => {
|
||||
|
||||
describe('一成系统及子话题', () => {
|
||||
const cases = [
|
||||
['一成系统是什么', '一成系统'],
|
||||
['一成系统怎么用', '一成系统'],
|
||||
['一成系统三大平台', '一成系统'],
|
||||
['一成系统行动圈', '一成系统'],
|
||||
['身未动梦已成', '一成系统'],
|
||||
['一部手机做天下', '一成系统'],
|
||||
['如何发展PM事业', '一成系统'],
|
||||
];
|
||||
for (const [q, expect] of cases) {
|
||||
it(`"${q}" → 改写含"${expect}"`, () => {
|
||||
const r = ToolExecutor.buildDeterministicKnowledgeQuery(q, []);
|
||||
assert.ok(r && r.includes(expect), `"${q}" → got "${r}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('PM公司相关', () => {
|
||||
const cases = [
|
||||
['德国PM公司介绍', 'PM'],
|
||||
['PM公司背景', 'PM'],
|
||||
['PM是不是传销', 'PM'],
|
||||
['PM公司合法吗', 'PM'],
|
||||
];
|
||||
for (const [q, expect] of cases) {
|
||||
it(`"${q}" → 改写含"${expect}"`, () => {
|
||||
const r = ToolExecutor.buildDeterministicKnowledgeQuery(q, []);
|
||||
assert.ok(r && r.includes(expect), `"${q}" → got "${r}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('NTC/火炉原理/阿育吠陀', () => {
|
||||
const cases = [
|
||||
['NTC营养保送系统是什么', 'NTC'],
|
||||
['NTC核心优势', 'NTC'],
|
||||
['火炉原理', '火炉原理'],
|
||||
['阿育吠陀是什么', '阿育吠陀'],
|
||||
];
|
||||
for (const [q, expect] of cases) {
|
||||
it(`"${q}" → 改写含"${expect}"`, () => {
|
||||
const r = ToolExecutor.buildDeterministicKnowledgeQuery(q, []);
|
||||
assert.ok(r && r.includes(expect), `"${q}" → got "${r}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('上下文追问改写 —— 各产品', () => {
|
||||
const ctxCases = [
|
||||
[{ role: 'assistant', content: '小红Activize是...' }, '成分是什么', 'Activize'],
|
||||
[{ role: 'assistant', content: '小白Restorate帮助修复...' }, '适合谁', 'Restorate'],
|
||||
[{ role: 'assistant', content: 'CC套装含有CC-Cell葡萄籽精华胶囊和乳霜' }, '怎么吃', 'CC'],
|
||||
[{ role: 'assistant', content: 'Hair+发宝防脱发口服发宝外用发健' }, '功效', 'Hair'],
|
||||
[{ role: 'assistant', content: '儿童倍适PowerCocktail Junior适合小朋友' }, '怎么用', '儿童倍适'],
|
||||
[{ role: 'assistant', content: 'D-Drink小绿排毒饮是14天排毒方案' }, '功效是什么', 'D-Drink'],
|
||||
[{ role: 'assistant', content: 'Apple Antioxy Zellschutz细胞抗氧素是独立小袋包装' }, '他的规格是什么', 'Apple Antioxy'],
|
||||
[{ role: 'assistant', content: '小白Restorate建议睡前空腹服用' }, '它一天几次', 'Restorate'],
|
||||
];
|
||||
for (const [ctxMsg, query, expect] of ctxCases) {
|
||||
it(`上下文"${ctxMsg.content.slice(0, 10)}..."追问"${query}" → 含"${expect}"`, () => {
|
||||
const r = ToolExecutor.buildDeterministicKnowledgeQuery(query, [ctxMsg]);
|
||||
assert.ok(r && r.includes(expect), `Got "${r}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('无匹配场景', () => {
|
||||
const noMatch = ['你好', '天气怎么样', '讲个故事', '几点了', '我要听音乐'];
|
||||
for (const q of noMatch) {
|
||||
it(`"${q}" → 无改写`, () => {
|
||||
const r = ToolExecutor.buildDeterministicKnowledgeQuery(q, []);
|
||||
assert.equal(r, '', `"${q}" should not rewrite, got "${r}"`);
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 3. 追问变体全覆盖 —— 各种追问句式 × 不同产品上下文
|
||||
// ================================================================
|
||||
describe('追问变体全覆盖 —— 不同追问句式', () => {
|
||||
const followUpPatterns = [
|
||||
'怎么吃', '怎么用', '功效是什么', '成分是什么', '多少钱',
|
||||
'适合谁', '哪里买', '什么意思', '有什么好处', '怎么服用',
|
||||
'详细说说', '介绍一下', '继续说', '展开说说', '配方',
|
||||
'原理是什么', '适合什么人', '怎么买', '具体内容',
|
||||
];
|
||||
|
||||
const ctxProducts = [
|
||||
buildCtx([['user', '大白产品'], ['assistant', '大白Basics...']]),
|
||||
buildCtx([['user', '小红功效'], ['assistant', '小红Activize...']]),
|
||||
buildCtx([['user', '一成系统'], ['assistant', '一成系统是...']]),
|
||||
];
|
||||
|
||||
for (let pi = 0; pi < ctxProducts.length; pi++) {
|
||||
for (const fup of followUpPatterns) {
|
||||
it(`ctx${pi + 1} + "${fup}" → 应走KB`, () => {
|
||||
assertKbRoute(fup, ctxProducts[pi]);
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 4. 多KB切换 —— 更多组合
|
||||
// ================================================================
|
||||
describe('多KB切换 —— 更多组合模式', () => {
|
||||
|
||||
it('产品→FAQ→科学: 小红→副作用→NTC原理', () => {
|
||||
assertKbRoute('小红有什么副作用');
|
||||
assertKbRoute('PM产品有副作用吗');
|
||||
assertKbRoute('NTC营养保送系统原理');
|
||||
});
|
||||
|
||||
it('事业→公司→产品→FAQ: 招商→PM背景→大白→见效时间', () => {
|
||||
assertKbRoute('招商代理政策');
|
||||
assertKbRoute('德国PM公司背景');
|
||||
assertKbRoute('大白怎么吃');
|
||||
assertKbRoute('多久见效');
|
||||
});
|
||||
|
||||
it('科学→产品→产品→产品: NTC→小红→小白→CC', () => {
|
||||
assertKbRoute('NTC是什么');
|
||||
assertKbRoute('小红功效');
|
||||
assertKbRoute('小白成分');
|
||||
assertKbRoute('CC套装怎么用');
|
||||
});
|
||||
|
||||
it('FAQ→FAQ→FAQ: 传销→副作用→见效→全套搭配', () => {
|
||||
assertKbRoute('PM是传销吗');
|
||||
assertKbRoute('PM产品有副作用吗');
|
||||
assertKbRoute('多久能见效');
|
||||
assertKbRoute('为什么要全套搭配');
|
||||
});
|
||||
|
||||
it('系统→事业→认证: 一成系统→做PM→邓白氏', () => {
|
||||
assertKbRoute('一成系统介绍');
|
||||
assertKbRoute('怎么做PM事业');
|
||||
assertKbRoute('邓白氏AAA+认证');
|
||||
});
|
||||
|
||||
it('产品→追问→切换→追问: 大白→怎么吃→小红→功效', () => {
|
||||
assertKbRoute('大白是什么');
|
||||
const ctx1 = buildCtx([['user', '大白'], ['assistant', '大白Basics...']]);
|
||||
assertKbRoute('怎么吃', ctx1);
|
||||
assertKbRoute('小红产品介绍');
|
||||
const ctx2 = buildCtx([['user', '小红'], ['assistant', '小红Activize...']]);
|
||||
assertKbRoute('功效是什么', ctx2);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 5. 质疑+追问混合 —— 更多组合场景
|
||||
// ================================================================
|
||||
describe('质疑+追问混合 —— 更多组合', () => {
|
||||
|
||||
it('质疑后继续追问详情', () => {
|
||||
assertKbRoute('大白功效');
|
||||
const ctx = buildCtx([['user', '大白功效'], ['assistant', '大白Basics帮助...']]);
|
||||
assertKbRoute('你搞错了吧', ctx);
|
||||
assertKbRoute('那正确的功效到底是什么', ctx);
|
||||
});
|
||||
|
||||
it('追问后发现错误再质疑', () => {
|
||||
assertKbRoute('CC套装包含什么');
|
||||
const ctx = buildCtx([['user', 'CC套装'], ['assistant', 'CC套装含有...']]);
|
||||
assertKbRoute('具体成分是什么', ctx);
|
||||
assertKbRoute('说的有问题，我记得不是这个成分', ctx);
|
||||
});
|
||||
|
||||
it('质疑→纠正→再追问另一个维度', () => {
|
||||
const ctx = buildCtx([['user', '小红怎么吃'], ['assistant', '小红冲服...']]);
|
||||
assertKbRoute('不对，不是冲着喝的', ctx);
|
||||
assertKbRoute('小红到底是什么剂型', ctx);
|
||||
assertKbRoute('小红适合什么人群', ctx);
|
||||
});
|
||||
|
||||
it('连续切换产品并质疑', () => {
|
||||
assertKbRoute('大白功效');
|
||||
assertKbRoute('你说的不对');
|
||||
assertKbRoute('小红功效');
|
||||
assertKbRoute('也不对吧');
|
||||
assertKbRoute('小白功效');
|
||||
assertKbRoute('说的不准确');
|
||||
});
|
||||
|
||||
it('先闲聊再切到KB质疑', () => {
|
||||
assertNotKbRoute('今天心情不错');
|
||||
assertKbRoute('对了基础三合一怎么吃');
|
||||
const ctx = buildCtx([['user', '基础三合一'], ['assistant', '大白小红小白...']]);
|
||||
assertKbRoute('不是这样吧', ctx);
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 6. 热答案可达性 —— 更多热门问题变体
|
||||
// ================================================================
|
||||
describe('热答案可达性 —— 问题变体', () => {
|
||||
const hotVariants = [
|
||||
// 基础三合一吃法的各种问法
|
||||
'基础三合一怎么吃', '基础三合一的吃法', '基础套装怎么服用',
|
||||
'大白小红小白怎么吃',
|
||||
// 传销/合法性的各种问法
|
||||
'PM是不是传销', 'PM合法吗', 'PM是传销吗', 'PM正规吗',
|
||||
// NTC
|
||||
'NTC是什么', 'NTC核心优势是什么', 'NTC营养保送系统',
|
||||
// 见效时间
|
||||
'多久见效', '多久能见效', '吃多久有效果',
|
||||
// 好转反应
|
||||
'好转反应', '好转反应是什么', '吃了不舒服正常吗',
|
||||
// 公司
|
||||
'德国PM公司介绍', 'PM公司背景', 'PM公司怎么样',
|
||||
// 价格
|
||||
'多少钱', '产品价格', '产品贵不贵',
|
||||
];
|
||||
|
||||
for (const q of hotVariants) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 7. 口语化查询 —— 更多自然说法
|
||||
// ================================================================
|
||||
describe('口语化查询 —— 真实用户的自然说法', () => {
|
||||
const colloquial = [
|
||||
'你们公司是做什么的',
|
||||
'你们那个产品怎么吃',
|
||||
'咱们这个东西多少钱',
|
||||
'那个什么三合一是什么',
|
||||
'帮我介绍一下你们产品',
|
||||
'我想了解PM产品',
|
||||
'说说你们公司',
|
||||
'讲讲那个什么系统',
|
||||
'查查那个产品',
|
||||
'帮我查一下基础三合一',
|
||||
'帮我问一下价格',
|
||||
'你们产品正规吗',
|
||||
'你们那个东西靠谱吗',
|
||||
'说说那个什么功效',
|
||||
'我想知道怎么加入',
|
||||
'你们卖的是什么东西',
|
||||
'健康产品有哪些',
|
||||
'帮我看看成分',
|
||||
'你们的东西有什么用',
|
||||
'咱吃这个有好处吗',
|
||||
];
|
||||
|
||||
for (const q of colloquial) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 8. 负面边界 —— 更多不应走KB的场景
|
||||
// ================================================================
|
||||
describe('负面边界 —— 更多不应走KB的场景', () => {
|
||||
const negatives = [
|
||||
'',
|
||||
' ',
|
||||
'？',
|
||||
'！！！',
|
||||
'。。。',
|
||||
'好',
|
||||
'嗯',
|
||||
'哦',
|
||||
'行',
|
||||
'啊',
|
||||
'对',
|
||||
'是的',
|
||||
'好的好的',
|
||||
'知道了知道了',
|
||||
'哈哈哈哈',
|
||||
'嘻嘻',
|
||||
'666',
|
||||
'999',
|
||||
'111',
|
||||
'早',
|
||||
'晚安',
|
||||
'ok',
|
||||
'OK',
|
||||
'hello',
|
||||
'hi',
|
||||
'bye',
|
||||
'good',
|
||||
'今天天气真好',
|
||||
'我要睡觉了',
|
||||
'你是AI吗',
|
||||
'你能做什么',
|
||||
];
|
||||
|
||||
for (const text of negatives) {
|
||||
it(`"${text}" → 不应走KB`, () => { assertNotKbRoute(text); });
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 9. 关键词嵌入长句 —— 确保关键词在句中也能命中
|
||||
// ================================================================
|
||||
describe('关键词嵌入长句 —— 子串匹配验证', () => {
|
||||
const embedded = [
|
||||
['我想了解一下基础三合一的功效', '基础三合一嵌入'],
|
||||
['请问德国PM公司在哪个城市', '公司嵌入'],
|
||||
['你能帮我查查一成系统怎么用吗', '一成系统嵌入'],
|
||||
['我朋友推荐我吃小红你能介绍下吗', '小红嵌入'],
|
||||
['听说有个叫NTC的营养保送系统', 'NTC嵌入'],
|
||||
['好转反应的话应该怎么处理', '好转反应嵌入'],
|
||||
['孕妇是不是不能吃PM的产品', '孕妇嵌入'],
|
||||
['为什么说要全套搭配使用呢', '全套搭配嵌入'],
|
||||
['听人家说邓白氏AAA+评级很厉害', '邓白氏嵌入'],
|
||||
['想问一下儿童倍适几岁能吃', '儿童倍适嵌入'],
|
||||
['我对火炉原理很感兴趣', '火炉原理嵌入'],
|
||||
['请介绍一下CC套装的功效', 'CC套装嵌入'],
|
||||
['Q10辅酵素是做什么用的', 'Q10嵌入'],
|
||||
['Hair+发宝真的能防脱吗', 'Hair+嵌入'],
|
||||
['想知道D-Drink排毒的原理', 'D-Drink嵌入'],
|
||||
];
|
||||
|
||||
for (const [text, label] of embedded) {
|
||||
it(`${label}: "${text}" → 应走KB`, () => { assertKbRoute(text); });
|
||||
}
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 10. 同义问法覆盖 —— 相同意图不同表达
|
||||
// ================================================================
|
||||
describe('同义问法 —— 相同意图不同表达', () => {
|
||||
|
||||
describe('询问功效的N种方式', () => {
|
||||
const efficacyCases = [
|
||||
'大白有什么功效', '大白的作用是什么', '大白有什么用',
|
||||
'大白好在哪', '吃大白有什么好处',
|
||||
];
|
||||
for (const q of efficacyCases) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('询问用法的N种方式', () => {
|
||||
const usageCases = [
|
||||
'小红怎么吃', '小红怎么服用', '小红怎么用',
|
||||
'小红的吃法', '小红用法',
|
||||
];
|
||||
for (const q of usageCases) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('询问价格的N种方式', () => {
|
||||
const priceCases = [
|
||||
'基础三合一多少钱', '基础三合一价格', '基础三合一贵不贵',
|
||||
'产品多少钱', '产品价格表',
|
||||
];
|
||||
for (const q of priceCases) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('询问合法性的N种方式', () => {
|
||||
const legalCases = [
|
||||
'PM是不是传销', 'PM正规吗', 'PM合法吗',
|
||||
'PM是传销吗', 'PM是直销还是传销', 'PM靠谱吗',
|
||||
];
|
||||
for (const q of legalCases) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
|
||||
describe('询问公司的N种方式', () => {
|
||||
const companyCases = [
|
||||
'PM公司介绍', '德国PM公司怎么样',
|
||||
'PM公司成立多久了', 'PM公司实力如何',
|
||||
];
|
||||
for (const q of companyCases) {
|
||||
it(`"${q}" → 应走KB`, () => { assertKbRoute(q); });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ================================================================
|
||||
// 11. 特殊人群×产品交叉
|
||||
// ================================================================
|
||||
describe('特殊人群×产品 —— 能不能吃的场景', () => {
|
||||
const groups = ['孕妇', '小孩', '老人', '糖尿病人', '高血压'];
|
||||
const products = ['大白', '小红', '基础三合一', 'PM产品'];
|
||||
|
||||
for (const g of groups) {
|
||||
for (const p of products) {
|
||||
it(`"${g}能吃${p}吗" → 应走KB`, () => {
|
||||
assertKbRoute(`${g}能吃${p}吗`);
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
console.log('\n=== KB场景扩展测试加载完成 ===\n');
|
||||
112
test2/server/tests/test_real_viking_kb.js
Normal file
112
test2/server/tests/test_real_viking_kb.js
Normal file
@@ -0,0 +1,112 @@
|
||||
const path = require('path');
|
||||
const { performance } = require('perf_hooks');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
const ToolExecutor = require('../services/toolExecutor');
|
||||
|
||||
const realTestQueries = [
|
||||
{ name: 'Product - Q10', query: 'Q10辅酵素氧修护有什么作用' },
|
||||
{ name: 'Product - IB5', query: 'IB5口腔免疫喷雾怎么使用' },
|
||||
{ name: 'Product - CC胶囊', query: 'CC胶囊适合什么人使用' },
|
||||
{ name: 'Company - 邓白氏', query: '德国PM的邓白氏认证是多少分' },
|
||||
{ name: 'Technology - 火炉原理', query: '请详细解释一下火炉原理' },
|
||||
{ name: 'Training - 新人起步', query: '培训新人起步三关是什么' },
|
||||
{ name: 'Product - 关节套装', query: '关节套装有什么功效' },
|
||||
{ name: 'Company - 培安烟台', query: '培安烟台是什么' },
|
||||
{ name: 'Product - 儿童倍适', query: '儿童倍适有什么成分' },
|
||||
{ name: 'Science - 阿育吠陀', query: '阿育吠陀医学原理是什么' }
|
||||
];
|
||||
|
||||
async function runRealKBTest() {
|
||||
console.log('='.repeat(80));
|
||||
console.log('VIKING REAL KNOWLEDGE BASE PERFORMANCE TEST');
|
||||
console.log('='.repeat(80));
|
||||
console.log('');
|
||||
|
||||
console.log('Warming up...');
|
||||
for (let i = 0; i < 2; i++) {
|
||||
for (const q of realTestQueries) {
|
||||
try {
|
||||
await ToolExecutor.searchKnowledge({ query: q.query, response_mode: 'answer' }, []);
|
||||
} catch (e) {}
|
||||
}
|
||||
}
|
||||
console.log('Warmup complete!\n');
|
||||
|
||||
console.log('Running latency tests (5 iterations each)...\n');
|
||||
|
||||
const allResults = [];
|
||||
|
||||
for (const { name, query } of realTestQueries) {
|
||||
console.log(`Testing: ${name}`);
|
||||
const latencies = [];
|
||||
const hits = [];
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const start = performance.now();
|
||||
let result;
|
||||
try {
|
||||
result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
latencies.push(latency);
|
||||
hits.push(!!result.hit);
|
||||
console.log(` Iteration ${i + 1}: ${latency.toFixed(2)}ms, hit=${result.hit}, source=${result.source}`);
|
||||
} catch (e) {
|
||||
console.log(` Iteration ${i + 1} error: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
const avgLatency = latencies.length ? latencies.reduce((a, b) => a + b, 0) / latencies.length : 0;
|
||||
const p50 = percentile(latencies, 50);
|
||||
const p95 = percentile(latencies, 95);
|
||||
const hitRate = hits.length ? hits.filter(h => h).length / hits.length : 0;
|
||||
|
||||
allResults.push({
|
||||
name,
|
||||
query,
|
||||
avgLatency,
|
||||
p50,
|
||||
p95,
|
||||
hitRate,
|
||||
latencies
|
||||
});
|
||||
|
||||
console.log(` → Avg: ${avgLatency.toFixed(2)}ms, P95: ${p95.toFixed(2)}ms, Hit Rate: ${(hitRate * 100).toFixed(1)}%\n`);
|
||||
}
|
||||
|
||||
console.log('='.repeat(80));
|
||||
console.log('SUMMARY');
|
||||
console.log('='.repeat(80));
|
||||
|
||||
const totalAvg = allResults.reduce((a, b) => a + b.avgLatency, 0) / allResults.length;
|
||||
const totalP95 = percentile(allResults.flatMap(r => r.latencies), 95);
|
||||
const totalHitRate = allResults.reduce((a, b) => a + b.hitRate, 0) / allResults.length;
|
||||
|
||||
console.log(`\nOverall Average Latency: ${totalAvg.toFixed(2)}ms`);
|
||||
console.log(`Overall P95 Latency: ${totalP95.toFixed(2)}ms`);
|
||||
console.log(`Overall Hit Rate: ${(totalHitRate * 100).toFixed(1)}%`);
|
||||
|
||||
console.log('\nTop 3 fastest queries:');
|
||||
allResults.sort((a, b) => a.avgLatency - b.avgLatency).slice(0, 3).forEach((r, i) => {
|
||||
console.log(` ${i + 1}. ${r.name}: ${r.avgLatency.toFixed(2)}ms`);
|
||||
});
|
||||
|
||||
console.log('\nTop 3 slowest queries:');
|
||||
allResults.sort((a, b) => b.avgLatency - a.avgLatency).slice(0, 3).forEach((r, i) => {
|
||||
console.log(` ${i + 1}. ${r.name}: ${r.avgLatency.toFixed(2)}ms`);
|
||||
});
|
||||
|
||||
console.log('\n' + '='.repeat(80));
|
||||
|
||||
return allResults;
|
||||
}
|
||||
|
||||
function percentile(arr, p) {
|
||||
if (arr.length === 0) return 0;
|
||||
const sorted = [...arr].sort((a, b) => a - b);
|
||||
const index = Math.ceil((p / 100) * sorted.length) - 1;
|
||||
return sorted[Math.max(0, index)];
|
||||
}
|
||||
|
||||
runRealKBTest().catch(console.error);
|
||||
@@ -0,0 +1,498 @@
|
||||
{
|
||||
"generatedAt": "2026-03-20T07:41:01.552Z",
|
||||
"mockMode": true,
|
||||
"summary": {
|
||||
"latency": {
|
||||
"Product Query - Xiaohong": {
|
||||
"avg": "221.59",
|
||||
"p95": "313.74",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Product Query - Dabai": {
|
||||
"avg": "173.13",
|
||||
"p95": "291.55",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Company Info": {
|
||||
"avg": "176.99",
|
||||
"p95": "244.86",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"NTC Technology": {
|
||||
"avg": "197.35",
|
||||
"p95": "295.00",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Hot Answer": {
|
||||
"avg": "209.49",
|
||||
"p95": "279.63",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"No Hit Query": {
|
||||
"avg": "204.48",
|
||||
"p95": "296.13",
|
||||
"hitRate": "0.0%"
|
||||
}
|
||||
},
|
||||
"cache": [
|
||||
{
|
||||
"name": "Product Query - Xiaohong",
|
||||
"speedup": "0.50x"
|
||||
},
|
||||
{
|
||||
"name": "Product Query - Dabai",
|
||||
"speedup": "0.79x"
|
||||
},
|
||||
{
|
||||
"name": "Company Info",
|
||||
"speedup": "0.86x"
|
||||
}
|
||||
],
|
||||
"concurrency": {
|
||||
"1": {
|
||||
"throughput": "8.00 req/s",
|
||||
"successRate": "100.0%"
|
||||
},
|
||||
"3": {
|
||||
"throughput": "11.41 req/s",
|
||||
"successRate": "100.0%"
|
||||
},
|
||||
"5": {
|
||||
"throughput": "16.96 req/s",
|
||||
"successRate": "100.0%"
|
||||
}
|
||||
}
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"type": "latency",
|
||||
"timestamp": "2026-03-20T07:40:56.968Z",
|
||||
"iterations": 10,
|
||||
"mockMode": true,
|
||||
"results": {
|
||||
"Product Query - Xiaohong": {
|
||||
"query": "小红产品有什么功效",
|
||||
"latencies": [
|
||||
215.31479999999965,
|
||||
248.87959999999975,
|
||||
313.73730000000023,
|
||||
216.14289999999983,
|
||||
107.47650000000021,
|
||||
122.7501999999995,
|
||||
185.82290000000012,
|
||||
216.10750000000007,
|
||||
294.0571,
|
||||
295.65119999999933
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 221.59399999999988,
|
||||
"minLatency": 107.47650000000021,
|
||||
"maxLatency": 313.73730000000023,
|
||||
"p50Latency": 216.10750000000007,
|
||||
"p95Latency": 313.73730000000023,
|
||||
"p99Latency": 313.73730000000023,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Product Query - Dabai": {
|
||||
"query": "大白产品怎么吃",
|
||||
"latencies": [
|
||||
108.59429999999975,
|
||||
109.21360000000004,
|
||||
189.07920000000013,
|
||||
141.89410000000044,
|
||||
264.7583999999997,
|
||||
123.83699999999953,
|
||||
291.5468000000001,
|
||||
188.14480000000003,
|
||||
203.59680000000026,
|
||||
110.61819999999989
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 173.12831999999997,
|
||||
"minLatency": 108.59429999999975,
|
||||
"maxLatency": 291.5468000000001,
|
||||
"p50Latency": 141.89410000000044,
|
||||
"p95Latency": 291.5468000000001,
|
||||
"p99Latency": 291.5468000000001,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Company Info": {
|
||||
"query": "德国PM公司介绍",
|
||||
"latencies": [
|
||||
122.88140000000021,
|
||||
140.60269999999946,
|
||||
170.5096999999996,
|
||||
140.1189000000004,
|
||||
187.21970000000056,
|
||||
218.0506000000005,
|
||||
220.02800000000025,
|
||||
244.85799999999927,
|
||||
124.36050000000068,
|
||||
201.2597999999998
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 176.98893000000007,
|
||||
"minLatency": 122.88140000000021,
|
||||
"maxLatency": 244.85799999999927,
|
||||
"p50Latency": 170.5096999999996,
|
||||
"p95Latency": 244.85799999999927,
|
||||
"p99Latency": 244.85799999999927,
|
||||
"hitRate": 1
|
||||
},
|
||||
"NTC Technology": {
|
||||
"query": "NTC营养保送系统原理",
|
||||
"latencies": [
|
||||
201.2448000000004,
|
||||
294.9982,
|
||||
123.97750000000087,
|
||||
217.60050000000047,
|
||||
250.01519999999982,
|
||||
154.38770000000113,
|
||||
123.49189999999908,
|
||||
294.09230000000025,
|
||||
140.3801999999996,
|
||||
173.35200000000077
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 197.35403000000025,
|
||||
"minLatency": 123.49189999999908,
|
||||
"maxLatency": 294.9982,
|
||||
"p50Latency": 173.35200000000077,
|
||||
"p95Latency": 294.9982,
|
||||
"p99Latency": 294.9982,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Hot Answer": {
|
||||
"query": "基础三合一怎么吃",
|
||||
"latencies": [
|
||||
185.3747000000003,
|
||||
279.6288999999997,
|
||||
139.79800000000068,
|
||||
278.0802999999996,
|
||||
124.75540000000001,
|
||||
278.8053999999993,
|
||||
139.687100000001,
|
||||
234.39990000000034,
|
||||
202.0474000000013,
|
||||
232.36249999999927
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 209.49396000000016,
|
||||
"minLatency": 124.75540000000001,
|
||||
"maxLatency": 279.6288999999997,
|
||||
"p50Latency": 202.0474000000013,
|
||||
"p95Latency": 279.6288999999997,
|
||||
"p99Latency": 279.6288999999997,
|
||||
"hitRate": 1
|
||||
},
|
||||
"No Hit Query": {
|
||||
"query": "今天天气怎么样",
|
||||
"latencies": [
|
||||
216.4937000000009,
|
||||
202.85559999999896,
|
||||
122.8827999999994,
|
||||
248.70260000000053,
|
||||
186.36679999999978,
|
||||
140.94870000000083,
|
||||
296.1263999999992,
|
||||
246.18019999999888,
|
||||
230.27289999999994,
|
||||
153.9431000000004
|
||||
],
|
||||
"hits": [
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
],
|
||||
"avgLatency": 204.4772799999999,
|
||||
"minLatency": 122.8827999999994,
|
||||
"maxLatency": 296.1263999999992,
|
||||
"p50Latency": 202.85559999999896,
|
||||
"p95Latency": 296.1263999999992,
|
||||
"p99Latency": 296.1263999999992,
|
||||
"hitRate": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "cache",
|
||||
"timestamp": "2026-03-20T07:41:00.864Z",
|
||||
"cacheHitsIterations": 5,
|
||||
"mockMode": true,
|
||||
"results": [
|
||||
{
|
||||
"name": "Product Query - Xiaohong",
|
||||
"query": "小红产品有什么功效",
|
||||
"firstHitLatency": 126.06420000000071,
|
||||
"cacheHitLatencies": [
|
||||
248.09699999999975,
|
||||
292.3535000000011,
|
||||
279.5563999999995,
|
||||
278.85109999999986,
|
||||
168.8062000000009
|
||||
],
|
||||
"avgCacheLatency": 253.53284000000022,
|
||||
"speedup": 0.49723026019035876,
|
||||
"firstHit": {
|
||||
"query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Product Query - Dabai",
|
||||
"query": "大白产品怎么吃",
|
||||
"firstHitLatency": 171.05249999999796,
|
||||
"cacheHitLatencies": [
|
||||
282.6729000000014,
|
||||
219.4409999999989,
|
||||
218.62560000000303,
|
||||
139.14129999999932,
|
||||
216.64949999999953
|
||||
],
|
||||
"avgCacheLatency": 215.30606000000043,
|
||||
"speedup": 0.7944620787728762,
|
||||
"firstHit": {
|
||||
"query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM大白Basics是基础营养素，早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Company Info",
|
||||
"query": "德国PM公司介绍",
|
||||
"firstHitLatency": 183.53129999999874,
|
||||
"cacheHitLatencies": [
|
||||
292.8653000000013,
|
||||
107.31589999999778,
|
||||
233.82210000000123,
|
||||
262.02170000000115,
|
||||
170.1095000000023
|
||||
],
|
||||
"avgCacheLatency": 213.22690000000074,
|
||||
"speedup": 0.8607323935206961,
|
||||
"firstHit": {
|
||||
"query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "concurrency",
|
||||
"timestamp": "2026-03-20T07:41:01.549Z",
|
||||
"concurrencyLevels": [
|
||||
1,
|
||||
3,
|
||||
5
|
||||
],
|
||||
"mockMode": true,
|
||||
"results": {
|
||||
"1": {
|
||||
"concurrency": 1,
|
||||
"totalTime": 125.04439999999886,
|
||||
"throughput": 7.997159408978004,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
]
|
||||
},
|
||||
"3": {
|
||||
"concurrency": 3,
|
||||
"totalTime": 262.9516000000003,
|
||||
"throughput": 11.408943699144618,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM大白Basics是基础营养素，早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
]
|
||||
},
|
||||
"5": {
|
||||
"concurrency": 5,
|
||||
"totalTime": 294.8380999999972,
|
||||
"throughput": 16.95845957493298,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM大白Basics是基础营养素，早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "模拟知识库结果",
|
||||
"content": "德国PM大白Basics是基础营养素，早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"hit": true,
|
||||
"source": "mock_knowledge"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,570 @@
|
||||
{
|
||||
"generatedAt": "2026-03-20T07:49:07.093Z",
|
||||
"mockMode": false,
|
||||
"summary": {
|
||||
"latency": {
|
||||
"Product Query - Xiaohong": {
|
||||
"avg": "0.15",
|
||||
"p95": "0.22",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Product Query - Dabai": {
|
||||
"avg": "0.15",
|
||||
"p95": "0.17",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Company Info": {
|
||||
"avg": "0.13",
|
||||
"p95": "0.15",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"NTC Technology": {
|
||||
"avg": "0.13",
|
||||
"p95": "0.19",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"Hot Answer": {
|
||||
"avg": "0.13",
|
||||
"p95": "0.14",
|
||||
"hitRate": "100.0%"
|
||||
},
|
||||
"No Hit Query": {
|
||||
"avg": "0.66",
|
||||
"p95": "0.87",
|
||||
"hitRate": "100.0%"
|
||||
}
|
||||
},
|
||||
"cache": [
|
||||
{
|
||||
"name": "Product Query - Xiaohong",
|
||||
"speedup": "1.11x"
|
||||
},
|
||||
{
|
||||
"name": "Product Query - Dabai",
|
||||
"speedup": "0.99x"
|
||||
},
|
||||
{
|
||||
"name": "Company Info",
|
||||
"speedup": "1.25x"
|
||||
}
|
||||
],
|
||||
"concurrency": {
|
||||
"1": {
|
||||
"throughput": "6016.85 req/s",
|
||||
"successRate": "100.0%"
|
||||
},
|
||||
"3": {
|
||||
"throughput": "6426.74 req/s",
|
||||
"successRate": "100.0%"
|
||||
},
|
||||
"5": {
|
||||
"throughput": "6439.15 req/s",
|
||||
"successRate": "100.0%"
|
||||
}
|
||||
}
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"type": "latency",
|
||||
"timestamp": "2026-03-20T07:49:07.084Z",
|
||||
"iterations": 10,
|
||||
"mockMode": false,
|
||||
"results": {
|
||||
"Product Query - Xiaohong": {
|
||||
"query": "小红产品有什么功效",
|
||||
"latencies": [
|
||||
0.2159999999998945,
|
||||
0.17509999999992942,
|
||||
0.1505000000001928,
|
||||
0.14269999999987704,
|
||||
0.15470000000004802,
|
||||
0.1368999999999687,
|
||||
0.12730000000010477,
|
||||
0.13599999999996726,
|
||||
0.13009999999985666,
|
||||
0.15689999999995052
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.15261999999997897,
|
||||
"minLatency": 0.12730000000010477,
|
||||
"maxLatency": 0.2159999999998945,
|
||||
"p50Latency": 0.14269999999987704,
|
||||
"p95Latency": 0.2159999999998945,
|
||||
"p99Latency": 0.2159999999998945,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Product Query - Dabai": {
|
||||
"query": "大白产品怎么吃",
|
||||
"latencies": [
|
||||
0.15099999999983993,
|
||||
0.14449999999987995,
|
||||
0.1378999999997177,
|
||||
0.13919999999961874,
|
||||
0.171100000000024,
|
||||
0.14480000000003201,
|
||||
0.14390000000003056,
|
||||
0.14840000000003783,
|
||||
0.14489999999977954,
|
||||
0.13879999999971915
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.14644999999986794,
|
||||
"minLatency": 0.1378999999997177,
|
||||
"maxLatency": 0.171100000000024,
|
||||
"p50Latency": 0.14449999999987995,
|
||||
"p95Latency": 0.171100000000024,
|
||||
"p99Latency": 0.171100000000024,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Company Info": {
|
||||
"query": "德国PM公司介绍",
|
||||
"latencies": [
|
||||
0.13020000000005894,
|
||||
0.12360000000035143,
|
||||
0.14609999999993306,
|
||||
0.13189999999985957,
|
||||
0.13970000000017535,
|
||||
0.12660000000005311,
|
||||
0.12400000000025102,
|
||||
0.11830000000009022,
|
||||
0.11290000000008149,
|
||||
0.13209999999980937
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.12854000000006635,
|
||||
"minLatency": 0.11290000000008149,
|
||||
"maxLatency": 0.14609999999993306,
|
||||
"p50Latency": 0.12660000000005311,
|
||||
"p95Latency": 0.14609999999993306,
|
||||
"p99Latency": 0.14609999999993306,
|
||||
"hitRate": 1
|
||||
},
|
||||
"NTC Technology": {
|
||||
"query": "NTC营养保送系统原理",
|
||||
"latencies": [
|
||||
0.12449999999989814,
|
||||
0.12459999999964566,
|
||||
0.13049999999975626,
|
||||
0.10840000000007421,
|
||||
0.19180000000005748,
|
||||
0.11740000000008877,
|
||||
0.12619999999969878,
|
||||
0.12239999999974316,
|
||||
0.11910000000034415,
|
||||
0.12409999999999854
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.12889999999993051,
|
||||
"minLatency": 0.10840000000007421,
|
||||
"maxLatency": 0.19180000000005748,
|
||||
"p50Latency": 0.12409999999999854,
|
||||
"p95Latency": 0.19180000000005748,
|
||||
"p99Latency": 0.19180000000005748,
|
||||
"hitRate": 1
|
||||
},
|
||||
"Hot Answer": {
|
||||
"query": "基础三合一怎么吃",
|
||||
"latencies": [
|
||||
0.13679999999976644,
|
||||
0.13400000000001455,
|
||||
0.13280000000031578,
|
||||
0.13779999999997017,
|
||||
0.12579999999979918,
|
||||
0.13449999999966167,
|
||||
0.14399999999977808,
|
||||
0.13839999999981956,
|
||||
0.125,
|
||||
0.1230000000000473
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.13320999999991728,
|
||||
"minLatency": 0.1230000000000473,
|
||||
"maxLatency": 0.14399999999977808,
|
||||
"p50Latency": 0.13400000000001455,
|
||||
"p95Latency": 0.14399999999977808,
|
||||
"p99Latency": 0.14399999999977808,
|
||||
"hitRate": 1
|
||||
},
|
||||
"No Hit Query": {
|
||||
"query": "今天天气怎么样",
|
||||
"latencies": [
|
||||
0.7092999999999847,
|
||||
0.647899999999936,
|
||||
0.869300000000294,
|
||||
0.8305000000000291,
|
||||
0.7291000000000167,
|
||||
0.5709999999999127,
|
||||
0.539299999999912,
|
||||
0.5781999999999243,
|
||||
0.6104999999997744,
|
||||
0.5345000000002074
|
||||
],
|
||||
"hits": [
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
],
|
||||
"avgLatency": 0.6619599999999991,
|
||||
"minLatency": 0.5345000000002074,
|
||||
"maxLatency": 0.869300000000294,
|
||||
"p50Latency": 0.6104999999997744,
|
||||
"p95Latency": 0.869300000000294,
|
||||
"p99Latency": 0.869300000000294,
|
||||
"hitRate": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "cache",
|
||||
"timestamp": "2026-03-20T07:49:07.088Z",
|
||||
"cacheHitsIterations": 5,
|
||||
"mockMode": false,
|
||||
"results": [
|
||||
{
|
||||
"name": "Product Query - Xiaohong",
|
||||
"query": "小红产品有什么功效",
|
||||
"firstHitLatency": 0.1477999999997337,
|
||||
"cacheHitLatencies": [
|
||||
0.14179999999987558,
|
||||
0.1406000000001768,
|
||||
0.13950000000022555,
|
||||
0.13410000000021682,
|
||||
0.10729999999966822
|
||||
],
|
||||
"avgCacheLatency": 0.13266000000003259,
|
||||
"speedup": 1.114126338004654,
|
||||
"firstHit": {
|
||||
"query": "小红产品有什么功效",
|
||||
"original_query": "小红产品有什么功效",
|
||||
"rewritten_query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分，通过NTC营养保送系统直达细胞。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。很多人喝完当天就能感受到精力明显提升。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Product Query - Dabai",
|
||||
"query": "大白产品怎么吃",
|
||||
"firstHitLatency": 0.1453999999998814,
|
||||
"cacheHitLatencies": [
|
||||
0.13940000000002328,
|
||||
0.13879999999971915,
|
||||
0.14170000000012806,
|
||||
0.14660000000003492,
|
||||
0.16849999999976717
|
||||
],
|
||||
"avgCacheLatency": 0.14699999999993452,
|
||||
"speedup": 0.9891156462581372,
|
||||
"firstHit": {
|
||||
"query": "大白产品怎么吃",
|
||||
"original_query": "大白产品怎么吃",
|
||||
"rewritten_query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM大白Basics是基础营养素，为身体提供全面的维生素和矿物质基础。它的核心作用：1.补充每日所需的基础营养。2.为细胞提供全面的营养支持。3.搭配小红和小白形成完整的NTC营养循环。建议早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Company Info",
|
||||
"query": "德国PM公司介绍",
|
||||
"firstHitLatency": 0.17620000000033542,
|
||||
"cacheHitLatencies": [
|
||||
0.1550999999999476,
|
||||
0.11990000000014334,
|
||||
0.15079999999989013,
|
||||
0.14349999999967622,
|
||||
0.134900000000016
|
||||
],
|
||||
"avgCacheLatency": 0.14083999999993466,
|
||||
"speedup": 1.2510650383443422,
|
||||
"firstHit": {
|
||||
"query": "德国PM公司介绍",
|
||||
"original_query": "德国PM公司介绍",
|
||||
"rewritten_query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。公司获邓白氏AAA+最高信用认证（99分），业务覆盖全球100多个国家。总部位于德国，在日本、美国、加拿大、香港等地设有分公司。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "concurrency",
|
||||
"timestamp": "2026-03-20T07:49:07.090Z",
|
||||
"concurrencyLevels": [
|
||||
1,
|
||||
3,
|
||||
5
|
||||
],
|
||||
"mockMode": false,
|
||||
"results": {
|
||||
"1": {
|
||||
"concurrency": 1,
|
||||
"totalTime": 0.1661999999996624,
|
||||
"throughput": 6016.847172094051,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"original_query": "小红产品有什么功效",
|
||||
"rewritten_query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分，通过NTC营养保送系统直达细胞。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。很多人喝完当天就能感受到精力明显提升。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"3": {
|
||||
"concurrency": 3,
|
||||
"totalTime": 0.4667999999996937,
|
||||
"throughput": 6426.735218513215,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"original_query": "小红产品有什么功效",
|
||||
"rewritten_query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分，通过NTC营养保送系统直达细胞。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。很多人喝完当天就能感受到精力明显提升。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"original_query": "大白产品怎么吃",
|
||||
"rewritten_query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM大白Basics是基础营养素，为身体提供全面的维生素和矿物质基础。它的核心作用：1.补充每日所需的基础营养。2.为细胞提供全面的营养支持。3.搭配小红和小白形成完整的NTC营养循环。建议早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "德国PM公司介绍",
|
||||
"original_query": "德国PM公司介绍",
|
||||
"rewritten_query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。公司获邓白氏AAA+最高信用认证（99分），业务覆盖全球100多个国家。总部位于德国，在日本、美国、加拿大、香港等地设有分公司。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"5": {
|
||||
"concurrency": 5,
|
||||
"totalTime": 0.7764999999999418,
|
||||
"throughput": 6439.150032196232,
|
||||
"successRate": 1,
|
||||
"results": [
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"original_query": "小红产品有什么功效",
|
||||
"rewritten_query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分，通过NTC营养保送系统直达细胞。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。很多人喝完当天就能感受到精力明显提升。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 1,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"original_query": "大白产品怎么吃",
|
||||
"rewritten_query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM大白Basics是基础营养素，为身体提供全面的维生素和矿物质基础。它的核心作用：1.补充每日所需的基础营养。2.为细胞提供全面的营养支持。3.搭配小红和小白形成完整的NTC营养循环。建议早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "德国PM公司介绍",
|
||||
"original_query": "德国PM公司介绍",
|
||||
"rewritten_query": "德国PM公司介绍",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。公司获邓白氏AAA+最高信用认证（99分），业务覆盖全球100多个国家。总部位于德国，在日本、美国、加拿大、香港等地设有分公司。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "小红产品有什么功效",
|
||||
"original_query": "小红产品有什么功效",
|
||||
"rewritten_query": "小红产品有什么功效",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分，通过NTC营养保送系统直达细胞。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。很多人喝完当天就能感受到精力明显提升。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
},
|
||||
{
|
||||
"query": "大白产品怎么吃",
|
||||
"original_query": "大白产品怎么吃",
|
||||
"rewritten_query": "大白产品怎么吃",
|
||||
"results": [
|
||||
{
|
||||
"title": "高频问题快速回答",
|
||||
"content": "德国PM大白Basics是基础营养素，为身体提供全面的维生素和矿物质基础。它的核心作用：1.补充每日所需的基础营养。2.为细胞提供全面的营养支持。3.搭配小红和小白形成完整的NTC营养循环。建议早上空腹服用，1平勺兑200-300ml温水。"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"source": "hot_answer_cache",
|
||||
"hit": true,
|
||||
"reason": "hot_answer",
|
||||
"latency_ms": 0,
|
||||
"hot_answer": true
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
91
test2/server/tests/test_viking_cold_start.js
Normal file
91
test2/server/tests/test_viking_cold_start.js
Normal file
@@ -0,0 +1,91 @@
|
||||
const path = require('path');
|
||||
const { performance } = require('perf_hooks');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
const ToolExecutor = require('../services/toolExecutor');
|
||||
|
||||
const coldTestQueryTemplates = [
|
||||
{ name: 'Product - Q10 Unique', template: 'Q10辅酵素氧修护有什么独特功效 ' },
|
||||
{ name: 'Product - IB5 Unique', template: 'IB5口腔免疫喷雾如何正确使用 ' },
|
||||
{ name: 'Product - CC胶囊 Unique', template: 'CC胶囊的主要适用人群有哪些 ' },
|
||||
{ name: 'Company - 邓白氏 Unique', template: '德国PM邓白氏认证的具体含义是什么 ' },
|
||||
{ name: 'Technology - 火炉原理 Unique', template: '请详细阐述一下火炉原理的核心思想 ' }
|
||||
];
|
||||
|
||||
async function runColdStartTest() {
|
||||
console.log('='.repeat(80));
|
||||
console.log('VIKING COLD START PERFORMANCE TEST');
|
||||
console.log('(No Cache - Real API Calls)');
|
||||
console.log('='.repeat(80));
|
||||
console.log('');
|
||||
|
||||
const allResults = [];
|
||||
|
||||
for (const { name, template } of coldTestQueryTemplates) {
|
||||
console.log(`Testing (Cold): ${name}`);
|
||||
const latencies = [];
|
||||
const hits = [];
|
||||
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const uniqueQuery = template + Math.random();
|
||||
const start = performance.now();
|
||||
let result;
|
||||
try {
|
||||
result = await ToolExecutor.searchKnowledge({ query: uniqueQuery, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
latencies.push(latency);
|
||||
hits.push(!!result.hit);
|
||||
console.log(` Attempt ${i + 1}: ${latency.toFixed(2)}ms, hit=${result.hit}, source=${result.source}`);
|
||||
} catch (e) {
|
||||
console.log(` Attempt ${i + 1} error: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (latencies.length > 0) {
|
||||
const avgLatency = latencies.reduce((a, b) => a + b, 0) / latencies.length;
|
||||
const minLatency = Math.min(...latencies);
|
||||
const maxLatency = Math.max(...latencies);
|
||||
const hitRate = hits.filter(h => h).length / hits.length;
|
||||
|
||||
allResults.push({
|
||||
name,
|
||||
template,
|
||||
avgLatency,
|
||||
minLatency,
|
||||
maxLatency,
|
||||
hitRate,
|
||||
latencies
|
||||
});
|
||||
|
||||
console.log(` → Avg: ${avgLatency.toFixed(2)}ms, Min: ${minLatency.toFixed(2)}ms, Max: ${maxLatency.toFixed(2)}ms, Hit Rate: ${(hitRate * 100).toFixed(1)}%\n`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('='.repeat(80));
|
||||
console.log('COLD START SUMMARY');
|
||||
console.log('='.repeat(80));
|
||||
|
||||
if (allResults.length > 0) {
|
||||
const totalAvg = allResults.reduce((a, b) => a + b.avgLatency, 0) / allResults.length;
|
||||
const totalMin = Math.min(...allResults.flatMap(r => r.latencies));
|
||||
const totalMax = Math.max(...allResults.flatMap(r => r.latencies));
|
||||
const totalHitRate = allResults.reduce((a, b) => a + b.hitRate, 0) / allResults.length;
|
||||
|
||||
console.log(`\nOverall Average Latency: ${totalAvg.toFixed(2)}ms`);
|
||||
console.log(`Overall Min Latency: ${totalMin.toFixed(2)}ms`);
|
||||
console.log(`Overall Max Latency: ${totalMax.toFixed(2)}ms`);
|
||||
console.log(`Overall Hit Rate: ${(totalHitRate * 100).toFixed(1)}%`);
|
||||
|
||||
console.log('\nFastest queries:');
|
||||
allResults.sort((a, b) => a.avgLatency - b.avgLatency).forEach((r, i) => {
|
||||
console.log(` ${i + 1}. ${r.name}: ${r.avgLatency.toFixed(2)}ms`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(80));
|
||||
|
||||
return allResults;
|
||||
}
|
||||
|
||||
runColdStartTest().catch(console.error);
|
||||
152
test2/server/tests/test_viking_direct_api.js
Normal file
152
test2/server/tests/test_viking_direct_api.js
Normal file
@@ -0,0 +1,152 @@
|
||||
const path = require('path');
|
||||
const { performance } = require('perf_hooks');
|
||||
const axios = require('axios');
|
||||
const https = require('https');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
const kbHttpAgent = new https.Agent({
|
||||
keepAlive: true,
|
||||
keepAliveMsecs: 30000,
|
||||
maxSockets: 6,
|
||||
timeout: 15000,
|
||||
});
|
||||
|
||||
const directTestQueries = [
|
||||
{ name: 'Q10 Direct', query: 'Q10辅酵素氧修护有什么独特功效' },
|
||||
{ name: 'IB5 Direct', query: 'IB5口腔免疫喷雾如何正确使用' },
|
||||
{ name: 'CC胶囊 Direct', query: 'CC胶囊的主要适用人群有哪些' },
|
||||
{ name: '邓白氏 Direct', query: '德国PM邓白氏认证的具体含义是什么' },
|
||||
{ name: '火炉原理 Direct', query: '请详细阐述一下火炉原理的核心思想' },
|
||||
];
|
||||
|
||||
async function callVikingDirectly(query, datasetIds) {
|
||||
const endpointId = process.env.VOLC_ARK_KNOWLEDGE_ENDPOINT_ID || process.env.VOLC_ARK_ENDPOINT_ID;
|
||||
const authKey = process.env.VOLC_ARK_API_KEY || process.env.VOLC_ACCESS_KEY_ID;
|
||||
const kbIds = datasetIds || (process.env.VOLC_ARK_KNOWLEDGE_BASE_IDS || '').split(',').map(id => id.trim()).filter(Boolean);
|
||||
const topK = parseInt(process.env.VOLC_ARK_KNOWLEDGE_TOP_K) || 3;
|
||||
const threshold = parseFloat(process.env.VOLC_ARK_KNOWLEDGE_THRESHOLD) || 0.4;
|
||||
|
||||
if (!endpointId || !authKey || kbIds.length === 0) {
|
||||
throw new Error('Missing required config');
|
||||
}
|
||||
|
||||
const systemContent = '你是企业知识库问答助手。只依据知识库信息回答，不补充不脑补。';
|
||||
|
||||
const body = {
|
||||
model: endpointId,
|
||||
messages: [
|
||||
{ role: 'system', content: systemContent },
|
||||
{ role: 'user', content: query }
|
||||
],
|
||||
metadata: {
|
||||
knowledge_base: {
|
||||
dataset_ids: kbIds,
|
||||
top_k: topK,
|
||||
threshold: threshold,
|
||||
},
|
||||
},
|
||||
stream: false,
|
||||
max_tokens: 400,
|
||||
};
|
||||
|
||||
const response = await axios.post(
|
||||
'https://ark.cn-beijing.volces.com/api/v3/chat/completions',
|
||||
body,
|
||||
{
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${authKey}`,
|
||||
},
|
||||
timeout: 30000,
|
||||
httpsAgent: kbHttpAgent,
|
||||
}
|
||||
);
|
||||
|
||||
return response.data;
|
||||
}
|
||||
|
||||
async function runDirectAPITest() {
|
||||
console.log('='.repeat(80));
|
||||
console.log('VIKING DIRECT API PERFORMANCE TEST');
|
||||
console.log('(No Query Rewrite, No Cache - Pure API Calls)');
|
||||
console.log('='.repeat(80));
|
||||
console.log('');
|
||||
|
||||
const allResults = [];
|
||||
|
||||
for (const { name, query } of directTestQueries) {
|
||||
console.log(`Testing (Direct): ${name}`);
|
||||
const latencies = [];
|
||||
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const uniqueSuffix = ` [${Date.now()}-${Math.random()}]`;
|
||||
const uniqueQuery = query + uniqueSuffix;
|
||||
|
||||
const start = performance.now();
|
||||
try {
|
||||
const result = await callVikingDirectly(uniqueQuery);
|
||||
const latency = performance.now() - start;
|
||||
latencies.push(latency);
|
||||
|
||||
const content = result?.choices?.[0]?.message?.content || 'N/A';
|
||||
console.log(` Attempt ${i + 1}: ${latency.toFixed(2)}ms, content length=${content.length}`);
|
||||
} catch (e) {
|
||||
console.log(` Attempt ${i + 1} error: ${e.message}`);
|
||||
if (e.response) {
|
||||
console.log(` Status: ${e.response.status}, Data:`, JSON.stringify(e.response.data).substring(0, 200));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (latencies.length > 0) {
|
||||
const avgLatency = latencies.reduce((a, b) => a + b, 0) / latencies.length;
|
||||
const minLatency = Math.min(...latencies);
|
||||
const maxLatency = Math.max(...latencies);
|
||||
|
||||
allResults.push({
|
||||
name,
|
||||
query,
|
||||
avgLatency,
|
||||
minLatency,
|
||||
maxLatency,
|
||||
latencies
|
||||
});
|
||||
|
||||
console.log(` → Avg: ${avgLatency.toFixed(2)}ms, Min: ${minLatency.toFixed(2)}ms, Max: ${maxLatency.toFixed(2)}ms\n`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('='.repeat(80));
|
||||
console.log('DIRECT API SUMMARY');
|
||||
console.log('='.repeat(80));
|
||||
|
||||
if (allResults.length > 0) {
|
||||
const totalAvg = allResults.reduce((a, b) => a + b.avgLatency, 0) / allResults.length;
|
||||
const totalMin = Math.min(...allResults.flatMap(r => r.latencies));
|
||||
const totalMax = Math.max(...allResults.flatMap(r => r.latencies));
|
||||
const allLatencies = allResults.flatMap(r => r.latencies);
|
||||
allLatencies.sort((a, b) => a - b);
|
||||
const p50Index = Math.ceil(0.5 * allLatencies.length) - 1;
|
||||
const p95Index = Math.ceil(0.95 * allLatencies.length) - 1;
|
||||
const p99Index = Math.ceil(0.99 * allLatencies.length) - 1;
|
||||
|
||||
console.log(`\nOverall Average Latency: ${totalAvg.toFixed(2)}ms`);
|
||||
console.log(`Overall P50 Latency: ${allLatencies[p50Index].toFixed(2)}ms`);
|
||||
console.log(`Overall P95 Latency: ${allLatencies[p95Index].toFixed(2)}ms`);
|
||||
console.log(`Overall P99 Latency: ${allLatencies[p99Index].toFixed(2)}ms`);
|
||||
console.log(`Overall Min Latency: ${totalMin.toFixed(2)}ms`);
|
||||
console.log(`Overall Max Latency: ${totalMax.toFixed(2)}ms`);
|
||||
|
||||
console.log('\nAll queries sorted by latency:');
|
||||
allResults.sort((a, b) => a.avgLatency - b.avgLatency).forEach((r, i) => {
|
||||
console.log(` ${i + 1}. ${r.name}: ${r.avgLatency.toFixed(2)}ms`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(80));
|
||||
|
||||
return allResults;
|
||||
}
|
||||
|
||||
runDirectAPITest().catch(console.error);
|
||||
349
test2/server/tests/viking_retrieval_performance.js
Normal file
349
test2/server/tests/viking_retrieval_performance.js
Normal file
@@ -0,0 +1,349 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { performance } = require('perf_hooks');
|
||||
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
|
||||
const ToolExecutor = require('../services/toolExecutor');
|
||||
|
||||
class VikingRetrievalPerformanceTester {
|
||||
constructor(options = {}) {
|
||||
this.results = [];
|
||||
this.outputDir = options.outputDir || path.join(__dirname, 'test_results');
|
||||
this.verbose = options.verbose !== false;
|
||||
this.warmupRuns = options.warmupRuns || 2;
|
||||
|
||||
if (!fs.existsSync(this.outputDir)) {
|
||||
fs.mkdirSync(this.outputDir, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
log(msg) {
|
||||
if (this.verbose) {
|
||||
console.log(`[VikingTest] ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
async warmup(queries) {
|
||||
this.log('Warming up...');
|
||||
for (let i = 0; i < this.warmupRuns; i++) {
|
||||
for (const query of queries) {
|
||||
try {
|
||||
await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
} catch (e) {
|
||||
// ignore warmup errors
|
||||
}
|
||||
}
|
||||
}
|
||||
this.log('Warmup complete');
|
||||
}
|
||||
|
||||
async testLatency(testQueries, iterations = 10) {
|
||||
this.log(`Starting latency test with ${iterations} iterations...`);
|
||||
|
||||
const results = {};
|
||||
|
||||
for (const { name, query } of testQueries) {
|
||||
this.log(`Testing query: ${name}`);
|
||||
const latencies = [];
|
||||
const hits = [];
|
||||
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
const start = performance.now();
|
||||
let result;
|
||||
try {
|
||||
result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
latencies.push(latency);
|
||||
hits.push(!!result.hit);
|
||||
this.log(` Iteration ${i + 1}: ${latency.toFixed(2)}ms, hit=${result.hit}`);
|
||||
} catch (e) {
|
||||
this.log(` Iteration ${i + 1} error: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
results[name] = {
|
||||
query,
|
||||
latencies,
|
||||
hits,
|
||||
avgLatency: latencies.length ? latencies.reduce((a, b) => a + b, 0) / latencies.length : 0,
|
||||
minLatency: latencies.length ? Math.min(...latencies) : 0,
|
||||
maxLatency: latencies.length ? Math.max(...latencies) : 0,
|
||||
p50Latency: this.percentile(latencies, 50),
|
||||
p95Latency: this.percentile(latencies, 95),
|
||||
p99Latency: this.percentile(latencies, 99),
|
||||
hitRate: hits.length ? hits.filter(h => h).length / hits.length : 0
|
||||
};
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'latency',
|
||||
timestamp: new Date().toISOString(),
|
||||
iterations,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
async testCacheEfficiency(queries, cacheHitsIterations = 5) {
|
||||
this.log('Testing cache efficiency...');
|
||||
|
||||
const results = [];
|
||||
|
||||
for (const { name, query } of queries) {
|
||||
this.log(`Testing cache for query: ${name}`);
|
||||
|
||||
const firstStart = performance.now();
|
||||
const firstResult = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const firstLatency = performance.now() - firstStart;
|
||||
|
||||
const cacheLatencies = [];
|
||||
for (let i = 0; i < cacheHitsIterations; i++) {
|
||||
const start = performance.now();
|
||||
const result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
cacheLatencies.push(latency);
|
||||
this.log(` Cache hit ${i + 1}: ${latency.toFixed(2)}ms, cache_hit=${!!result.cache_hit}`);
|
||||
}
|
||||
|
||||
const avgCacheLatency = cacheLatencies.reduce((a, b) => a + b, 0) / cacheLatencies.length;
|
||||
|
||||
results.push({
|
||||
name,
|
||||
query,
|
||||
firstHitLatency: firstLatency,
|
||||
cacheHitLatencies,
|
||||
avgCacheLatency,
|
||||
speedup: firstLatency / avgCacheLatency,
|
||||
firstHit: firstResult
|
||||
});
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'cache',
|
||||
timestamp: new Date().toISOString(),
|
||||
cacheHitsIterations,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
async testConcurrency(queries, concurrencyLevels = [1, 5, 10, 20]) {
|
||||
this.log('Testing concurrency...');
|
||||
|
||||
const results = {};
|
||||
|
||||
for (const concurrency of concurrencyLevels) {
|
||||
this.log(`Testing concurrency level: ${concurrency}`);
|
||||
|
||||
const startTime = performance.now();
|
||||
const promises = [];
|
||||
|
||||
for (let i = 0; i < concurrency; i++) {
|
||||
const queryObj = queries[i % queries.length];
|
||||
promises.push(
|
||||
ToolExecutor.searchKnowledge({ query: queryObj.query, response_mode: 'answer' }, [])
|
||||
);
|
||||
}
|
||||
|
||||
const allResults = await Promise.all(promises);
|
||||
const totalTime = performance.now() - startTime;
|
||||
|
||||
const successCount = allResults.filter(r => r && !r.error).length;
|
||||
const latencies = allResults.map((r, i) => {
|
||||
return totalTime / concurrency;
|
||||
});
|
||||
|
||||
results[concurrency] = {
|
||||
concurrency,
|
||||
totalTime,
|
||||
throughput: concurrency / (totalTime / 1000),
|
||||
successRate: successCount / concurrency,
|
||||
results: allResults
|
||||
};
|
||||
|
||||
this.log(` Throughput: ${results[concurrency].throughput.toFixed(2)} req/s`);
|
||||
this.log(` Success rate: ${(results[concurrency].successRate * 100).toFixed(1)}%`);
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'concurrency',
|
||||
timestamp: new Date().toISOString(),
|
||||
concurrencyLevels,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
async testQueryTypes(queryGroups) {
|
||||
this.log('Testing different query types...');
|
||||
|
||||
const results = {};
|
||||
|
||||
for (const [groupName, queries] of Object.entries(queryGroups)) {
|
||||
this.log(`Testing group: ${groupName}`);
|
||||
|
||||
const groupResults = [];
|
||||
|
||||
for (const query of queries) {
|
||||
const start = performance.now();
|
||||
const result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
|
||||
groupResults.push({
|
||||
query,
|
||||
latency,
|
||||
hit: !!result.hit,
|
||||
result
|
||||
});
|
||||
}
|
||||
|
||||
results[groupName] = {
|
||||
queries: groupResults,
|
||||
avgLatency: groupResults.reduce((a, b) => a + b.latency, 0) / groupResults.length,
|
||||
hitRate: groupResults.filter(r => r.hit).length / groupResults.length
|
||||
};
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'query_types',
|
||||
timestamp: new Date().toISOString(),
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
percentile(arr, p) {
|
||||
if (arr.length === 0) return 0;
|
||||
const sorted = [...arr].sort((a, b) => a - b);
|
||||
const index = Math.ceil((p / 100) * sorted.length) - 1;
|
||||
return sorted[Math.max(0, index)];
|
||||
}
|
||||
|
||||
generateReport() {
|
||||
const report = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
summary: {},
|
||||
tests: this.results
|
||||
};
|
||||
|
||||
for (const test of this.results) {
|
||||
if (test.type === 'latency') {
|
||||
report.summary.latency = Object.fromEntries(
|
||||
Object.entries(test.results).map(([name, data]) => [
|
||||
name,
|
||||
{
|
||||
avg: data.avgLatency.toFixed(2),
|
||||
p95: data.p95Latency.toFixed(2),
|
||||
hitRate: (data.hitRate * 100).toFixed(1) + '%'
|
||||
}
|
||||
])
|
||||
);
|
||||
} else if (test.type === 'cache') {
|
||||
report.summary.cache = test.results.map(r => ({
|
||||
name: r.name,
|
||||
speedup: r.speedup.toFixed(2) + 'x'
|
||||
}));
|
||||
} else if (test.type === 'concurrency') {
|
||||
report.summary.concurrency = Object.fromEntries(
|
||||
Object.entries(test.results).map(([level, data]) => [
|
||||
level,
|
||||
{
|
||||
throughput: data.throughput.toFixed(2) + ' req/s',
|
||||
successRate: (data.successRate * 100).toFixed(1) + '%'
|
||||
}
|
||||
])
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
saveReport(filename = null) {
|
||||
const report = this.generateReport();
|
||||
const filepath = path.join(
|
||||
this.outputDir,
|
||||
filename || `viking_performance_${Date.now()}.json`
|
||||
);
|
||||
fs.writeFileSync(filepath, JSON.stringify(report, null, 2));
|
||||
this.log(`Report saved to ${filepath}`);
|
||||
return filepath;
|
||||
}
|
||||
|
||||
printSummary() {
|
||||
console.log('\n' + '='.repeat(80));
|
||||
console.log('VIKING RETRIEVAL PERFORMANCE TEST SUMMARY');
|
||||
console.log('='.repeat(80));
|
||||
|
||||
const report = this.generateReport();
|
||||
|
||||
if (report.summary.latency) {
|
||||
console.log('\n--- Latency Test ---');
|
||||
for (const [name, data] of Object.entries(report.summary.latency)) {
|
||||
console.log(` ${name}:`);
|
||||
console.log(` Avg: ${data.avg}ms, P95: ${data.p95}ms, Hit Rate: ${data.hitRate}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (report.summary.cache) {
|
||||
console.log('\n--- Cache Efficiency ---');
|
||||
for (const r of report.summary.cache) {
|
||||
console.log(` ${r.name}: Speedup ${r.speedup}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (report.summary.concurrency) {
|
||||
console.log('\n--- Concurrency Test ---');
|
||||
for (const [level, data] of Object.entries(report.summary.concurrency)) {
|
||||
console.log(` ${level} concurrent:`);
|
||||
console.log(` Throughput: ${data.throughput}, Success: ${data.successRate}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(80));
|
||||
}
|
||||
|
||||
async runFullSuite() {
|
||||
this.log('Starting full Viking retrieval performance test suite...');
|
||||
|
||||
const testQueries = [
|
||||
{ name: 'Product Query - Xiaohong', query: '小红产品有什么功效' },
|
||||
{ name: 'Product Query - Dabai', query: '大白产品怎么吃' },
|
||||
{ name: 'Company Info', query: '德国PM公司介绍' },
|
||||
{ name: 'NTC Technology', query: 'NTC营养保送系统原理' },
|
||||
{ name: 'Hot Answer', query: '基础三合一怎么吃' },
|
||||
{ name: 'No Hit Query', query: '今天天气怎么样' }
|
||||
];
|
||||
|
||||
await this.warmup(testQueries.map(q => q.query));
|
||||
await this.testLatency(testQueries, 10);
|
||||
await this.testCacheEfficiency(testQueries.slice(0, 3), 5);
|
||||
await this.testConcurrency(testQueries.slice(0, 3), [1, 3, 5]);
|
||||
|
||||
const queryGroups = {
|
||||
product: ['小红有什么功效', '大白怎么吃', '小白的作用'],
|
||||
company: ['PM公司介绍', '邓白氏认证', '总部在哪里'],
|
||||
technical: ['NTC原理', '火炉原理', '好转反应']
|
||||
};
|
||||
await this.testQueryTypes(queryGroups);
|
||||
|
||||
this.printSummary();
|
||||
this.saveReport();
|
||||
|
||||
return this.results;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = VikingRetrievalPerformanceTester;
|
||||
|
||||
if (require.main === module) {
|
||||
(async () => {
|
||||
const tester = new VikingRetrievalPerformanceTester();
|
||||
await tester.runFullSuite();
|
||||
})();
|
||||
}
|
||||
354
test2/server/tests/viking_retrieval_performance_with_mock.js
Normal file
354
test2/server/tests/viking_retrieval_performance_with_mock.js
Normal file
@@ -0,0 +1,354 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { performance } = require('perf_hooks');
|
||||
|
||||
class MockToolExecutor {
|
||||
static async searchKnowledge({ query, response_mode }, context) {
|
||||
await new Promise(resolve => setTimeout(resolve, 100 + Math.random() * 200));
|
||||
|
||||
const mockResults = {
|
||||
'小红产品有什么功效': {
|
||||
hit: true,
|
||||
content: 'FitLine小红Activize Oxyplus的核心功效是提升细胞能量。它含有维生素B族、C和辅酶Q10等成分。主要作用：1.提升精力和体能。2.促进细胞氧气利用。3.增强免疫力。'
|
||||
},
|
||||
'大白产品怎么吃': {
|
||||
hit: true,
|
||||
content: '德国PM大白Basics是基础营养素，早上空腹服用，1平勺兑200-300ml温水。'
|
||||
},
|
||||
'德国PM公司介绍': {
|
||||
hit: true,
|
||||
content: '德国PM-International是1993年创立的国际营养品直销企业。核心品牌FitLine专注细胞营养，拥有独家NTC营养保送系统技术。'
|
||||
},
|
||||
'NTC营养保送系统原理': {
|
||||
hit: true,
|
||||
content: 'NTC营养保送系统是PM产品的核心技术优势。它能确保营养素在体内精准保送到细胞层面。'
|
||||
},
|
||||
'基础三合一怎么吃': {
|
||||
hit: true,
|
||||
hot_answer: true,
|
||||
content: '基础三合一这样吃：1.大白Basics：早上空腹，1平勺兑200-300ml温水。2.小红Activize：大白喝完15-30分钟后，兑100-150ml温水。3.小白Restorate：睡前空腹，1平勺兑200ml温水。'
|
||||
},
|
||||
'今天天气怎么样': {
|
||||
hit: false,
|
||||
content: '知识库中暂未找到相关信息。'
|
||||
}
|
||||
};
|
||||
|
||||
const result = mockResults[query] || { hit: false, content: '未找到相关信息' };
|
||||
|
||||
return {
|
||||
query,
|
||||
results: [{ title: '模拟知识库结果', content: result.content }],
|
||||
hit: result.hit,
|
||||
source: result.hot_answer ? 'hot_answer_cache' : 'mock_knowledge',
|
||||
hot_answer: result.hot_answer
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
class VikingRetrievalPerformanceTester {
|
||||
constructor(options = {}) {
|
||||
this.results = [];
|
||||
this.outputDir = options.outputDir || path.join(__dirname, 'test_results');
|
||||
this.verbose = options.verbose !== false;
|
||||
this.warmupRuns = options.warmupRuns || 2;
|
||||
this.mockMode = options.mockMode !== false;
|
||||
|
||||
if (!fs.existsSync(this.outputDir)) {
|
||||
fs.mkdirSync(this.outputDir, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
log(msg) {
|
||||
if (this.verbose) {
|
||||
console.log(`[VikingTest] ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
async warmup(queries) {
|
||||
this.log('Warming up...');
|
||||
const ToolExecutor = this.mockMode ? MockToolExecutor : require('../services/toolExecutor');
|
||||
for (let i = 0; i < this.warmupRuns; i++) {
|
||||
for (const query of queries) {
|
||||
try {
|
||||
await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
} catch (e) {
|
||||
}
|
||||
}
|
||||
}
|
||||
this.log('Warmup complete');
|
||||
}
|
||||
|
||||
async testLatency(testQueries, iterations = 10) {
|
||||
this.log(`Starting latency test with ${iterations} iterations...`);
|
||||
const ToolExecutor = this.mockMode ? MockToolExecutor : require('../services/toolExecutor');
|
||||
|
||||
const results = {};
|
||||
|
||||
for (const { name, query } of testQueries) {
|
||||
this.log(`Testing query: ${name}`);
|
||||
const latencies = [];
|
||||
const hits = [];
|
||||
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
const start = performance.now();
|
||||
let result;
|
||||
try {
|
||||
result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
latencies.push(latency);
|
||||
hits.push(!!result.hit);
|
||||
this.log(` Iteration ${i + 1}: ${latency.toFixed(2)}ms, hit=${result.hit}`);
|
||||
} catch (e) {
|
||||
this.log(` Iteration ${i + 1} error: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
results[name] = {
|
||||
query,
|
||||
latencies,
|
||||
hits,
|
||||
avgLatency: latencies.length ? latencies.reduce((a, b) => a + b, 0) / latencies.length : 0,
|
||||
minLatency: latencies.length ? Math.min(...latencies) : 0,
|
||||
maxLatency: latencies.length ? Math.max(...latencies) : 0,
|
||||
p50Latency: this.percentile(latencies, 50),
|
||||
p95Latency: this.percentile(latencies, 95),
|
||||
p99Latency: this.percentile(latencies, 99),
|
||||
hitRate: hits.length ? hits.filter(h => h).length / hits.length : 0
|
||||
};
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'latency',
|
||||
timestamp: new Date().toISOString(),
|
||||
iterations,
|
||||
mockMode: this.mockMode,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
async testCacheEfficiency(queries, cacheHitsIterations = 5) {
|
||||
this.log('Testing cache efficiency...');
|
||||
const ToolExecutor = this.mockMode ? MockToolExecutor : require('../services/toolExecutor');
|
||||
|
||||
const results = [];
|
||||
|
||||
for (const { name, query } of queries) {
|
||||
this.log(`Testing cache for query: ${name}`);
|
||||
|
||||
const firstStart = performance.now();
|
||||
const firstResult = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const firstLatency = performance.now() - firstStart;
|
||||
|
||||
const cacheLatencies = [];
|
||||
for (let i = 0; i < cacheHitsIterations; i++) {
|
||||
const start = performance.now();
|
||||
const result = await ToolExecutor.searchKnowledge({ query, response_mode: 'answer' }, []);
|
||||
const latency = performance.now() - start;
|
||||
cacheLatencies.push(latency);
|
||||
this.log(` Cache hit ${i + 1}: ${latency.toFixed(2)}ms`);
|
||||
}
|
||||
|
||||
const avgCacheLatency = cacheLatencies.reduce((a, b) => a + b, 0) / cacheLatencies.length;
|
||||
|
||||
results.push({
|
||||
name,
|
||||
query,
|
||||
firstHitLatency: firstLatency,
|
||||
cacheHitLatencies: cacheLatencies,
|
||||
avgCacheLatency,
|
||||
speedup: firstLatency / avgCacheLatency,
|
||||
firstHit: firstResult
|
||||
});
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'cache',
|
||||
timestamp: new Date().toISOString(),
|
||||
cacheHitsIterations,
|
||||
mockMode: this.mockMode,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
async testConcurrency(queries, concurrencyLevels = [1, 5, 10, 20]) {
|
||||
this.log('Testing concurrency...');
|
||||
const ToolExecutor = this.mockMode ? MockToolExecutor : require('../services/toolExecutor');
|
||||
|
||||
const results = {};
|
||||
|
||||
for (const concurrency of concurrencyLevels) {
|
||||
this.log(`Testing concurrency level: ${concurrency}`);
|
||||
|
||||
const startTime = performance.now();
|
||||
const promises = [];
|
||||
|
||||
for (let i = 0; i < concurrency; i++) {
|
||||
const queryObj = queries[i % queries.length];
|
||||
promises.push(
|
||||
ToolExecutor.searchKnowledge({ query: queryObj.query, response_mode: 'answer' }, [])
|
||||
);
|
||||
}
|
||||
|
||||
const allResults = await Promise.all(promises);
|
||||
const totalTime = performance.now() - startTime;
|
||||
|
||||
const successCount = allResults.filter(r => r && !r.error).length;
|
||||
|
||||
results[concurrency] = {
|
||||
concurrency,
|
||||
totalTime,
|
||||
throughput: concurrency / (totalTime / 1000),
|
||||
successRate: successCount / concurrency,
|
||||
results: allResults
|
||||
};
|
||||
|
||||
this.log(` Throughput: ${results[concurrency].throughput.toFixed(2)} req/s`);
|
||||
this.log(` Success rate: ${(results[concurrency].successRate * 100).toFixed(1)}%`);
|
||||
}
|
||||
|
||||
this.results.push({
|
||||
type: 'concurrency',
|
||||
timestamp: new Date().toISOString(),
|
||||
concurrencyLevels,
|
||||
mockMode: this.mockMode,
|
||||
results
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
percentile(arr, p) {
|
||||
if (arr.length === 0) return 0;
|
||||
const sorted = [...arr].sort((a, b) => a - b);
|
||||
const index = Math.ceil((p / 100) * sorted.length) - 1;
|
||||
return sorted[Math.max(0, index)];
|
||||
}
|
||||
|
||||
generateReport() {
|
||||
const report = {
|
||||
generatedAt: new Date().toISOString(),
|
||||
mockMode: this.mockMode,
|
||||
summary: {},
|
||||
tests: this.results
|
||||
};
|
||||
|
||||
for (const test of this.results) {
|
||||
if (test.type === 'latency') {
|
||||
report.summary.latency = Object.fromEntries(
|
||||
Object.entries(test.results).map(([name, data]) => [
|
||||
name,
|
||||
{
|
||||
avg: data.avgLatency.toFixed(2),
|
||||
p95: data.p95Latency.toFixed(2),
|
||||
hitRate: (data.hitRate * 100).toFixed(1) + '%'
|
||||
}
|
||||
])
|
||||
);
|
||||
} else if (test.type === 'cache') {
|
||||
report.summary.cache = test.results.map(r => ({
|
||||
name: r.name,
|
||||
speedup: r.speedup.toFixed(2) + 'x'
|
||||
}));
|
||||
} else if (test.type === 'concurrency') {
|
||||
report.summary.concurrency = Object.fromEntries(
|
||||
Object.entries(test.results).map(([level, data]) => [
|
||||
level,
|
||||
{
|
||||
throughput: data.throughput.toFixed(2) + ' req/s',
|
||||
successRate: (data.successRate * 100).toFixed(1) + '%'
|
||||
}
|
||||
])
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
saveReport(filename = null) {
|
||||
const report = this.generateReport();
|
||||
const filepath = path.join(
|
||||
this.outputDir,
|
||||
filename || `viking_performance_${Date.now()}.json`
|
||||
);
|
||||
fs.writeFileSync(filepath, JSON.stringify(report, null, 2));
|
||||
this.log(`Report saved to ${filepath}`);
|
||||
return filepath;
|
||||
}
|
||||
|
||||
printSummary() {
|
||||
console.log('\n' + '='.repeat(80));
|
||||
console.log('VIKING RETRIEVAL PERFORMANCE TEST SUMMARY');
|
||||
if (this.mockMode) {
|
||||
console.log('(Mock Mode - For Framework Validation)');
|
||||
}
|
||||
console.log('='.repeat(80));
|
||||
|
||||
const report = this.generateReport();
|
||||
|
||||
if (report.summary.latency) {
|
||||
console.log('\n--- Latency Test ---');
|
||||
for (const [name, data] of Object.entries(report.summary.latency)) {
|
||||
console.log(` ${name}:`);
|
||||
console.log(` Avg: ${data.avg}ms, P95: ${data.p95}ms, Hit Rate: ${data.hitRate}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (report.summary.cache) {
|
||||
console.log('\n--- Cache Efficiency ---');
|
||||
for (const r of report.summary.cache) {
|
||||
console.log(` ${r.name}: Speedup ${r.speedup}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (report.summary.concurrency) {
|
||||
console.log('\n--- Concurrency Test ---');
|
||||
for (const [level, data] of Object.entries(report.summary.concurrency)) {
|
||||
console.log(` ${level} concurrent:`);
|
||||
console.log(` Throughput: ${data.throughput}, Success: ${data.successRate}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(80));
|
||||
}
|
||||
|
||||
async runFullSuite() {
|
||||
this.log('Starting full Viking retrieval performance test suite...');
|
||||
|
||||
const testQueries = [
|
||||
{ name: 'Product Query - Xiaohong', query: '小红产品有什么功效' },
|
||||
{ name: 'Product Query - Dabai', query: '大白产品怎么吃' },
|
||||
{ name: 'Company Info', query: '德国PM公司介绍' },
|
||||
{ name: 'NTC Technology', query: 'NTC营养保送系统原理' },
|
||||
{ name: 'Hot Answer', query: '基础三合一怎么吃' },
|
||||
{ name: 'No Hit Query', query: '今天天气怎么样' }
|
||||
];
|
||||
|
||||
await this.warmup(testQueries.map(q => q.query));
|
||||
await this.testLatency(testQueries, 10);
|
||||
await this.testCacheEfficiency(testQueries.slice(0, 3), 5);
|
||||
await this.testConcurrency(testQueries.slice(0, 3), [1, 3, 5]);
|
||||
|
||||
this.printSummary();
|
||||
this.saveReport();
|
||||
|
||||
return this.results;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = VikingRetrievalPerformanceTester;
|
||||
|
||||
if (require.main === module) {
|
||||
(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const mockMode = args[0] !== 'real';
|
||||
|
||||
const tester = new VikingRetrievalPerformanceTester({ mockMode });
|
||||
await tester.runFullSuite();
|
||||
})();
|
||||
}
|
||||
Reference in New Issue
Block a user