436 lines
11 KiB
Markdown
436 lines
11 KiB
Markdown
# RunningHub Sora2 使用指南
|
||
|
||
**版本：** v2.1.0
|
||
**更新时间：** 2025-10-20
|
||
|
||
---
|
||
|
||
## 📋 模型列表
|
||
|
||
系统已预配置以下RunningHub Sora2模型（共12个）：
|
||
|
||
### 文生视频模型（webappId: 1973555977595301890）
|
||
|
||
| 模型名称 | 说明 | 时长 | 分辨率 | 积分消耗 |
|
||
|---------|------|------|--------|---------|
|
||
| `rh_sora2_text_portrait` | 竖屏视频 | 10秒 | 704x1280 | 160 |
|
||
| `rh_sora2_text_landscape` | 横屏视频 | 10秒 | 1280x704 | 160 |
|
||
| `rh_sora2_text_portrait_hd` | 高清竖屏 | 10秒 | 1024x1792 | 420 |
|
||
| `rh_sora2_text_landscape_hd` | 高清横屏 | 10秒 | 1792x1024 | 420 |
|
||
| `rh_sora2_text_portrait_15s` | 竖屏视频（长） | 15秒 | 704x1280 | 260 |
|
||
| `rh_sora2_text_landscape_15s` | 横屏视频（长） | 15秒 | 1280x704 | 260 |
|
||
|
||
### 图生视频模型（webappId: 1973555366057390081）
|
||
|
||
| 模型名称 | 说明 | 时长 | 分辨率 | 积分消耗 |
|
||
|---------|------|------|--------|---------|
|
||
| `rh_sora2_img_portrait` | 竖屏视频 | 10秒 | 704x1280 | 180 |
|
||
| `rh_sora2_img_landscape` | 横屏视频 | 10秒 | 1280x704 | 180 |
|
||
| `rh_sora2_img_portrait_hd` | 高清竖屏 | 10秒 | 1024x1792 | 480 |
|
||
| `rh_sora2_img_landscape_hd` | 高清横屏 | 10秒 | 1792x1024 | 480 |
|
||
| `rh_sora2_img_portrait_15s` | 竖屏视频（长） | 15秒 | 704x1280 | 280 |
|
||
| `rh_sora2_img_landscape_15s` | 横屏视频（长） | 15秒 | 1280x704 | 280 |
|
||
|
||
---
|
||
|
||
## 🚀 使用示例
|
||
|
||
### 1. 文生视频（Text to Video）
|
||
|
||
只需要提供 `prompt`，系统会自动选择横竖屏和时长。
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8081/user/ai/tasks/submit" \
|
||
-H "Authorization: Bearer YOUR_JWT_OR_API_KEY" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"modelName": "rh_sora2_text_portrait",
|
||
"prompt": "第一镜（0–3秒）静谧晨光\n窗外的城市尚在沉睡，镜头推向一只放在床头的 Apple Watch。屏幕亮起的瞬间，柔和的光映照出主人的脸庞，他睁开眼，呼吸与心率同步闪烁。\n第二镜（3–7秒）节奏苏醒\n主角在晨跑，步伐稳健。手表屏幕显示心率曲线与路线图，汗水顺着手臂滑落。阳光从高楼间洒下，镜头追随腕间的微光与动作的力量。\n第三镜（7–10秒）自我回归\n跑步结束，他停在桥上深吸一口气，城市的天际线在他背后延伸。镜头慢慢拉远，Apple Watch的屏幕定格在闪烁的数字上，文字浮现：掌控每一秒的呼吸。"
|
||
}'
|
||
```
|
||
|
||
**实际发送到RunningHub的请求：**
|
||
```json
|
||
{
|
||
"webappId": "1973555977595301890",
|
||
"apiKey": "5c44cef12da3470e9f24da70c63787dc",
|
||
"nodeInfoList": [
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "prompt",
|
||
"fieldValue": "第一镜（0–3秒）静谧晨光...",
|
||
"description": "输入文本"
|
||
},
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "model",
|
||
"fieldValue": "portrait",
|
||
"fieldData": "[{\"name\":\"portrait\",...}]",
|
||
"description": "横竖模式"
|
||
},
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "duration_seconds",
|
||
"fieldValue": "10",
|
||
"fieldData": "[[10, 15], {\"default\": 10}]",
|
||
"description": "时长（秒）"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 2. 图生视频（Image to Video）
|
||
|
||
需要提供 `imageUrl` 或 `imageBase64`，以及可选的 `prompt`。
|
||
|
||
#### 方式1：使用图片URL（推荐）
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8081/user/ai/tasks/submit" \
|
||
-H "Authorization: Bearer YOUR_JWT_OR_API_KEY" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"modelName": "rh_sora2_img_landscape",
|
||
"prompt": "镜头一（0s–3s）从空中俯拍，镜头缓缓向下俯冲穿越云层，紫蓝色霓虹反射在摩天大楼玻璃幕墙上...",
|
||
"imageUrl": "https://example.com/my-reference-image.jpg"
|
||
}'
|
||
```
|
||
|
||
#### 方式2：使用图片Base64
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8081/user/ai/tasks/submit" \
|
||
-H "Authorization: Bearer YOUR_JWT_OR_API_KEY" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"modelName": "rh_sora2_img_portrait_hd",
|
||
"prompt": "让这个场景动起来，添加生动的细节",
|
||
"imageBase64": "data:image/jpeg;base64,/9j/4AAQSkZJRg..."
|
||
}'
|
||
```
|
||
|
||
**实际发送到RunningHub的请求：**
|
||
```json
|
||
{
|
||
"webappId": "1973555366057390081",
|
||
"apiKey": "5c44cef12da3470e9f24da70c63787dc",
|
||
"nodeInfoList": [
|
||
{
|
||
"nodeId": "2",
|
||
"fieldName": "image",
|
||
"fieldValue": "https://example.com/my-reference-image.jpg",
|
||
"description": "上传图像（不支持真人做图像的输入）"
|
||
},
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "prompt",
|
||
"fieldValue": "镜头一（0s–3s）从空中俯拍...",
|
||
"description": "输入文本"
|
||
},
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "model",
|
||
"fieldValue": "landscape",
|
||
"fieldData": "[{\"name\":\"portrait\",...}]",
|
||
"description": "横竖模式"
|
||
},
|
||
{
|
||
"nodeId": "1",
|
||
"fieldName": "duration_seconds",
|
||
"fieldValue": "10",
|
||
"fieldData": "[[10, 15], {\"default\": 10}]",
|
||
"description": "时长（秒）"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 关键实现细节
|
||
|
||
### 1. 自动任务类型识别
|
||
|
||
系统根据 `providerConfig.taskType` 自动识别任务类型：
|
||
|
||
```java
|
||
// points_config表中的配置示例
|
||
{
|
||
"webappId": "1973555366057390081",
|
||
"taskType": "image2video", // 或 "text2video"
|
||
"model": "landscape",
|
||
"duration": 10
|
||
}
|
||
```
|
||
|
||
### 2. 图片参数处理
|
||
|
||
- **优先级**：`imageUrl` > `imageBase64`
|
||
- **支持格式**：
|
||
- 完整URL：`https://example.com/image.jpg`
|
||
- RunningHub文件名：`825b8cb2f5603b068704ef435df77d570f081be814a40f652f080b8d4bc6ba03.png`
|
||
- Base64编码：`data:image/jpeg;base64,/9j/4AAQSkZJRg...`
|
||
|
||
```java
|
||
// RunningHubProviderImpl中的逻辑
|
||
if (isImage2Video) {
|
||
String imageValue = request.getImageUrl();
|
||
if (imageValue == null || imageValue.trim().isEmpty()) {
|
||
imageValue = request.getImageBase64();
|
||
}
|
||
|
||
// 添加到nodeInfoList
|
||
nodeInfoList.add(RunningHubNodeInfo.builder()
|
||
.nodeId("2")
|
||
.fieldName("image")
|
||
.fieldValue(imageValue) // 支持完整URL
|
||
.description("上传图像（不支持真人做图像的输入）")
|
||
.build());
|
||
}
|
||
```
|
||
|
||
### 3. 节点顺序
|
||
|
||
系统按照RunningHub要求的顺序构建节点：
|
||
|
||
**文生视频：**
|
||
1. prompt节点（nodeId="1"）
|
||
2. model节点（nodeId="1"）
|
||
3. duration_seconds节点（nodeId="1"）
|
||
|
||
**图生视频：**
|
||
1. image节点（nodeId="2"）
|
||
2. prompt节点（nodeId="1"，可选）
|
||
3. model节点（nodeId="1"）
|
||
4. duration_seconds节点（nodeId="1"）
|
||
|
||
---
|
||
|
||
## 📊 任务流程
|
||
|
||
### 文生视频流程
|
||
|
||
```
|
||
用户提交
|
||
↓
|
||
提取prompt
|
||
↓
|
||
读取points_config配置
|
||
taskType="text2video"
|
||
webappId="1973555977595301890"
|
||
↓
|
||
构建nodeInfoList:
|
||
- prompt节点
|
||
- model节点
|
||
- duration节点
|
||
↓
|
||
调用RunningHub API
|
||
↓
|
||
获得taskId
|
||
status='processing'
|
||
↓
|
||
5秒轮询一次
|
||
↓
|
||
检测到SUCCESS
|
||
↓
|
||
获取视频URL
|
||
status='completed'
|
||
```
|
||
|
||
### 图生视频流程
|
||
|
||
```
|
||
用户提交
|
||
↓
|
||
提取prompt + imageUrl
|
||
↓
|
||
读取points_config配置
|
||
taskType="image2video"
|
||
webappId="1973555366057390081"
|
||
↓
|
||
构建nodeInfoList:
|
||
- image节点（完整URL）
|
||
- prompt节点
|
||
- model节点
|
||
- duration节点
|
||
↓
|
||
调用RunningHub API
|
||
↓
|
||
获得taskId
|
||
status='processing'
|
||
↓
|
||
5秒轮询一次
|
||
↓
|
||
检测到SUCCESS
|
||
↓
|
||
获取视频URL
|
||
status='completed'
|
||
```
|
||
|
||
---
|
||
|
||
## ⚠️ 注意事项
|
||
|
||
### 1. 图片要求
|
||
|
||
- **图生视频模型不支持真人图像作为输入**
|
||
- 建议使用风景、物体、场景类图片
|
||
- 支持完整的HTTP/HTTPS URL，无需预先上传到RunningHub
|
||
|
||
### 2. Prompt建议
|
||
|
||
- **文生视频**：详细描述镜头、场景、动作、时间节点
|
||
- **图生视频**：描述如何让图片"动起来"，添加什么动态效果
|
||
|
||
### 3. 处理时间
|
||
|
||
- 文生视频：约2-5分钟
|
||
- 图生视频：约2-5分钟（取决于服务器负载）
|
||
|
||
### 4. 积分消耗
|
||
|
||
- 任务提交时立即扣除积分（冻结）
|
||
- 任务成功：确认消耗积分
|
||
- 任务失败：自动退还积分
|
||
|
||
---
|
||
|
||
## 🧪 测试步骤
|
||
|
||
### 测试1：文生视频（竖屏10秒）
|
||
|
||
```bash
|
||
# 1. 提交任务
|
||
TASK_RESPONSE=$(curl -X POST "http://localhost:8081/user/ai/tasks/submit" \
|
||
-H "Authorization: Bearer YOUR_TOKEN" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"modelName": "rh_sora2_text_portrait",
|
||
"prompt": "一个人在海边奔跑，镜头从远到近"
|
||
}')
|
||
|
||
echo $TASK_RESPONSE
|
||
|
||
# 2. 提取taskNo
|
||
TASK_NO=$(echo $TASK_RESPONSE | jq -r '.data.taskNo')
|
||
|
||
# 3. 等待2-5分钟后查询
|
||
curl "http://localhost:8081/user/ai/tasks/$TASK_NO" \
|
||
-H "Authorization: Bearer YOUR_TOKEN"
|
||
```
|
||
|
||
### 测试2：图生视频（横屏高清）
|
||
|
||
```bash
|
||
# 1. 提交任务
|
||
curl -X POST "http://localhost:8081/user/ai/tasks/submit" \
|
||
-H "Authorization: Bearer YOUR_TOKEN" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"modelName": "rh_sora2_img_landscape_hd",
|
||
"prompt": "镜头缓缓向下俯冲穿越云层，紫蓝色霓虹反射在摩天大楼玻璃幕墙上",
|
||
"imageUrl": "https://example.com/city-skyline.jpg"
|
||
}'
|
||
|
||
# 2. 查看日志确认图片URL正确传递
|
||
tail -f logs/application.log | grep "RunningHub Provider"
|
||
```
|
||
|
||
---
|
||
|
||
## 📝 数据库配置
|
||
|
||
所有模型配置已通过 `V5__add_provider_support.sql` 自动创建：
|
||
|
||
```sql
|
||
-- 查看所有RunningHub模型
|
||
SELECT model_name, description, points_cost, provider_config
|
||
FROM points_config
|
||
WHERE provider_type = 'runninghub'
|
||
ORDER BY points_cost;
|
||
|
||
-- 查看某个模型的配置
|
||
SELECT provider_config
|
||
FROM points_config
|
||
WHERE model_name = 'rh_sora2_img_landscape';
|
||
|
||
-- 结果示例：
|
||
{
|
||
"webappId": "1973555366057390081",
|
||
"taskType": "image2video",
|
||
"model": "landscape",
|
||
"duration": 10
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🔍 故障排查
|
||
|
||
### 问题1：图生视频任务失败
|
||
|
||
**错误信息：** "图生视频任务必须提供imageUrl或imageBase64"
|
||
|
||
**解决方案：**
|
||
- 确认请求中包含 `imageUrl` 或 `imageBase64` 字段
|
||
- 检查图片URL是否可访问
|
||
- 确认使用的是图生视频模型（包含 `_img_` 的模型名）
|
||
|
||
### 问题2：任务一直处于processing状态
|
||
|
||
**可能原因：**
|
||
- RunningHub服务器负载高
|
||
- 网络连接问题
|
||
- 任务确实在处理中
|
||
|
||
**解决方案：**
|
||
```bash
|
||
# 1. 查看轮询日志
|
||
tail -f logs/application.log | grep "RunningHub轮询"
|
||
|
||
# 2. 检查数据库中的provider_task_id
|
||
SELECT task_no, provider_task_id, status, update_time
|
||
FROM ai_task
|
||
WHERE task_no = 'YOUR_TASK_NO';
|
||
|
||
# 3. 手动查询RunningHub状态（仅用于调试）
|
||
curl -X POST "https://www.runninghub.cn/task/openapi/status" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"apiKey": "YOUR_API_KEY",
|
||
"taskId": "PROVIDER_TASK_ID"
|
||
}'
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 性能监控
|
||
|
||
```sql
|
||
-- 查看RunningHub任务统计
|
||
SELECT
|
||
status,
|
||
COUNT(*) as count,
|
||
AVG(TIMESTAMPDIFF(SECOND, start_time, complete_time)) as avg_seconds
|
||
FROM ai_task
|
||
WHERE provider_type = 'runninghub'
|
||
AND create_time > DATE_SUB(NOW(), INTERVAL 1 DAY)
|
||
GROUP BY status;
|
||
|
||
-- 查看最近的失败任务
|
||
SELECT task_no, model_name, error_message, create_time
|
||
FROM ai_task
|
||
WHERE provider_type = 'runninghub'
|
||
AND status = 'failed'
|
||
ORDER BY create_time DESC
|
||
LIMIT 10;
|
||
```
|
||
|
||
---
|
||
|
||
**RunningHub Sora2集成完成！** 🎉
|
||
|
||
系统现已支持文生视频和图生视频两种模式，共12个预配置模型可供使用！
|