Skip to content

Y
yaoai-video
Commit cf966a19 authored by yaoke.yk's avatar yaoke.yk
Browse files
Options

docs: add video generation structured redesign spec

将分镜页视频生成从「@图N + 自然语言」改造为「左图 + 右提示词」结构化调用方式。 本次为修订版（v1.0.1），修正三处实施阻塞项： - 迁移版本号 V7→V14（V7 已被 billing 占用） - 复用现有 prompt 字段，不新增 final_prompt（避免语义重叠） - 明确老接口 POST /video-tasks/generate 为替换目标，补分镜页迁移路径 Co-Authored-By: 's avatarClaude Opus 4.7 <noreply@anthropic.com>
parent 5bc25c5f
Hide whitespace changes
Inline Side-by-side
Showing with 932 additions and 821 deletions
design-docs/video-generation-structured-redesign.md 0 → 100644
View file @ cf966a19
# 视频生成结构化改造计划
## 一、改造目标
将用户手动写 `@图1 + 场景图` 的视频生成方式，改造成 **"左侧结构化素材 + 右侧提示词"** 的工程化调用方式。
---
## 二、用户需求确认
### 2.1 改造范围
- ✅ 前端 UI 改造
- ✅ 后端接口改造
- ✅ Prompt 拼接逻辑
- ✅ 数据库扩展
### 2.2 素材选择方式
- ✅ 从项目角色库选择
- ✅ 从项目场景库选择
- ✅ 支持本地上传
- ✅ 支持混合使用
### 2.3 视频参数
- ✅ 模型选择（Seedance 2.0 / Seedance 2.0 Fast）
- ✅ 时长调节（5s/10s/15s/20s/30s）
- ✅ 宽高比（16:9 / 9:16 / 1:1）
- ✅ 生成配音开关
---
## 三、数据结构设计
### 3.1 前端请求数据结构
```typescript
// 用户结构化提示词
interface UserPrompt {
characterAction?: string; // 人物动作
sceneEvent?: string; // 场景事件
cameraMovement?: string; // 镜头运动
lightingAtmosphere?: string; // 光影氛围
videoStyle?: string; // 视频风格
}
// 角色视图类型
type CharacterView = "front" | "side" | "back";
// 角色选择数据
interface CharacterSelection {
characterId: string;
view: CharacterView; // 选择哪个视图
imageKey: string; // 实际使用的 TOS key
}
// 结构化视频生成请求
interface StructuredVideoGenerateRequest {
// 必填
character: CharacterSelection; // 角色选择（含视图）
sceneImageKey: string; // 场景图 TOS key
userPrompt: UserPrompt; // 结构化提示词
// 可选
propImageKeys: string[]; // 道具图 TOS keys（本地上传）
styleImageKey: string; // 风格图 TOS key（本地上传）
// 参数
model: "seedance-2.0" | "seedance-2.0-fast";
duration: 5 | 10 | 15 | 20 | 30;
ratio: "16:9" | "9:16" | "1:1";
generateAudio: boolean;
}
// 发送给后端的请求格式（序列化后）
interface ApiVideoGenerateRequest {
character_image_key: string;
scene_image_key: string;
user_prompt: string; // JSON 字符串化
prop_image_keys?: string[];
style_image_key?: string;
model: string;
duration: number;
ratio: string;
generate_audio: boolean;
}
```
### 3.2 后端扩展字段
#### 现有字段语义（保留，复用）
| 字段名 | 现状 | 在结构化场景下的角色 |
|--------|------|-----------|
| `prompt` | 任务的最终 Prompt 字符串（旧实现里就是用户输入） | **复用为 `final_prompt`**：系统拼接后提交给 Seedance 的完整 Prompt（含 @图N） |
| `input_image_key` | 旧版「图生视频」单图入参（`POST /video-tasks` 用） | **保留给旧任务**，新结构化任务该列为 NULL |
> **决定**：不新增 `final_prompt` 字段——旧 `prompt` 字段语义即「最终提交给模型的 Prompt」，结构化场景延用即可。新增字段 `user_prompt`（JSON，原始结构化输入）与 `prompt`（拼接后字符串）配对，便于回溯。
#### 新增字段
| 字段名 | 类型 | 说明 |
|--------|------|------|
| character_image_key | VARCHAR | 角色图 TOS key（图1） |
| scene_image_key | VARCHAR | 场景图 TOS key（图2） |
| prop_image_keys | TEXT | 道具图 TOS keys（JSON数组，图3+） |
| style_image_key | VARCHAR | 风格图 TOS key（可选，最后一张） |
| user_prompt | TEXT | 用户原始结构化提示词（JSON，用于追溯） |
| video_duration | INT | 视频时长（秒） |
| video_ratio | VARCHAR | 视频宽高比 |
| generate_audio | BOOLEAN | 是否生成配音 |
### 3.3 后端 DTO 定义
```java
// 用户结构化提示词
@Data
public class UserPrompt {
private String characterAction;
private String sceneEvent;
private String cameraMovement;
private String lightingAtmosphere;
private String videoStyle;
}
// 结构化视频生成请求
@Data
public class StructuredVideoGenerateRequest {
@NotBlank(message = "角色图不能为空")
private String characterImageKey;
@NotBlank(message = "场景图不能为空")
private String sceneImageKey;
@NotNull(message = "用户提示词不能为空")
@Valid
private UserPrompt userPrompt;
private List<String> propImageKeys;
private String styleImageKey;
@Pattern(regexp = "seedance-2-0|seedance-2-0-fast", message = "模型参数错误")
private String model = "seedance-2-0";
@Min(5) @Max(30)
private Integer duration = 15;
@Pattern(regexp = "16:9|9:16|1:1", message = "宽高比参数错误")
private String ratio = "16:9";
private Boolean generateAudio = false;
}
```
---
## 四、前端改造方案
### 4.1 新增组件：StructuredVideoGenerator
位置：`doc/html/src/app/components/StructuredVideoGenerator.tsx`
#### 4.1.1 布局结构
```
┌─────────────────────────────────────────────────────────────┐
│ 分镜视频生成器 [展开/收起] │
├──────────────┬──────────────────────────────────────────────┤
│ 素材选择区 │ 提示词编辑区 │
│ │ │
│ [角色图] │ ┌────────────────────────────────────┐ │
│ ○ 角色库选择 │ │ 人物动作： │ │
│ ○ 本地上传 │ │ │ │
│ [预览] │ │ │ │
│ │ │ 场景事件： │ │
│ [场景图] │ │ │ │
│ ○ 场景库选择 │ │ │ │
│ ○ 本地上传 │ │ 镜头运动： │ │
│ [预览] │ │ │ │
│ │ │ 光影氛围： │ │
│ [道具图] + │ │ │ │
│ [道具图] + │ │ 视频风格： │ │
│ │ └────────────────────────────────────┘ │
│ [风格图] │ │
│ ○ 本地上传 │ AI 生成描述 保存 │
│ [预览] │ │
├──────────────┴──────────────────────────────────────────────┤
│ 模型: Seedance 2.0 ▼ 时长: 15s ▼ 比例: 16:9 ▼ 配音: ✅ │
│ [生成视频 (2500积分)] │
└─────────────────────────────────────────────────────────────┘
```
#### 4.1.2 素材选择组件
**角色图选择器** (`CharacterImageSelector`)
- 从项目角色库下拉选择
- 选择角色后，显示三视图缩略图（正面/侧面/背面）
- 默认选中正面视图，用户可切换
- 支持点击"上传"按钮上传临时角色图
- 上传后可预览
**视图切换逻辑**：
```tsx
<div className="flex gap-2 mt-2">
<button
onClick={() => setView("front")}
className={view === "front" ? "ring-2 ring-primary" : ""}
>
<img src={character.frontImageUrl} alt="正面" />
<span>正面</span>
</button>
<button
onClick={() => setView("side")}
className={view === "side" ? "ring-2 ring-primary" : ""}
>
<img src={character.sideImageUrl} alt="侧面" />
<span>侧面</span>
</button>
<button
onClick={() => setView("back")}
className={view === "back" ? "ring-2 ring-primary" : ""}
>
<img src={character.backImageUrl} alt="背面" />
<span>背面</span>
</button>
</div>
```
**视图映射**：
- 正面 → 使用 `character.frontImageTosKey` 或 `character.imageTosKey`
- 侧面 → 使用 `character.sideImageTosKey`
- 背面 → 使用 `character.backImageTosKey`
**场景图选择器** (`SceneImageSelector`)
- 从项目场景库下拉选择
- 支持本地上传
**道具图选择器** (`PropImageSelector`)
- 支持多选
- 当前仅支持本地上传（后续扩展：从道具库选择）
- 每个道具显示缩略图，可单独删除
- 支持拖拽排序（影响 Prompt 中 @图N 的顺序）
- 添加按钮：`+ 添加道具`
> **注**：V1.0 阶段道具仅支持本地上传。V1.5 将新增 `props` 表支持从库选择。
**风格图选择器** (`StyleImageSelector`)
- 仅支持本地上传
- 单图
### 4.2 提示词输入区改造
移除原有 `@图N` 引用方式，改为结构化输入框：
```tsx
<div className="space-y-3">
<div>
<label>人物动作</label>
<textarea placeholder="例如：陆长生站在高楼边缘，俯瞰城市..." />
</div>
<div>
<label>场景事件</label>
<textarea placeholder="例如：城市开始崩塌，建筑倒塌..." />
</div>
<div>
<label>镜头运动</label>
<textarea placeholder="例如：镜头缓慢推进，景深变化..." />
</div>
<div>
<label>光影氛围</label>
<textarea placeholder="例如：夕阳余晖，尘土飞扬，史诗感..." />
</div>
<div>
<label>视频风格</label>
<textarea placeholder="例如：电影质感，暗黑科幻风格..." />
</div>
</div>
```
### 4.3 参数选择区
```tsx
<div className="flex gap-2">
<select value={model} onChange={(e) => setModel(e.target.value)}>
<option value="seedance-2.0">Seedance 2.0</option>
<option value="seedance-2.0-fast">Seedance 2.0 Fast</option>
</select>
<select value={duration} onChange={(e) => setDuration(Number(e.target.value))}>
<option value={5}>5秒</option>
<option value={10}>10秒</option>
<option value={15}>15秒</option>
<option value={20}>20秒</option>
<option value={30}>30秒</option>
</select>
<select value={ratio} onChange={(e) => setRatio(e.target.value)}>
<option value="16:9">16:9 (横屏)</option>
<option value="9:16">9:16 (竖屏)</option>
<option value="1:1">1:1 (方形)</option>
</select>
<label>
<input type="checkbox" checked={generateAudio} onChange={...} />
生成配音
</label>
</div>
```
---
## 五、后端接口改造
### 5.1 新增接口
**POST** `/projects/{projectId}/video-tasks/generate-structured`
#### 请求体
```json
{
"character_image_key": "tos://yaoai/characters/xxx.png",
"scene_image_key": "tos://yaoai/scenes/yyy.png",
"user_prompt": {
"characterAction": "陆长生站在高楼边缘",
"sceneEvent": "城市开始崩塌",
"cameraMovement": "镜头缓慢推进",
"lightingAtmosphere": "夕阳余晖，尘土飞扬",
"videoStyle": "暗黑科幻，电影质感"
},
"prop_image_keys": ["tos://yaoai/props/zzz.png", "tos://yaoai/props/aaa.png"],
"style_image_key": "tos://yaoai/styles/www.png",
"model": "seedance-2.0",
"duration": 15,
"ratio": "16:9",
"generate_audio": true
}
```
**字段说明**：
- `user_prompt`：用户结构化输入（JSON 对象）
- `prop_image_keys`：道具图数组，顺序对应图3、图4...
- `style_image_key`：风格图，对应最后一张图（图6，如果有2个道具的话）
#### 响应
```json
{
"code": 0,
"message": "success",
"data": {
"id": "123456789",
"status": "pending",
"createdAt": "2026-04-25T10:00:00Z"
}
}
```
#### 错误响应示例
```json
{
"code": 400,
"message": "参数校验失败",
"errors": [
"角色图不能为空",
"请填写至少一项分镜内容"
]
}
```
#### 完整响应 DTO
```json
{
"id": "123456789",
"status": "pending",
"characterImageKey": "tos://yaoai/characters/xxx.png",
"sceneImageKey": "tos://yaoai/scenes/yyy.png",
"propImageKeys": ["tos://yaoai/props/zzz.png"],
"styleImageKey": "tos://yaoai/styles/www.png",
"userPrompt": {
"characterAction": "陆长生站在高楼边缘",
"sceneEvent": "城市开始崩塌"
},
"finalPrompt": "@图1 是角色参考图...\n@图2 是场景参考图...",
"videoDuration": 15,
"videoRatio": "16:9",
"generateAudio": true,
"model": "seedance-2.0",
"createdAt": "2026-04-25T10:00:00Z",
"updatedAt": "2026-04-25T10:00:00Z"
}
```
### 5.2 Controller 改造
```java
@Operation(summary = "结构化视频生成（推荐方式）")
@PostMapping("/generate-structured")
public ApiResponse<AiTaskDTO> generateStructured(
@PathVariable Long projectId,
@RequestBody StructuredVideoGenerateRequest request
) {
// 1. 验证必填字段
// 2. 拼接最终 Prompt
// 3. 创建 pending 任务
// 4. 异步处理
return ApiResponse.success(AiTaskDTO.from(task));
}
```
---
## 六、Prompt 拼接逻辑
### 6.1 工具类：SeedancePromptBuilder
位置：`yaoai-comic-studio/yaoai-ai-providers/src/main/java/com/yaoai/ai/providers/util/SeedancePromptBuilder.java`
```java
public class SeedancePromptBuilder {
/**
* 构建最终 Prompt，包含图片说明和用户输入
*
* @param request 结构化请求
* @return 最终 Prompt（含 @图N 标注）
*/
public static String buildPrompt(StructuredVideoGenerateRequest request) {
StringBuilder sb = new StringBuilder();
// 1. 固定图说明
sb.append("@图1 是角色参考图，请保持人物脸型、发型、服装、气质一致，不要随意改变角色身份。\n");
sb.append("@图2 是场景参考图，请参考空间结构、环境布局、光影、氛围和构图。\n");
// 2. 道具说明（图3 开始）
int imageCount = 2; // 角色 + 场景
if (request.getPropImageKeys() != null && !request.getPropImageKeys().isEmpty()) {
int propCount = request.getPropImageKeys().size();
int propStart = imageCount + 1;
int propEnd = propStart + propCount - 1;
if (propCount == 1) {
sb.append("@图").append(propStart).append(" 是道具参考图，请在画面中合理使用。\n");
} else {
sb.append("@图").append(propStart).append("-").append(propEnd)
.append(" 是道具参考图，请在画面中合理使用这些道具。\n");
}
imageCount += propCount;
}
// 3. 风格说明（最后一张，序号动态计算）
if (request.getStyleImageKey() != null && !request.getStyleImageKey().isBlank()) {
sb.append("@图").append(imageCount + 1).append(" 是风格参考图，请参考整体美术风格、色彩和质感。\n");
}
// 4. 用户输入的分镜内容
sb.append("\n分镜内容：\n");
if (request.getUserPrompt() != null) {
UserPrompt up = request.getUserPrompt();
if (up.getCharacterAction() != null && !up.getCharacterAction().isBlank()) {
sb.append("- 人物动作：").append(up.getCharacterAction()).append("\n");
}
if (up.getSceneEvent() != null && !up.getSceneEvent().isBlank()) {
sb.append("- 场景事件：").append(up.getSceneEvent()).append("\n");
}
if (up.getCameraMovement() != null && !up.getCameraMovement().isBlank()) {
sb.append("- 镜头运动：").append(up.getCameraMovement()).append("\n");
}
if (up.getLightingAtmosphere() != null && !up.getLightingAtmosphere().isBlank()) {
sb.append("- 光影氛围：").append(up.getLightingAtmosphere()).append("\n");
}
if (up.getVideoStyle() != null && !up.getVideoStyle().isBlank()) {
sb.append("- 视频风格：").append(up.getVideoStyle()).append("\n");
}
}
// 5. 生成要求
sb.append("\n生成要求：\n");
sb.append("- 保持角色一致性\n");
sb.append("- 保持场景逻辑合理\n");
sb.append("- 人物与环境融合自然\n");
sb.append("- 镜头运动流畅\n");
sb.append("- 画面电影感\n");
sb.append("- 不要出现多余人物\n");
sb.append("- 不要出现明显变脸、错位、肢体畸形\n");
return sb.toString();
}
/**
* 构建参考图片列表（按顺序传给 Seedance API）
*
* @param request 结构化请求
* @return 图片 URL 列表
*/
public static List<String> buildImageUrls(StructuredVideoGenerateRequest request, TosService tosService) {
List<String> urls = new ArrayList<>();
// 图1：角色图
urls.add(tosService.presignedGetUrl(request.getCharacterImageKey(), 3600));
// 图2：场景图
urls.add(tosService.presignedGetUrl(request.getSceneImageKey(), 3600));
// 图3+：道具图
if (request.getPropImageKeys() != null) {
for (String key : request.getPropImageKeys()) {
urls.add(tosService.presignedGetUrl(key, 3600));
}
}
// 最后一张：风格图
if (request.getStyleImageKey() != null && !request.getStyleImageKey().isBlank()) {
urls.add(tosService.presignedGetUrl(request.getStyleImageKey(), 3600));
}
return urls;
}
}
```
### 6.2 图片顺序示例
| 图序号 | 类型 | 来源 |
|--------|------|------|
| @图1 | 角色图 | character_image_key（固定） |
| @图2 | 场景图 | scene_image_key（固定） |
| @图3 | 道具图 | prop_image_keys[0]（可选） |
| @图4 | 道具图 | prop_image_keys[1]（可选） |
| @图5 | 道具图 | prop_image_keys[2]（可选） |
| @图6 | 风格图 | style_image_key（可选，最后一张） |
**示例 1**：角色 + 场景 + 3个道具 + 风格
```
图1=角色, 图2=场景, 图3-5=道具, 图6=风格
```
**示例 2**：角色 + 场景 + 风格
```
图1=角色, 图2=场景, 图3=风格
```
---
## 七、数据库迁移
### 7.1 迁移脚本
位置：`yaoai-comic-studio/yaoai-bootstrap/src/main/resources/db/migration/V14__extend_ai_tasks_structured.sql`
> 仓库迁移现状：V1~V13 已存在（V7 是 billing，V11 是角色三视图，V13 是 admin 后台）。本次迁移占用 **V14**。
```sql
-- V14__extend_ai_tasks_structured.sql
-- 扩展 ai_tasks 表，支持结构化视频生成
-- 注意：复用现有 prompt 字段作为 final_prompt，不新增 final_prompt 字段，避免语义重叠
-- 现有 input_image_key 保留给旧任务，新结构化任务该字段为 NULL
ALTER TABLE ai_tasks
ADD COLUMN character_image_key VARCHAR(255) COMMENT '角色图 TOS key（图1）',
ADD COLUMN scene_image_key VARCHAR(255) COMMENT '场景图 TOS key（图2）',
ADD COLUMN prop_image_keys TEXT COMMENT '道具图 TOS keys（JSON数组，图3+）',
ADD COLUMN style_image_key VARCHAR(255) COMMENT '风格图 TOS key（可选，最后一张）',
ADD COLUMN user_prompt TEXT COMMENT '用户原始结构化提示词（JSON，用于追溯，与 prompt 字段配对）',
ADD COLUMN video_duration INT DEFAULT 15 COMMENT '视频时长（秒）',
ADD COLUMN video_ratio VARCHAR(10) DEFAULT '16:9' COMMENT '视频宽高比',
ADD COLUMN generate_audio BOOLEAN DEFAULT FALSE COMMENT '是否生成配音';
-- 添加索引，提升查询性能
CREATE INDEX idx_ai_tasks_character_key ON ai_tasks(character_image_key);
CREATE INDEX idx_ai_tasks_scene_key ON ai_tasks(scene_image_key);
CREATE INDEX idx_ai_tasks_duration ON ai_tasks(video_duration);
CREATE INDEX idx_ai_tasks_ratio ON ai_tasks(video_ratio);
```
### 7.2 实体更新
```java
@Data
@TableName("ai_tasks")
public class AiTask {
// ... 原有字段（id / tenantId / projectId / episodeId / storyboardId / taskType /
// externalTaskId / status / inputImageKey / prompt /
// resultVideoUrl / resultTosKey / errorMessage / createdAt / updatedAt）
//
// 字段语义说明：
// - prompt：复用为「最终提交给 Seedance 的完整 Prompt」（含 @图N），新旧任务通用
// - inputImageKey：仅旧版单图任务使用，新结构化任务为 null
// - taskType："image_to_video" / "text_to_video" / "ref_image_to_video" /
// "structured_video"（新增）
// 结构化视频生成字段（新增）
private String characterImageKey; // 角色 TOS key（图1）
private String sceneImageKey; // 场景 TOS key（图2）
private String propImageKeys; // 道具 TOS keys（JSON 数组字符串）
private String styleImageKey; // 风格 TOS key（可选，最后一张）
private String userPrompt; // 用户原始结构化输入（JSON 字符串，用于追溯）
private Integer videoDuration; // 视频时长（秒）
private String videoRatio; // 宽高比
private Boolean generateAudio; // 是否生成配音
}
```
### 7.3 DTO 更新
```java
@Data
public class AiTaskDTO {
// ... 原有字段（含 prompt → 即 finalPrompt 语义）
private String characterImageKey;
private String sceneImageKey;
private List<String> propImageKeys; // 从 JSON 反序列化
private String styleImageKey;
private Map<String, String> userPrompt; // 原始用户输入（结构化）
// finalPrompt 不单独建字段，前端用现有 prompt 字段（语义已是「最终提交的 Prompt」）
private Integer videoDuration;
private String videoRatio;
private Boolean generateAudio;
public static AiTaskDTO from(AiTask task) {
AiTaskDTO dto = new AiTaskDTO();
// ... 原有映射
dto.setCharacterImageKey(task.getCharacterImageKey());
dto.setSceneImageKey(task.getSceneImageKey());
// JSON 反序列化
if (task.getPropImageKeys() != null) {
try {
dto.setPropImageKeys(OBJECT_MAPPER.readValue(
task.getPropImageKeys(),
new TypeReference<List<String>>() {}
));
} catch (Exception e) {
log.warn("Failed to parse propImageKeys: {}", task.getPropImageKeys());
}
}
if (task.getUserPrompt() != null) {
try {
dto.setUserPrompt(OBJECT_MAPPER.readValue(
task.getUserPrompt(),
new TypeReference<Map<String, String>>() {}
));
} catch (Exception e) {
log.warn("Failed to parse userPrompt: {}", task.getUserPrompt());
}
}
dto.setStyleImageKey(task.getStyleImageKey());
// finalPrompt 已在 dto.setPrompt(task.getPrompt()) 处映射，无需重复
dto.setVideoDuration(task.getVideoDuration());
dto.setVideoRatio(task.getVideoRatio());
dto.setGenerateAudio(task.getGenerateAudio());
return dto;
}
}
```
---
## 八、改造流程
### 8.1 实施步骤
| 步骤 | 任务 | 文件 | 预估工时 |
|------|------|------|----------|
| 1 | 数据库迁移 | V7__extend_ai_tasks.sql | 0.5h |
| 2 | 后端实体更新 | AiTask.java | 0.5h |
| 3 | Prompt 工具类 | SeedancePromptBuilder.java | 1.5h |
| 4 | DTO 定义 | StructuredVideoGenerateRequest.java, UserPrompt.java, AiTaskDTO.java | 1.5h |
| 5 | Controller 新接口 | VideoTaskController.java | 1h |
| 6 | Service 层改造 | VideoTaskPipelineServiceImpl.java | 2.5h |
| 7 | 角色选择器 | CharacterImageSelector.tsx | 2h |
| 8 | 场景选择器 | SceneImageSelector.tsx | 1h |
| 9 | 道具选择器 | PropImageSelector.tsx | 1.5h |
| 10 | 风格选择器 | StyleImageSelector.tsx | 0.5h |
| 11 | 主生成器组件 | StructuredVideoGenerator.tsx | 3h |
| 12 | 集成到工作区 | StoryboardWorkspace.tsx | 2h |
| 13 | 测试与调试 | - | 3h |
**总计：约 20 小时**
### 8.2 向后兼容
#### 现有 video-tasks 接口现状
`VideoTaskController.java` 当前已有三个写入端点：
| 路径 | 用途 | 在改造中的去向 |
|------|------|----------------|
| `POST /video-tasks` | 单图 → 视频（直接传 `image_key + prompt`） | 保留，给后台/脚本/工具用 |
| `POST /video-tasks/generate` | 文生图 → 图生视频（异步，接 `image_keys[]` + `duration` + 自然语言 `@图N`） | **本次改造的主要替换目标** —— 分镜页前端从此接口切到新 `/generate-structured` |
| `POST /video-tasks/generate-structured`（新） | 结构化视频生成（左图 + 右提示词） | 新接口 |
#### 切换策略
- **`/generate` 接口暂不删**：标 `@Deprecated`，前端 `aiApi.generateVideo`（`doc/html/src/lib/api/ai.ts:227`）保留 1 个版本周期，分镜页（`StoryboardWorkspace.tsx`）切到 `aiApi.generateVideoStructured` 后，旧方法仅用于回滚兜底
- **数据库不分表**：新老任务都进 `ai_tasks`，靠新增的 `character_image_key` / `user_prompt` 等字段是否非空判断任务来源
- **旧任务**：新增字段全部为 null，列表查询/详情查询保持兼容
- **`taskType` 区分**：新结构化任务统一 `taskType="structured_video"`，老的保持原值（`text_to_video` / `ref_image_to_video` / `image_to_video`）
#### 分镜页（StoryboardWorkspace）改造要点
- 当前页面用 `aiApi.generateVideo(...)` 把 `prompt` 中嵌入的 `@图N`（用户从右侧参考图面板点击插入）+ 顺序化的 `image_keys[]` 一起传给后端（`StoryboardWorkspace.tsx:237/272/294`）
- 改造后：右侧不再是「@图N 引用面板」，而是 4.1.1 节描述的「素材选择区（角色/场景/道具/风格）+ 结构化提示词区」
- 用户原本在 prompt 里的自然语言文本 → 拆进 `userPrompt.{characterAction, sceneEvent, cameraMovement, lightingAtmosphere, videoStyle}` 五个字段
- @图N 的拼接交给后端 `SeedancePromptBuilder`（第六节）
**前端切换策略**：
```tsx
// 在 Settings 中添加切换开关
const [useStructuredMode, setUseStructuredMode] = useState(true);
// 组件中使用
if (useStructuredMode) {
return <StructuredVideoGenerator />;
} else {
return <LegacyVideoGenerator />; // 旧版文本框方式
}
```
**数据迁移**：
- 已有任务的新字段默认为 null
- 查询时兼容两种方式：有结构化字段用结构化，否则用旧逻辑
- 不需要数据迁移脚本（增量兼容）
**API 版本标识**：
- 旧接口：`POST /video-tasks/generate` (v1，分镜页历史调用，自然语言 + image_keys 顺序绑定)
- 新接口：`POST /video-tasks/generate-structured` (v2，推荐，左图 + 右提示词)
---
### 8.3 Props 表扩展（V1.5）
**当前（V1.0）**：道具仅支持本地上传
**V1.5 目标**：
```sql
CREATE TABLE props (
id BIGINT PRIMARY KEY AUTO_INCREMENT,
tenant_id BIGINT NOT NULL,
project_id BIGINT NOT NULL,
name VARCHAR(100) NOT NULL,
description TEXT,
image_tos_key VARCHAR(255) NOT NULL,
image_url VARCHAR(500),
status VARCHAR(20) DEFAULT 'draft',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_tenant_project (tenant_id, project_id)
) COMMENT='项目道具表';
```
---
## 九、校验规则
### 9.1 前端校验
```typescript
function validateRequest(req: StructuredVideoGenerateRequest): string[] {
const errors: string[] = [];
// 必填校验
if (!req.character?.imageKey) {
errors.push("请选择角色图");
}
if (!req.sceneImageKey) {
errors.push("请选择场景图");
}
// 提示词校验：至少填写一项
const up = req.userPrompt;
const hasContent = Object.values(up || {}).some(v => v && v.trim() !== "");
if (!hasContent) {
errors.push("请填写至少一项分镜内容（人物动作/场景事件/镜头运动/光影氛围/视频风格）");
}
// 检测是否手动写了 @图（警告，不报错）
const hasAtImage = Object.values(up || {})
.some(v => v && v.includes("@图"));
if (hasAtImage) {
// 可以是 warning 而非 error
console.warn("提示：系统已自动绑定图1为角色图、图2为场景图，无需手动填写 @图");
}
return errors;
}
// 使用示例
const errors = validateRequest(request);
if (errors.length > 0) {
// 显示错误提示
toast.error(errors[0]);
return;
}
```
### 9.2 后端校验
```java
private void validateRequest(StructuredVideoGenerateRequest request) {
if (StringUtils.isBlank(request.getCharacterImageKey())) {
throw new BizException(ErrorCode.INVALID_PARAM, "角色图不能为空");
}
if (StringUtils.isBlank(request.getSceneImageKey())) {
throw new BizException(ErrorCode.INVALID_PARAM, "场景图不能为空");
}
if (request.getUserPrompt() == null || isEmpty(request.getUserPrompt())) {
throw new BizException(ErrorCode.INVALID_PARAM, "用户提示词不能为空");
}
}
```
---
## 十、重要约束
### 10.1 图片顺序固定
图片顺序严格按以下规则排列，Prompt 中的 @图N 对应：
| 序号 | 字段来源 | 说明 |
|------|----------|------|
| @图1 | character_image_key | 角色图（必选） |
| @图2 | scene_image_key | 场景图（必选） |
| @图3+ | prop_image_keys[0..n] | 道具图（可选，按顺序） |
| 最后一张 | style_image_key | 风格图（可选） |
**序号计算规则**：
```
总图数 = 2（角色+场景）+ 道具数 + (有风格图 ? 1 : 0)
示例1：2个道具 + 风格 → 2+2+1=5张，风格图是@图5
示例2：3个道具 + 风格 → 2+3+1=6张，风格图是@图6
示例3：无道具 + 风格 → 2+0+1=3张，风格图是@图3
示例4：无道具无风格 → 2+0+0=2张，只有@图1和@图2
```
### 10.2 用户输入规范
**必须**：
- `user_prompt` 是结构化对象，不是字符串
- 至少填写一项：人物动作 / 场景事件 / 镜头运动 / 光影氛围 / 视频风格
**禁止**：
- 用户不要手动写 `@图1`、`@图2` 等
- 如检测到用户手写 @图，前端 console.warn 警告（不报错）
**保存规范**：
- `user_prompt` 字段存储 JSON 字符串（原始输入）
- `final_prompt` 字段存储系统拼接后的完整 Prompt（含 @图N）
- 两字段都保存，方便复盘和问题排查
### 10.3 角色视图映射
| 用户选择 | 使用的字段 | 后端存储 |
|----------|-----------|----------|
| 正面 | frontImageTosKey / imageTosKey | character_image_key |
| 侧面 | sideImageTosKey | character_image_key |
| 背面 | backImageTosKey | character_image_key |
### 10.4 向后兼容
- 旧接口保留，标记 `@Deprecated`
- 新任务使用新字段，旧任务字段为 null
- 查询时兼容两种方式
---
## 十一、后续优化
### V1.1（短期）
1. **生成模板保存**：支持保存为"生成模板"，快速复用
2. **历史记录**：提供历史记录面板，支持一键恢复
3. **AI 辅助填写**：根据选中的角色和场景，AI 自动生成初始提示词
4. **预览功能**：显示图片顺序和最终 Prompt
### V1.5（中期）
5. **道具库表**：创建 `props` 表，支持从库选择道具
6. **批量生成**：支持多个分镜批量应用同一素材和提示词
7. **风格市场**：提供预设风格库，一键应用
### V2.0（长期）
8. **多角色支持**：一个分镜支持多个角色（图1、图2 为角色1、角色2）
9. **场景分镜**：支持场景本身的分镜描述和参考图
10. **AI 动作推荐**：根据提示词推荐合适的镜头运动
---
## 附录：参考资料
### A. Seedance 2.0 API 格式
```json
{
"model": "doubao-seedance-2-0",
"content": [
{ "type": "text", "text": "@图1 是角色参考图..." },
{ "type": "image_url", "image_url": { "url": "https://..." }, "role": "reference_image" },
{ "type": "image_url", "image_url": { "url": "https://..." }, "role": "reference_image" }
],
"generate_audio": true,
"ratio": "16:9",
"duration": 15,
"watermark": false
}
```
### B. 错误码定义
| 错误码 | 说明 |
|--------|------|
| 4001 | 角色图不能为空 |
| 4002 | 场景图不能为空 |
| 4003 | 用户提示词不能为空 |
| 4004 | 参数格式错误 |
| 5001 | 视频生成任务提交失败 |
| 5002 | Seedance API 调用失败 |
### C. 版本历史
| 版本 | 日期 | 变更 |
|------|------|------|
| 1.0 | 2026-04-25 | 初始版本，结构化视频生成 |
| 1.0.1 | 2026-04-25 | 修订：①迁移版本号 V7→V14（V7 已被 billing 占用）；②不新增 `final_prompt` 字段，复用现有 `prompt`；③明确「老接口」指 `POST /video-tasks/generate`，并补充分镜页改造路径 |
| 1.1 | 待定 | 模板、历史、AI 辅助 |
| 1.5 | 待定 | 道具库、批量生成 |
| 2.0 | 待定 | 多角色、场景分镜、AI 推荐 |
docs/superpowers/specs/2026-04-24-canvas-mode-project-workbench-design.md deleted 100644 → 0
View file @ 5bc25c5f
# Canvas Mode Project Workbench Design
## Summary
Introduce a new project mode named `canvas` alongside the existing `normal` mode in `yaoaivideo`.
- `normal` mode keeps the current step-based project flow unchanged.
- `canvas` mode opens a new project-level workbench centered on a visual canvas.
- The canvas workbench is the main operating surface for canvas projects, but it reuses the existing domain objects for projects, episodes, characters, scenes, storyboards, video tasks, and assembly tasks.
- Canvas-specific data such as node layout, grouping, links, viewport, and action history is stored separately.
This design intentionally avoids project-mode switching in v1. A project chooses its mode at creation time and keeps it permanently.
## Goals
- Add a second creation path for projects: `normal` or `canvas`.
- Provide a new project-level workbench page for canvas projects.
- Visualize script, episodes, characters, scenes, storyboards, video tasks, and assembly tasks on a single canvas.
- Allow users to control generation actions from the canvas.
- Allow users to organize and edit the canvas using drag, selection, grouping, linking, and batch operations.
- Reuse the existing backend pipeline and API capabilities wherever possible.
- Keep the current normal-mode experience stable and isolated from the new workbench.
## Non-Goals
- No switching between `normal` and `canvas` after project creation.
- No attempt to make the canvas the sole source of truth for all business data in v1.
- No direct copy-paste migration of the Tapnow monolithic `src/App.jsx` architecture.
- No iframe or separately deployed micro-frontend workbench.
- No full free-form workflow engine in v1.
## Product Decisions
### Project Mode
Add a project mode field with at least the following values:
- `normal`
- `canvas`
Behavior:
- New projects default to `normal` unless the user actively chooses `canvas`.
- A `normal` project continues to use the current page sequence.
- A `canvas` project opens into a new workbench route and uses a canvas-first workflow.
- Mode is immutable after creation.
### User Experience Strategy
For `canvas` projects:
- The new workbench becomes the primary entry point.
- Existing detail pages can remain available as support pages for editing or fallback operations.
- The top project navigation changes to expose `工作台` instead of forcing users through the current step-by-step flow.
For `normal` projects:
- No behavior change.
- Existing routes, tabs, and project detail flow remain intact.
## Existing System Fit
This design is aligned with the current `yaoaivideo` structure:
- Frontend routing already supports project-scoped pages in `doc/html/src/app/routes.tsx`.
- Project top tabs already exist in `doc/html/src/app/components/Layout.tsx`.
- New project creation already flows through `doc/html/src/app/pages/NewProject.tsx`.
- Frontend data hooks and APIs already exist for projects and AI flows, including:
- `doc/html/src/lib/api/projects.ts`
- `doc/html/src/lib/api/ai.ts`
- `doc/html/src/hooks/useProjects.ts`
- `doc/html/src/hooks/useAi.ts`
- Backend domain and controller structure already supports projects, storyboards, video tasks, and agent runs.
This makes the workbench a host-integrated feature rather than a separate product.
## Architecture Overview
The recommended architecture has four layers.
### 1. Domain Truth Layer
Keep existing domain tables and services as the source of truth for:
- projects
- outlines
- episodes
- characters
- scenes
- storyboards
- video tasks
- assembly tasks
The canvas will not replace these tables in v1.
### 2. Canvas Orchestration Layer
Introduce a separate workbench data model to store:
- nodes
- edges
- viewport
- groups
- layout
- per-node display state
- canvas action history
- snapshots
This layer references existing business objects instead of duplicating them.
### 3. Workbench Control Layer
Add a dedicated backend module surface for canvas-oriented actions such as:
- bootstrap workbench from project data
- save canvas layout
- trigger extraction and generation actions from nodes
- expose aggregated workbench state
- record action history
This layer should internally call existing services and pipelines whenever possible.
### 4. Frontend Workbench Layer
Build a new modular workbench frontend inside the current `doc/html` app:
- page shell
- canvas renderer
- inspector
- library panel
- action toolbar
- bottom task panel
- workbench store
- workbench API layer
## Data Model Design
### Extend Existing Project Data
Add `projectMode` to project-level data.
Backend changes:
- `projects` table: add `project_mode`
- `Project` entity: add `projectMode`
- `ProjectCreateRequest`: accept `projectMode`
- `ProjectDTO`: expose `projectMode`
- project create/update service: validate and persist it
Recommended values:
- `normal`
- `canvas`
Default:
- `normal`
### New Workbench Tables
Recommended new tables:
#### `project_workbenches`
Purpose:
- One primary workbench record per project.
- Stores global canvas state.
Suggested fields:
- `id`
- `project_id`
- `tenant_id`
- `version`
- `viewport_x`
- `viewport_y`
- `zoom`
- `layout_mode`
- `created_at`
- `updated_at`
Constraints:
- unique key on `project_id`
#### `project_workbench_nodes`
Purpose:
- Stores canvas nodes.
Suggested fields:
- `id`
- `workbench_id`
- `project_id`
- `tenant_id`
- `node_type`
- `ref_type`
- `ref_id`
- `title`
- `status`
- `x`
- `y`
- `width`
- `height`
- `config_json`
- `meta_json`
- `sort_order`
- `created_at`
- `updated_at`
Notes:
- `ref_type` + `ref_id` ties nodes to existing business objects.
- `config_json` stores node-local UI and action parameters.
- `meta_json` stores display and temporary state that should still persist.
#### `project_workbench_edges`
Purpose:
- Stores links between nodes.
Suggested fields:
- `id`
- `workbench_id`
- `project_id`
- `tenant_id`
- `source_node_id`
- `target_node_id`
- `edge_type`
- `label`
- `config_json`
- `created_at`
- `updated_at`
#### `project_workbench_snapshots`
Purpose:
- Save restore points for the workbench.
Suggested fields:
- `id`
- `workbench_id`
- `project_id`
- `version`
- `snapshot_json`
- `created_by`
- `created_at`
#### `project_workbench_actions`
Purpose:
- Records explicit canvas-triggered actions.
Suggested fields:
- `id`
- `project_id`
- `workbench_id`
- `node_id`
- `action_type`
- `status`
- `request_json`
- `result_json`
- `error_message`
- `created_at`
- `updated_at`
### Relationship Model
Canvas nodes should not duplicate full business records.
Recommended mapping examples:
- `character` node -> `ref_type = character`, `ref_id = characters.id`
- `scene` node -> `ref_type = scene`, `ref_id = scenes.id`
- `storyboard` node -> `ref_type = storyboard`, `ref_id = storyboards.id`
- `video_task` node -> `ref_type = video_task`, `ref_id = ai_tasks.id`
- `assembly` node -> `ref_type = assembly_task`, `ref_id = assembly_tasks.id`
This keeps business truth and canvas orchestration cleanly separated.
## Canvas Node System
### Core Node Types
Recommended first-wave node types:
- `project`
- `script`
- `episode_group`
- `episode`
- `character_group`
- `character`
- `scene_group`
- `scene`
- `storyboard_group`
- `storyboard`
- `video_task`
- `assembly`
- `control`
- `note`
- `group`
### Edge Types
Recommended first-wave edge semantics:
- `contains`
- `depends_on`
- `references`
- `produces`
- `controls`
- `related_to`
The UI may render all of these visually, but v1 should restrict which edge types users can create manually.
### Node Status Model
Recommended shared status language:
- `idle`
- `ready`
- `running`
- `success`
- `failed`
- `blocked`
- `missing_dependency`
- `attention`
Statuses should be derived from underlying business data when possible, and only stored on the node if there is real workbench-only meaning.
## Product Interaction Design
### Default Entry Flow
For `canvas` projects:
- user creates project in `NewProject`
- selects `画布模式`
- backend creates project with `projectMode = canvas`
- frontend redirects to `/project/:projectId/workbench`
- first open triggers workbench bootstrap if it does not exist yet
For `normal` projects:
- unchanged current redirect and page flow
### Workbench Layout
Recommended page structure:
- top: project-level command bar
- left: node library, filters, views
- center: main canvas
- right: inspector and node details
- bottom: action queue, logs, task history
### Primary Workbench Capabilities
#### Visualization
- show project production chain in one place
- show current state of episodes, characters, scenes, storyboards, video tasks, and assembly
- expose dependencies and groupings visually
#### Control
- extract characters
- extract scenes
- generate storyboards
- generate video
- retry failed actions
- batch-run selected nodes or groups
#### Editing and Orchestration
- drag nodes
- box select
- multi-select
- group nodes
- create note nodes
- lock and unlock nodes
- collapse and expand groups
- create selected link types
- save custom layout
- restore snapshots
## Bootstrap Strategy
Canvas projects should not start from an empty board.
Recommended behavior:
- first open runs `bootstrap`
- backend reads project state and creates a default graph
- graph is organized into production lanes
Suggested default lanes from left to right:
- project/script
- episodes
- characters/scenes
- storyboards
- video tasks
- assembly
Benefits:
- immediate value on first entry
- no blank-canvas confusion
- existing project content becomes instantly visible and controllable
## Route and Navigation Design
### New Route
Add:
- `/project/:projectId/workbench`
### Existing Routes
Keep existing routes untouched, including:
- `/project/:projectId`
- `/project/:projectId/outline`
- `/project/:projectId/episodes`
- `/project/:projectId/characters`
- `/project/:projectId/scenes`
- `/project/:projectId/props`
- `/project/:projectId/storyboard/:episodeId?`
- `/project/:projectId/video`
- `/project/:projectId/agent`
### Navigation Rules
For `canvas` projects, recommended project tabs in `Layout.tsx`:
- `工作台`
- `资产` or `设定`
- `视频`
- `Agent`
- `设置`
For `normal` projects:
- keep current tab structure
Optional behavior:
- `ProjectDetail` can remain available for canvas projects, but should not be the primary entry page
- alternatively, `ProjectDetail` for canvas projects can become a summary card page with a prominent workbench entry
## API Design
Introduce a new workbench API surface.
### Workbench Read APIs
- `GET /projects/{projectId}/workbench`
- returns full workbench model for rendering
- `POST /projects/{projectId}/workbench/bootstrap`
- creates default workbench from current project data if absent or on explicit reset
- `GET /projects/{projectId}/workbench/actions`
- lists recent workbench actions
- `GET /projects/{projectId}/workbench/snapshots`
- lists snapshots
### Workbench Write APIs
- `PUT /projects/{projectId}/workbench/viewport`
- `PUT /projects/{projectId}/workbench/layout`
- `POST /projects/{projectId}/workbench/nodes`
- `PATCH /projects/{projectId}/workbench/nodes/{nodeId}`
- `DELETE /projects/{projectId}/workbench/nodes/{nodeId}`
- `POST /projects/{projectId}/workbench/edges`
- `DELETE /projects/{projectId}/workbench/edges/{edgeId}`
- `POST /projects/{projectId}/workbench/snapshots`
- `POST /projects/{projectId}/workbench/snapshots/{snapshotId}/restore`
### Workbench Action API
- `POST /projects/{projectId}/workbench/actions`
Suggested request payload:
- `actionType`
- `scope`
- `nodeIds`
- `refType`
- `refIds`
- `params`
Suggested `actionType` values:
- `extract_characters`
- `extract_scenes`
- `generate_storyboards`
- `generate_video`
- `retry_video_task`
- `regenerate_prompt`
- `auto_layout`
- `sync_from_project`
### Reuse of Existing APIs
The frontend workbench should still use or indirectly reuse the existing APIs in `ai.ts` and `projects.ts` where sensible.
The new backend workbench layer should orchestrate those existing domain operations instead of duplicating them.
## Frontend Module Design
Do not migrate Tapnow as one monolithic file.
Recommended new frontend files under `doc/html/src`:
- `app/pages/ProjectWorkbench.tsx`
- `app/components/workbench/WorkbenchShell.tsx`
- `app/components/workbench/WorkbenchCanvas.tsx`
- `app/components/workbench/WorkbenchToolbar.tsx`
- `app/components/workbench/WorkbenchInspector.tsx`
- `app/components/workbench/WorkbenchBottomPanel.tsx`
- `app/components/workbench/WorkbenchNode.tsx`
- `app/components/workbench/nodes/*`
- `hooks/useWorkbench.ts`
- `hooks/useWorkbenchActions.ts`
- `stores/workbenchStore.ts`
- `lib/api/workbench.ts`
- `lib/workbench/nodeTypes.ts`
- `lib/workbench/layout.ts`
- `lib/workbench/mappers.ts`
### State Management Split
Recommended state split:
React Query:
- project data
- workbench fetches
- actions
- polling
- invalidation
Zustand:
- current selection
- hover state
- drag state
- temporary viewport state
- panel open/close state
- in-memory interaction state
Avoid placing full server truth inside Zustand.
## Backend Module Design
Recommended new backend package areas:
- `yaoai-api/.../controller/ProjectWorkbenchController.java`
- `yaoai-api/.../dto/workbench/*`
- `yaoai-api/.../service/ProjectWorkbenchFacade.java`
- `yaoai-api/.../service/impl/ProjectWorkbenchFacadeImpl.java`
- `yaoai-pipeline/.../service/WorkbenchActionService.java`
- `yaoai-domain/.../entity/ProjectWorkbench*.java`
- `yaoai-domain/.../mapper/ProjectWorkbench*.java`
- new Flyway migration for workbench tables and project mode
Service split recommendation:
- `WorkbenchBootstrapService`
- `WorkbenchLayoutService`
- `WorkbenchActionService`
- `WorkbenchSnapshotService`
- `WorkbenchProjectionService`
## Mapping Strategy from Existing Project Data
The workbench is a projection over current project content.
Recommended projection rules:
- project -> one root node
- uploaded script / outline -> one script node
- episodes -> episode nodes grouped under an episode lane
- characters -> character nodes
- scenes -> scene nodes
- storyboards -> storyboard nodes grouped by episode
- video tasks -> video task nodes attached to storyboard or episode nodes
- assembly task -> one assembly node per episode if available
Where relationships are not fully explicit in current data, derive minimal useful links rather than blocking the feature.
## Editing Semantics
There are two kinds of edits.
### Business Object Edits
When the user edits:
- character fields
- scene fields
- storyboard prompt-related fields
- video task launch parameters
The workbench should call the existing formal backend APIs and update the actual business record.
### Canvas-Only Edits
When the user edits:
- x/y position
- node size
- group membership
- manual edge creation
- collapse state
- node note text
- visual category tags
The workbench should only persist into workbench tables.
## Batch Operations
Batch operations are central to the value of the canvas mode.
Recommended batch actions in v1:
- extract all characters for a project
- extract all scenes for a project
- generate storyboards for an episode or selected episodes
- generate videos for selected storyboards
- retry failed video tasks for selected nodes
- auto-layout selected node clusters
Batch actions should show:
- queued
- running
- success count
- failure count
- per-item error details
## Error Handling
Recommended behavior:
- layout save failures do not corrupt business data
- action failures appear at node level and in the bottom log panel
- missing referenced objects become `orphaned` or `missing_dependency` nodes instead of crashing the page
- bootstrap is idempotent where possible
- snapshot restore restores canvas orchestration state, not necessarily every domain object mutation
- destructive actions require confirmation
## Performance Considerations
Key risks:
- very large projects with many storyboard nodes
- frequent node movement causing excessive writes
- many concurrent task polls
Recommended mitigations:
- debounce layout persistence
- virtualize long side panels and lists
- split workbench fetches if graph becomes very large
- poll task status at sensible intervals rather than per-node aggressive polling
- support lazy expansion of episode groups
## Migration Strategy from Tapnow
Recommended rule:
- migrate interaction concepts, not code shape
Carry over:
- canvas-centric mental model
- node-based orchestration
- preview and generation linkage
- batch execution UX ideas
- visual status and queue thinking
Do not carry over directly:
- single giant `App.jsx`
- localStorage-heavy persistence as the primary source
- provider configuration UI embedded into the canvas runtime
- tight coupling between canvas rendering and request-template management
## Security and Multi-Tenant Considerations
Because `yaoaivideo` is tenant-aware, all new workbench records must include tenant scoping.
Requirements:
- all workbench reads and writes validate project ownership within tenant scope
- node refs must only target records inside the same project and tenant
- snapshot restore must not allow cross-project contamination
- action execution must honor the same auth and billing rules as existing flows
## Billing and Usage Considerations
Canvas-triggered actions should still flow through existing billing logic.
Requirements:
- extraction and generation triggered from the workbench count the same as when triggered from existing pages
- action history should record enough metadata for cost attribution
- workbench mode should not bypass quotas or watermark settings
## Rollout Plan
### Phase 1: Project Mode and Host
- add `projectMode` to project data model
- update project create flow UI and DTOs
- add workbench route
- update layout tab logic for canvas projects
### Phase 2: Workbench Persistence and Bootstrap
- add workbench tables
- add bootstrap service and APIs
- render read-only projected graph
- save viewport and layout
### Phase 3: Workbench Controls
- support canvas-triggered extraction and generation
- support node inspector editing for selected domain objects
- show per-node and bottom-panel action status
### Phase 4: Editing and Orchestration Enhancements
- add groups
- add edge editing
- add snapshots
- add advanced batch operations
- add auto-layout variants
## Testing Strategy
### Backend Tests
- migration tests for project mode and workbench tables
- bootstrap graph generation
- workbench node and edge CRUD
- action dispatch tests
- tenant isolation tests
### Frontend Tests
- project create mode selection
- route branching by project mode
- workbench initial render
- node selection and inspector behavior
- layout persistence debounce behavior
- batch action feedback
### E2E Tests
- create canvas project
- open workbench
- bootstrap graph appears
- trigger character extraction
- trigger scene extraction
- trigger storyboard generation
- trigger video generation
- save layout and reload
- restore a snapshot
## Risks
### Main Risks
- underestimating the effort of building a good canvas host inside the current frontend architecture
- letting workbench state and business data drift apart
- trying to make every edge fully editable too early
- rebuilding too much of Tapnow instead of integrating with `yaoaivideo`
- overloading v1 with too many node types and actions
### Mitigations
- keep mode separation strict
- keep business truth outside the workbench tables
- start with a constrained node and edge model
- ship bootstrap plus practical actions first
- preserve existing pages as fallback and support surfaces for canvas projects
## Open Decisions Already Resolved
These decisions are considered fixed for this design:
- use project mode split instead of replacing the existing flow
- create a new workbench route rather than condition-heavy retrofitting into existing pages
- keep mode immutable after project creation
- keep canvas as orchestration and projection layer, not sole truth layer
- avoid direct code transplant of Tapnow monolith
## Final Recommendation
Proceed with:
- `projectMode` on project creation
- dedicated `/project/:projectId/workbench` route
- independent workbench persistence tables
- existing domain object reuse for formal data truth
- modular frontend implementation in `doc/html`
- phased rollout with bootstrap, visualization, control, then advanced orchestration
This is the lowest-risk path that still preserves the product ambition of a project-level visual workbench with control and editing capabilities.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment