Evilom
ab2ca1d836
Reusable 6-step pipeline for collecting DingTalk group messages, extracting Feishu doc links, fetching content, and generating summaries. Skills structure: - SKILL.md: trigger rules, workflow, config reference - config.yaml: group IDs, collection settings - scripts/paths.py: shared path resolution - scripts/step1-6: modular pipeline steps - scripts/run_all.py: one-click runner
181 lines
5.1 KiB
Markdown
181 lines
5.1 KiB
Markdown
# 钉钉群聊飞书文档采集与知识整理 SOP
|
||
|
||
## 触发条件
|
||
|
||
当用户提到以下关键词时使用此技能：
|
||
- "收集钉钉消息"、"拉取群聊"、"采集飞书文档"
|
||
- "整理群里的链接"、"汇总飞书报告"
|
||
- "每日收集"、"定时采集"
|
||
- 涉及钉钉群聊 + 飞书文档的工作流
|
||
|
||
## 一句话概述
|
||
|
||
从钉钉群聊中自动采集飞书文档链接和文件附件，下载内容，生成结构化总结。
|
||
|
||
## 前置条件
|
||
|
||
| 工具 | 用途 | 获取方式 |
|
||
|------|------|----------|
|
||
| dws CLI | 钉钉消息采集 + 文件下载 | 悟空(Wukong)自带，或从 GitHub 下载 |
|
||
| lark-cli | 飞书文档内容读取 | `npm install -g @larksuite/cli` |
|
||
| Python 3.10+ | 数据处理 | 系统自带或悟空内置 |
|
||
|
||
**认证要求**：
|
||
- dws CLI：运行 `dws auth status` 确认已登录（钉钉扫码）
|
||
- lark-cli：运行 `lark-cli auth status` 确认已授权（飞书浏览器OAuth）
|
||
|
||
## 完整工作流（6步）
|
||
|
||
### Step 1: 拉取钉钉群消息
|
||
|
||
```bash
|
||
python scripts/step1_collect_messages.py # 全量
|
||
python scripts/step1_collect_messages.py --days 1 # 增量
|
||
```
|
||
|
||
**输出**：`data/raw-messages/all_messages_combined.json`
|
||
|
||
### Step 2: 提取链接与附件
|
||
|
||
```bash
|
||
python scripts/step2_extract_links.py # 全量
|
||
python scripts/step2_extract_links.py --incremental # 增量
|
||
```
|
||
|
||
正则提取飞书链接、文件附件、Kimi链接。
|
||
|
||
**输出**：`data/links/all_feishu_links.json` + `data/links/all_file_attachments.json`
|
||
|
||
### Step 3: 拉取飞书文档内容
|
||
|
||
```bash
|
||
python scripts/step3_fetch_feishu_docs.py # 全量
|
||
python scripts/step3_fetch_feishu_docs.py --incremental # 增量
|
||
```
|
||
|
||
wiki/docx 自动识别，Block API 解析为 Markdown。
|
||
|
||
**输出**：`output/feishu-docs/<doc_id>.md` + `data/links/all_feishu_content.json`
|
||
|
||
### Step 4: 下载文件附件
|
||
|
||
```bash
|
||
python scripts/step4_download_files.py # 全量
|
||
python scripts/step4_download_files.py --incremental # 增量
|
||
```
|
||
|
||
使用 `dws drive download` 下载 HTML/MD/XLSX/PPTX/PDF 等附件。
|
||
|
||
**输出**：`output/downloaded-files/html-md/` 和 `output/downloaded-files/other/`
|
||
|
||
### Step 5: 生成结构化总结
|
||
|
||
```bash
|
||
python scripts/step5_generate_summary.py # 全量
|
||
python scripts/step5_generate_summary.py --since today # 今日
|
||
python scripts/step5_generate_summary.py --brief # 简要
|
||
```
|
||
|
||
按群组/人物/主题聚类，生成增量报告。
|
||
|
||
**输出**：`output/reports/latest_summary.md`
|
||
|
||
### Step 6: 更新知识库（可选）
|
||
|
||
```bash
|
||
python scripts/step6_update_knowledge_base.py
|
||
```
|
||
|
||
同步到 Obsidian 知识库 + 知识图谱。
|
||
|
||
## 配置文件
|
||
|
||
所有可定制项在 `config.yaml`：
|
||
|
||
```yaml
|
||
groups:
|
||
dc战略问题研究院: "cidoUneRB4Db8TAXaTrKxkQAw=="
|
||
创新组: "cidMuM+itt5PeY7xNSWsv3M0g=="
|
||
|
||
collection:
|
||
start_date: "2026-05-01 00:00:00"
|
||
days_back: 3
|
||
limit: 200
|
||
|
||
dws_path: auto # auto | /path/to/dws.exe
|
||
|
||
output_dir: "./output"
|
||
data_dir: "./data"
|
||
```
|
||
|
||
## Agent 调用约定
|
||
|
||
### 增量模式（日常）
|
||
```bash
|
||
python scripts/step1_collect_messages.py --days 1
|
||
python scripts/step2_extract_links.py --incremental
|
||
python scripts/step3_fetch_feishu_docs.py --incremental
|
||
python scripts/step4_download_files.py --incremental
|
||
python scripts/step5_generate_summary.py --since today
|
||
```
|
||
|
||
### 全量模式（首次/重建）
|
||
```bash
|
||
python scripts/step1_collect_messages.py --full
|
||
python scripts/step2_extract_links.py
|
||
python scripts/step3_fetch_feishu_docs.py
|
||
python scripts/step4_download_files.py
|
||
python scripts/step5_generate_summary.py --full
|
||
```
|
||
|
||
### 快速模式（只看今天）
|
||
```bash
|
||
python scripts/step1_collect_messages.py --days 1
|
||
python scripts/step2_extract_links.py --incremental
|
||
python scripts/step5_generate_summary.py --since today --brief
|
||
```
|
||
|
||
## 输出产物
|
||
|
||
| 目录 | 内容 | 格式 |
|
||
|------|------|------|
|
||
| `data/raw-messages/` | 钉钉原始消息 | JSON |
|
||
| `data/links/` | 链接索引 | JSON |
|
||
| `output/feishu-docs/` | 飞书文档 | .md |
|
||
| `output/downloaded-files/` | 文件附件 | 原始格式 |
|
||
| `output/reports/` | 汇总报告 | .md |
|
||
| `output/obsidian-vault/` | Obsidian 知识库 | .md |
|
||
| `output/knowledge-graph/` | 知识图谱 | JSON + HTML |
|
||
|
||
## 常见问题
|
||
|
||
**dws 未登录**：`dws auth login` 扫码。悟空内置路径 `C:\Users\<user>\.real\.bin\dws\bin\dws.exe`
|
||
|
||
**lark-cli 过期**：`lark-cli auth login --domain docs,drive,wiki --recommend`
|
||
|
||
**wiki 链接解析**：脚本自动处理 wiki→docx 的节点ID解析
|
||
|
||
**消息分页**：脚本内置 openMessageId 去重 + 分段拉取
|
||
|
||
## 定时任务
|
||
|
||
```powershell
|
||
# Windows 每天 18:00 增量采集
|
||
schtasks /create /tn "DingTalkFeishuCollector" /tr "python <project>\scripts\step1_collect_messages.py --days 1" /sc daily /st 18:00
|
||
```
|
||
|
||
## 技术栈
|
||
|
||
```
|
||
钉钉群聊 → dws CLI (Go) → JSON消息
|
||
↓
|
||
Python 正则提取 → 链接索引
|
||
↓
|
||
lark-cli (Node) → 飞书文档内容
|
||
↓
|
||
Python 处理 → Markdown总结 + Obsidian库 + 知识图谱
|
||
```
|
||
|
||
**总依赖**：dws CLI (14MB) + lark-cli (npm) + Python 3.10+ + beautifulsoup4
|
||
**总成本**：0 元（需要飞书账号 + 钉钉群访问权限）
|