Compare commits
13 Commits
867bf67b16
...
master
Author | SHA1 | Date | |
---|---|---|---|
fe2803f3da | |||
9ed05dd58d | |||
211bcc9066 | |||
8c3f3df826 | |||
8d7c4e3730 | |||
cdf7e36ba3 | |||
fbca92ba77 | |||
0b1dc6b8ca | |||
eabca97350 | |||
01e323a7ba | |||
d42cefd9ca | |||
0ac375eb50 | |||
8097e9b769 |
132
.dockerignore
Normal file
132
.dockerignore
Normal file
@@ -0,0 +1,132 @@
|
||||
# BigDataTool Docker 构建忽略文件
|
||||
# 排除不需要打包到镜像中的文件和目录
|
||||
|
||||
# 版本控制
|
||||
.git
|
||||
.gitignore
|
||||
.gitattributes
|
||||
|
||||
# IDE和编辑器文件
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Python缓存文件
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# 虚拟环境
|
||||
venv/
|
||||
env/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# 测试文件
|
||||
.tox/
|
||||
.coverage
|
||||
.pytest_cache/
|
||||
htmlcov/
|
||||
.coverage.*
|
||||
coverage.xml
|
||||
*.cover
|
||||
.hypothesis/
|
||||
|
||||
# 文档构建
|
||||
docs/_build/
|
||||
.sphinx/
|
||||
|
||||
# 日志文件
|
||||
*.log
|
||||
logs/*.log
|
||||
|
||||
# 本地数据库文件(可选,如果需要持久化可以移除这行)
|
||||
*.db
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
|
||||
# 配置文件(敏感信息)
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
*.pem
|
||||
*.key
|
||||
config/secrets/
|
||||
|
||||
# Docker相关文件
|
||||
Dockerfile.dev
|
||||
docker-compose.dev.yml
|
||||
docker-compose.override.yml
|
||||
|
||||
# 临时文件
|
||||
*.tmp
|
||||
*.temp
|
||||
tmp/
|
||||
temp/
|
||||
|
||||
# macOS
|
||||
.DS_Store
|
||||
.DS_Store?
|
||||
._*
|
||||
.Spotlight-V100
|
||||
.Trashes
|
||||
ehthumbs.db
|
||||
Thumbs.db
|
||||
|
||||
# Windows
|
||||
*.exe
|
||||
*.msi
|
||||
*.msm
|
||||
*.msp
|
||||
|
||||
# Linux
|
||||
*~
|
||||
|
||||
# 备份文件
|
||||
*.bak
|
||||
*.backup
|
||||
*.old
|
||||
|
||||
# 本地开发文件
|
||||
local/
|
||||
.local/
|
||||
dev/
|
||||
|
||||
# 缓存目录
|
||||
.cache/
|
||||
*.cache
|
||||
|
||||
# 运行时文件
|
||||
*.pid
|
||||
*.sock
|
||||
|
||||
# 监控和性能分析
|
||||
*.prof
|
||||
*.pstats
|
||||
|
||||
# 其他不需要的文件
|
||||
README.dev.md
|
||||
CONTRIBUTING.md
|
||||
.github/
|
||||
scripts/dev/
|
394
CLAUDE.md
394
CLAUDE.md
@@ -1,394 +0,0 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## 项目架构
|
||||
|
||||
这是一个基于Flask的现代化数据库查询比对工具,支持Cassandra和Redis两大数据源的数据一致性验证。采用模块化架构设计,支持单表查询、TWCS分表查询、多主键查询和Redis集群比对等多种复杂场景。
|
||||
|
||||
### 核心组件架构
|
||||
|
||||
**主应用 (app.py)**
|
||||
- 应用入口和全局配置管理
|
||||
- 模块导入和路由设置
|
||||
- 日志系统初始化
|
||||
|
||||
**模块化后端 (modules/)**
|
||||
- `api_routes.py`: 所有Flask路由和请求处理逻辑
|
||||
- Cassandra查询API:`/api/query`, `/api/sharding-query`
|
||||
- Redis比对API:`/api/redis/compare`
|
||||
- 配置管理API:配置组CRUD操作
|
||||
- 查询历史API:历史记录的保存和回放
|
||||
- `database.py`: SQLite数据库管理
|
||||
- 数据库初始化和表结构创建
|
||||
- 版本控制和字段动态添加
|
||||
- 事务处理和连接管理
|
||||
- `query_engine.py`: Cassandra查询引擎
|
||||
- 单表查询和分表查询执行
|
||||
- 多主键查询支持(复合主键SQL构建)
|
||||
- 并行查询和性能优化
|
||||
- `redis_query.py`: Redis查询引擎
|
||||
- 全Redis数据类型支持(String/Hash/List/Set/ZSet/Stream)
|
||||
- 随机采样和指定Key两种查询模式
|
||||
- 集群模式的自动检测和连接
|
||||
- `data_comparison.py`: 数据比对引擎
|
||||
- JSON和数组的智能深度比较
|
||||
- 复合主键的精确匹配算法
|
||||
- 数据质量评估和建议生成
|
||||
- `cassandra_client.py` / `redis_client.py`: 数据库客户端
|
||||
- 连接管理和错误处理
|
||||
- 性能监控和连接池优化
|
||||
- `config_groups.db`: SQLite数据库,存储配置组、查询历史和日志
|
||||
|
||||
**前端 (原生JavaScript + Bootstrap)**
|
||||
- `templates/index.html`: 工具集合首页,提供功能导航
|
||||
- `templates/db_compare.html`: Cassandra比对界面
|
||||
- 支持单表、分表和多主键三种查询模式
|
||||
- 分表模式切换和参数配置
|
||||
- 实时查询日志和性能监控
|
||||
- `templates/redis_compare.html`: Redis比对界面
|
||||
- 集群配置和连接管理
|
||||
- 随机采样和指定Key两种查询模式
|
||||
- 全数据类型支持的结果展示
|
||||
- `static/js/app.js`: Cassandra查询的前端逻辑
|
||||
- 配置管理和表单处理
|
||||
- 分页展示和数据可视化
|
||||
- 多主键查询的UI适配
|
||||
- `static/js/redis_compare.js`: Redis比对的前端逻辑
|
||||
- Redis集群配置管理
|
||||
- 查询模式切换和参数设置
|
||||
- 多类型数据的格式化展示
|
||||
|
||||
### 关键功能模块
|
||||
|
||||
**Cassandra数据比对引擎**
|
||||
- 支持复杂JSON字段的深度比较
|
||||
- 数组字段的顺序无关比较
|
||||
- 字段级别的差异统计和分析
|
||||
- 数据质量评估和建议生成
|
||||
- 支持包含和排除特定字段的比较
|
||||
- 多主键数据比对:支持复合主键的精确匹配和差异检测
|
||||
|
||||
**Redis数据比对引擎**
|
||||
- 全数据类型支持:String、Hash、List、Set、ZSet、Stream
|
||||
- 智能JSON检测和深度比较
|
||||
- 集群和单节点模式的自动适配
|
||||
- 随机采样和指定Key两种查询模式
|
||||
- 性能监控和连接时间统计
|
||||
|
||||
**分表查询功能模块**
|
||||
- **时间戳提取算法**:使用 `re.sub(r'\D', '', key)` 删除Key中所有非数字字符
|
||||
- **分表索引计算**:公式 `int(numbers) // interval_seconds % table_count`
|
||||
- **混合查询场景**:支持生产分表+测试单表等组合
|
||||
- **并行查询**:多分表同时查询以提高性能
|
||||
|
||||
**多主键查询功能模块**
|
||||
- **复合主键格式**:主键字段逗号分隔,查询值逗号分隔
|
||||
- **SQL构建逻辑**:自动选择IN查询或OR条件组合
|
||||
- **数据匹配算法**:统一处理单主键和复合主键匹配
|
||||
- **向后兼容**:完全兼容现有单主键查询
|
||||
|
||||
**用户界面特性**
|
||||
- 分页系统(差异记录和相同记录)
|
||||
- 实时搜索和过滤
|
||||
- 原生数据展示(JSON语法高亮、树形视图)
|
||||
- 配置导入导出和管理
|
||||
- 详细的错误诊断和故障排查指南
|
||||
- 查询历史记录和复用
|
||||
- 查询日志系统:实时显示SQL执行日志,支持日志级别过滤
|
||||
|
||||
## 开发相关命令
|
||||
|
||||
### 环境设置
|
||||
```bash
|
||||
# 安装依赖
|
||||
pip install -r requirements.txt
|
||||
|
||||
# 运行应用(默认端口5000)
|
||||
python app.py
|
||||
|
||||
# 开发模式启动(支持热重载)
|
||||
# app.py中默认开启debug=True
|
||||
|
||||
# 生产环境运行
|
||||
# 使用WSGI服务器如gunicorn
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 app:app
|
||||
```
|
||||
|
||||
### 测试和验证
|
||||
```bash
|
||||
# 主要通过Web界面进行功能测试
|
||||
# Cassandra单表查询:http://localhost:5000/db-compare
|
||||
# Redis数据比对:http://localhost:5000/redis-compare
|
||||
# 工具集合首页:http://localhost:5000
|
||||
|
||||
# Cassandra多主键查询测试示例:
|
||||
# 1. 在主键字段中输入:docid,id
|
||||
# 2. 在查询Key值中输入(每行一组):
|
||||
# 8825C293B3609175B2224236E984FEDB,8825C293B3609175B2224236E984FED
|
||||
# 9925C293B3609175B2224236E984FEDB,9925C293B3609175B2224236E984FED
|
||||
|
||||
# Redis查询测试示例:
|
||||
# 1. 配置两个Redis集群连接
|
||||
# 2. 选择随机采样模式,设置采样数量
|
||||
# 3. 或选择指定Key模式,输入要比对的Key列表
|
||||
|
||||
# 数据库初始化(如果config_groups.db不存在)
|
||||
# 通过访问Web界面会自动创建数据库表结构
|
||||
```
|
||||
|
||||
### 依赖项
|
||||
- Flask==2.3.3
|
||||
- cassandra-driver==3.29.1
|
||||
- redis==5.0.1
|
||||
|
||||
### 项目特点
|
||||
- **模块化架构**:清晰的代码组织和职责分离
|
||||
- **双数据源支持**:同时支持Cassandra和Redis数据比对
|
||||
- **智能查询引擎**:针对不同数据源的优化查询策略
|
||||
- **SQLite本地存储**:配置组、查询历史和日志的本地持久化
|
||||
- **前端原生实现**:使用原生JavaScript + Bootstrap,无现代前端框架依赖
|
||||
- **多模式支持**:单表查询、分表查询、多主键查询、Redis比对的统一架构
|
||||
|
||||
## API架构说明
|
||||
|
||||
### Cassandra相关API端点
|
||||
- `GET /api/default-config`: 获取默认Cassandra配置
|
||||
- `POST /api/query`: 执行单表数据库查询比对(支持多主键查询)
|
||||
- `POST /api/sharding-query`: 执行分表查询比对(支持多主键查询)
|
||||
- `GET /api/config-groups`: 获取所有Cassandra配置组
|
||||
- `POST /api/config-groups`: 创建新Cassandra配置组
|
||||
- `GET /api/config-groups/<id>`: 获取特定Cassandra配置组
|
||||
- `DELETE /api/config-groups/<id>`: 删除Cassandra配置组
|
||||
|
||||
### Redis相关API端点
|
||||
- `POST /api/redis/compare`: 执行Redis数据比对
|
||||
- `POST /api/redis/test-connection`: 测试Redis连接
|
||||
- `GET /api/redis/config-groups`: 获取所有Redis配置组
|
||||
- `POST /api/redis/config-groups`: 创建新Redis配置组
|
||||
- `GET /api/redis/config-groups/<id>`: 获取特定Redis配置组
|
||||
- `DELETE /api/redis/config-groups/<id>`: 删除Redis配置组
|
||||
- `GET /api/redis/query-history`: 获取Redis查询历史
|
||||
- `POST /api/redis/query-history`: 保存Redis查询历史
|
||||
- `GET /api/redis/query-history/<id>`: 获取特定Redis历史记录
|
||||
- `DELETE /api/redis/query-history/<id>`: 删除Redis历史记录
|
||||
|
||||
### 通用API端点
|
||||
- `POST /api/init-db`: 初始化SQLite数据库
|
||||
- `GET /api/query-history`: 获取Cassandra查询历史
|
||||
- `POST /api/query-history`: 保存Cassandra查询历史
|
||||
- `GET /api/query-history/<id>`: 获取特定Cassandra历史记录
|
||||
- `GET /api/query-history/<id>/results`: 获取历史记录的完整结果数据
|
||||
- `DELETE /api/query-history/<id>`: 删除Cassandra历史记录
|
||||
- `GET /api/query-logs`: 获取查询日志(支持limit参数)
|
||||
- `GET /api/query-logs/history/<id>`: 获取特定历史记录的相关日志
|
||||
- `DELETE /api/query-logs`: 清空查询日志
|
||||
|
||||
### 查询比对流程
|
||||
|
||||
**Cassandra单表查询流程(`/api/query`)**:
|
||||
1. 前端发送配置和Key值列表到 `/api/query`
|
||||
2. 后端通过 `cassandra_client.py` 创建两个Cassandra连接(生产+测试)
|
||||
3. `query_engine.py` 并行执行查询,获取原始数据
|
||||
4. `data_comparison.py` 运行比较算法,生成差异报告
|
||||
5. 返回完整结果(差异、统计、原始数据)
|
||||
|
||||
**Cassandra分表查询流程(`/api/sharding-query`)**:
|
||||
1. 前端发送配置、Key值列表和分表配置到 `/api/sharding-query`
|
||||
2. 后端使用 `sharding.py` 中的 `ShardingCalculator` 解析Key中的时间戳
|
||||
3. 根据分表算法计算每个Key对应的分表名称
|
||||
4. `query_engine.py` 创建分表映射关系,并行执行分表查询
|
||||
5. 汇总所有分表结果,执行比较算法
|
||||
6. 返回包含分表信息的完整结果
|
||||
|
||||
**Redis数据比对流程(`/api/redis/compare`)**:
|
||||
1. 前端发送Redis集群配置和查询参数到 `/api/redis/compare`
|
||||
2. 后端通过 `redis_client.py` 创建两个Redis连接
|
||||
3. `redis_query.py` 根据查询模式执行数据获取:
|
||||
- 随机采样模式:从源集群随机获取指定数量的Key
|
||||
- 指定Key模式:查询用户提供的Key列表
|
||||
4. 针对每个Key,查询其在两个集群中的值和数据类型
|
||||
5. 执行智能数据比较(根据数据类型选择比较策略)
|
||||
6. 返回比对结果和统计信息
|
||||
|
||||
## 数据结构和配置
|
||||
|
||||
### Cassandra配置结构
|
||||
|
||||
**单表查询配置**:
|
||||
```javascript
|
||||
{
|
||||
pro_config: {
|
||||
cluster_name, datacenter, hosts[], port,
|
||||
username, password, keyspace, table
|
||||
},
|
||||
test_config: { /* 同上 */ },
|
||||
keys: ["主键字段名"], // 支持多个字段,如 ["docid", "id"]
|
||||
fields_to_compare: ["字段1", "字段2"], // 空数组=全部字段
|
||||
exclude_fields: ["排除字段"],
|
||||
values: ["key1", "key2", "key3"] // 单主键或复合主键值
|
||||
}
|
||||
```
|
||||
|
||||
**分表查询配置**:
|
||||
```javascript
|
||||
{
|
||||
pro_config: { /* 基础配置同上 */ },
|
||||
test_config: { /* 基础配置同上 */ },
|
||||
keys: ["主键字段名"], // 支持复合主键
|
||||
fields_to_compare: ["字段1", "字段2"],
|
||||
exclude_fields: ["排除字段"],
|
||||
values: ["key1", "key2", "key3"], // 支持复合主键值
|
||||
sharding_config: {
|
||||
use_sharding_for_pro: true, // 生产环境是否使用分表
|
||||
use_sharding_for_test: false, // 测试环境是否使用分表
|
||||
interval_seconds: 604800, // 分表时间间隔(默认7天)
|
||||
table_count: 14 // 分表数量(默认14张表)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Redis配置结构
|
||||
|
||||
**Redis集群配置**:
|
||||
```javascript
|
||||
{
|
||||
source_config: {
|
||||
name: "源集群名称",
|
||||
nodes: [
|
||||
{host: "192.168.1.200", port: 7000},
|
||||
{host: "192.168.1.201", port: 7001}
|
||||
],
|
||||
password: "redis_password",
|
||||
socket_timeout: 3,
|
||||
socket_connect_timeout: 3,
|
||||
max_connections_per_node: 16
|
||||
},
|
||||
target_config: { /* 同上 */ },
|
||||
query_config: {
|
||||
mode: "random_sample", // 或 "specific_keys"
|
||||
sample_size: 1000, // 随机采样数量(random_sample模式)
|
||||
keys: ["key1", "key2"], // 指定Key列表(specific_keys模式)
|
||||
key_pattern: "*" // Key匹配模式(可选)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**多主键查询格式示例**:
|
||||
```javascript
|
||||
// 复合主键配置
|
||||
keys: ["docid", "id"]
|
||||
|
||||
// 复合主键查询值(逗号分隔)
|
||||
values: [
|
||||
"8825C293B3609175B2224236E984FEDB,8825C293B3609175B2224236E984FED",
|
||||
"9925C293B3609175B2224236E984FEDB,9925C293B3609175B2224236E984FED"
|
||||
]
|
||||
```
|
||||
|
||||
### 查询结果结构
|
||||
|
||||
**Cassandra查询结果**:
|
||||
```javascript
|
||||
{
|
||||
total_keys, pro_count, test_count,
|
||||
differences: [{
|
||||
key: {docid: "val1", id: "val2"}, // 支持复合主键对象
|
||||
field, pro_value, test_value, message
|
||||
}],
|
||||
identical_results: [{
|
||||
key: {docid: "val1", id: "val2"}, // 支持复合主键对象
|
||||
pro_fields, test_fields
|
||||
}],
|
||||
field_diff_count: { "field_name": count },
|
||||
raw_pro_data: [], raw_test_data: [],
|
||||
summary: { overview, percentages, field_analysis, recommendations },
|
||||
|
||||
// 分表查询特有字段
|
||||
sharding_info: {
|
||||
pro_shard_mapping: { "key1": "table_name_0", "key2": "table_name_1" },
|
||||
test_shard_mapping: { /* 同上 */ },
|
||||
failed_keys: [], // 时间戳提取失败的Key
|
||||
shard_stats: {
|
||||
pro_tables_used: ["table_0", "table_1"],
|
||||
test_tables_used: ["table_0"],
|
||||
timestamp_extraction_success_rate: 95.5
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Redis查询结果**:
|
||||
```javascript
|
||||
{
|
||||
total_keys, source_count, target_count,
|
||||
identical_count, different_count, source_only_count, target_only_count,
|
||||
|
||||
// 详细比对结果
|
||||
identical_keys: ["key1", "key2"],
|
||||
different_keys: [
|
||||
{
|
||||
key: "key3",
|
||||
source_type: "string", target_type: "string",
|
||||
source_value: "value1", target_value: "value2",
|
||||
message: "Value mismatch"
|
||||
}
|
||||
],
|
||||
source_only_keys: ["key4"], // 仅源集群存在
|
||||
target_only_keys: ["key5"], // 仅目标集群存在
|
||||
|
||||
// 统计信息
|
||||
type_distribution: {
|
||||
"string": 500, "hash": 200, "list": 100,
|
||||
"set": 50, "zset": 30, "stream": 20
|
||||
},
|
||||
consistency_percentage: 85.5,
|
||||
|
||||
// 性能统计
|
||||
query_time: 2.5,
|
||||
source_connection_time: 0.1,
|
||||
target_connection_time: 0.15
|
||||
}
|
||||
```
|
||||
|
||||
## 开发注意事项
|
||||
|
||||
### 代码修改指导
|
||||
- **模块化开发**:功能按模块组织,修改时注意模块间的依赖关系
|
||||
- **数据库模式变更**:修改SQLite表结构需要考虑向后兼容性
|
||||
- **前端JavaScript**:分别位于 `static/js/app.js`(Cassandra)和 `static/js/redis_compare.js`(Redis)
|
||||
- **HTML模板**:使用Jinja2模板引擎,主要文件在 `templates/` 目录
|
||||
|
||||
### 关键模块和类位置
|
||||
- **主应用**:`app.py` - 应用入口和模块集成
|
||||
- **路由管理**:`modules/api_routes.py` - 所有API端点的实现
|
||||
- **数据库管理**:`modules/database.py` - SQLite数据库操作
|
||||
- **Cassandra客户端**:`modules/cassandra_client.py` - 连接管理和查询执行
|
||||
- **Redis客户端**:`modules/redis_client.py` - Redis连接和性能监控
|
||||
- **查询引擎**:`modules/query_engine.py` - Cassandra查询逻辑
|
||||
- **Redis查询**:`modules/redis_query.py` - Redis数据比对逻辑
|
||||
- **数据比较**:`modules/data_comparison.py` - 智能数据比较算法
|
||||
- **分表计算**:`modules/sharding.py` - TWCS分表逻辑
|
||||
- **配置管理**:`modules/config_manager.py` - 配置组管理
|
||||
- **日志收集**:`modules/query_logger.py` - 查询日志系统
|
||||
|
||||
### 模块依赖关系
|
||||
```
|
||||
app.py
|
||||
├── modules/api_routes.py (路由层)
|
||||
├── modules/config_manager.py (配置管理)
|
||||
├── modules/cassandra_client.py (Cassandra连接)
|
||||
├── modules/redis_client.py (Redis连接)
|
||||
├── modules/query_engine.py (Cassandra查询)
|
||||
│ ├── modules/sharding.py (分表计算)
|
||||
│ └── modules/data_comparison.py (数据比较)
|
||||
├── modules/redis_query.py (Redis查询)
|
||||
└── modules/database.py (SQLite数据库)
|
||||
```
|
||||
|
||||
### 开发最佳实践
|
||||
- **错误处理**:每个模块都有详细的错误分类和处理机制
|
||||
- **日志记录**:使用统一的日志系统,支持不同级别的日志输出
|
||||
- **性能监控**:查询时间和连接时间的详细统计
|
||||
- **配置管理**:支持配置的导入导出和版本管理
|
||||
- **数据安全**:敏感信息(密码)的安全处理
|
71
Dockerfile
Normal file
71
Dockerfile
Normal file
@@ -0,0 +1,71 @@
|
||||
# BigDataTool Docker镜像
|
||||
# 基于Python 3.9 Alpine镜像构建轻量级容器
|
||||
|
||||
# 使用官方Python 3.9 Alpine镜像作为基础镜像
|
||||
FROM python:3.9-alpine
|
||||
|
||||
# 设置维护者信息
|
||||
LABEL maintainer="BigDataTool Team"
|
||||
LABEL version="2.0"
|
||||
LABEL description="BigDataTool - 大数据查询比对工具容器化版本"
|
||||
|
||||
# 设置环境变量
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PIP_NO_CACHE_DIR=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
FLASK_HOST=0.0.0.0 \
|
||||
FLASK_PORT=5000
|
||||
|
||||
# 设置工作目录
|
||||
WORKDIR /app
|
||||
|
||||
# 安装系统依赖
|
||||
# Alpine需要的构建工具和运行时库
|
||||
RUN apk add --no-cache \
|
||||
gcc \
|
||||
musl-dev \
|
||||
libffi-dev \
|
||||
openssl-dev \
|
||||
cargo \
|
||||
rust \
|
||||
&& apk add --no-cache --virtual .build-deps \
|
||||
build-base \
|
||||
python3-dev
|
||||
|
||||
# 复制requirements文件
|
||||
COPY requirements.txt .
|
||||
|
||||
# 安装Python依赖
|
||||
# 使用国内镜像源加速下载
|
||||
RUN pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/
|
||||
|
||||
# 清理构建依赖以减小镜像大小
|
||||
RUN apk del .build-deps
|
||||
|
||||
# 复制应用代码
|
||||
COPY . .
|
||||
|
||||
# 创建必要的目录
|
||||
RUN mkdir -p logs && \
|
||||
chmod +x docker-entrypoint.sh || true
|
||||
|
||||
# 创建非root用户运行应用
|
||||
RUN addgroup -g 1001 -S appgroup && \
|
||||
adduser -u 1001 -S appuser -G appgroup
|
||||
|
||||
# 更改文件所有权
|
||||
RUN chown -R appuser:appgroup /app
|
||||
|
||||
# 切换到非root用户
|
||||
USER appuser
|
||||
|
||||
# 暴露端口
|
||||
EXPOSE 5000
|
||||
|
||||
# 健康检查
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD wget --no-verbose --tries=1 --spider http://localhost:5000/api/health || exit 1
|
||||
|
||||
# 设置启动命令
|
||||
CMD ["python", "app.py"]
|
21
LICENSE
Normal file
21
LICENSE
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 BigDataTool项目组
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
148
Makefile
Normal file
148
Makefile
Normal file
@@ -0,0 +1,148 @@
|
||||
# BigDataTool Docker 管理 Makefile
|
||||
.PHONY: help build run stop clean logs shell health
|
||||
|
||||
# 默认目标
|
||||
help:
|
||||
@echo "BigDataTool Docker 管理命令:"
|
||||
@echo ""
|
||||
@echo " build 构建Docker镜像"
|
||||
@echo " run 启动服务(简化版本)"
|
||||
@echo " run-full 启动完整服务(包含缓存和监控)"
|
||||
@echo " stop 停止服务"
|
||||
@echo " restart 重启服务"
|
||||
@echo " clean 清理容器和镜像"
|
||||
@echo " logs 查看服务日志"
|
||||
@echo " shell 进入容器shell"
|
||||
@echo " health 检查服务健康状态"
|
||||
@echo " ps 查看运行状态"
|
||||
@echo ""
|
||||
@echo "环境变量设置:"
|
||||
@echo " export SECRET_KEY=your-secret-key"
|
||||
@echo " export FLASK_ENV=production"
|
||||
@echo ""
|
||||
|
||||
# 构建镜像
|
||||
build:
|
||||
@echo "构建BigDataTool Docker镜像..."
|
||||
docker build -t bigdatatool:latest .
|
||||
|
||||
# 快速运行(简化版本)
|
||||
run:
|
||||
@echo "启动BigDataTool服务(简化版本)..."
|
||||
docker-compose -f docker-compose.simple.yml up -d
|
||||
@echo "服务启动中,请等待30秒后访问 http://localhost:5000"
|
||||
|
||||
# 完整运行(包含缓存和监控)
|
||||
run-full:
|
||||
@echo "启动BigDataTool完整服务..."
|
||||
docker-compose up -d
|
||||
@echo "服务启动中,请等待30秒后访问:"
|
||||
@echo " - 主应用: http://localhost:5000"
|
||||
@echo " - Redis缓存: localhost:6379"
|
||||
|
||||
# 生产环境运行(包含Nginx)
|
||||
run-prod:
|
||||
@echo "启动生产环境服务..."
|
||||
docker-compose --profile production up -d
|
||||
@echo "生产环境服务启动,访问地址:"
|
||||
@echo " - HTTP: http://localhost"
|
||||
@echo " - HTTPS: https://localhost (需要SSL证书)"
|
||||
|
||||
# 监控环境运行
|
||||
run-monitor:
|
||||
@echo "启动监控环境..."
|
||||
docker-compose --profile monitoring up -d
|
||||
@echo "监控服务启动,访问地址:"
|
||||
@echo " - 主应用: http://localhost:5000"
|
||||
@echo " - Prometheus: http://localhost:9090"
|
||||
|
||||
# 停止服务
|
||||
stop:
|
||||
@echo "停止所有服务..."
|
||||
docker-compose down
|
||||
docker-compose -f docker-compose.simple.yml down
|
||||
|
||||
# 重启服务
|
||||
restart: stop run
|
||||
|
||||
# 查看日志
|
||||
logs:
|
||||
@echo "查看服务日志..."
|
||||
docker-compose logs -f bigdatatool
|
||||
|
||||
# 查看特定服务日志
|
||||
logs-app:
|
||||
docker-compose logs -f bigdatatool
|
||||
|
||||
logs-redis:
|
||||
docker-compose logs -f redis-cache
|
||||
|
||||
logs-nginx:
|
||||
docker-compose logs -f nginx
|
||||
|
||||
# 进入容器shell
|
||||
shell:
|
||||
@echo "进入BigDataTool容器..."
|
||||
docker-compose exec bigdatatool /bin/bash
|
||||
|
||||
# 健康检查
|
||||
health:
|
||||
@echo "检查服务健康状态..."
|
||||
@docker-compose ps
|
||||
@echo ""
|
||||
@echo "应用健康检查:"
|
||||
@curl -s http://localhost:5000/api/health | python -m json.tool || echo "服务未响应"
|
||||
|
||||
# 查看运行状态
|
||||
ps:
|
||||
@echo "容器运行状态:"
|
||||
@docker-compose ps
|
||||
|
||||
# 清理资源
|
||||
clean:
|
||||
@echo "清理Docker资源..."
|
||||
docker-compose down -v --remove-orphans
|
||||
docker-compose -f docker-compose.simple.yml down -v --remove-orphans
|
||||
docker system prune -f
|
||||
@echo "清理完成"
|
||||
|
||||
# 强制清理(包括镜像)
|
||||
clean-all: clean
|
||||
@echo "强制清理所有资源..."
|
||||
docker rmi bigdatatool:latest || true
|
||||
docker volume prune -f
|
||||
docker network prune -f
|
||||
|
||||
# 更新镜像
|
||||
update: clean build run
|
||||
|
||||
# 查看资源使用
|
||||
stats:
|
||||
@echo "Docker资源使用情况:"
|
||||
@docker stats --no-stream
|
||||
|
||||
# 备份数据
|
||||
backup:
|
||||
@echo "备份数据库和配置..."
|
||||
@mkdir -p backups/$(shell date +%Y%m%d_%H%M%S)
|
||||
@docker cp bigdatatool:/app/config_groups.db backups/$(shell date +%Y%m%d_%H%M%S)/
|
||||
@echo "备份完成: backups/$(shell date +%Y%m%d_%H%M%S)/"
|
||||
|
||||
# 开发模式运行
|
||||
dev:
|
||||
@echo "开发模式运行..."
|
||||
@docker run --rm -it \
|
||||
-p 5000:5000 \
|
||||
-v $(PWD):/app \
|
||||
-e FLASK_ENV=development \
|
||||
-e FLASK_DEBUG=True \
|
||||
bigdatatool:latest
|
||||
|
||||
# 构建并推送到仓库(需要先登录Docker Hub)
|
||||
publish: build
|
||||
@echo "推送镜像到Docker Hub..."
|
||||
@read -p "请输入Docker Hub用户名: " username && \
|
||||
docker tag bigdatatool:latest $$username/bigdatatool:latest && \
|
||||
docker tag bigdatatool:latest $$username/bigdatatool:2.0 && \
|
||||
docker push $$username/bigdatatool:latest && \
|
||||
docker push $$username/bigdatatool:2.0
|
365
README.md
365
README.md
@@ -35,23 +35,141 @@ BigDataTool是一个功能强大的数据库查询比对工具,专门用于Cas
|
||||
|
||||
## 🛠️ 安装部署
|
||||
|
||||
### 1. 克隆项目
|
||||
### 快速开始
|
||||
|
||||
#### 方式1:直接运行(推荐开发环境)
|
||||
```bash
|
||||
# 1. 克隆项目
|
||||
git clone https://github.com/your-org/BigDataTool.git
|
||||
cd BigDataTool
|
||||
```
|
||||
|
||||
### 2. 安装依赖
|
||||
```bash
|
||||
# 2. 安装依赖
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 3. 启动应用
|
||||
```bash
|
||||
# 3. 启动应用
|
||||
python app.py
|
||||
```
|
||||
|
||||
应用将在 `http://localhost:5000` 启动
|
||||
#### 方式2:Docker容器化部署(推荐生产环境)
|
||||
```bash
|
||||
# 1. 克隆项目
|
||||
git clone https://github.com/your-org/BigDataTool.git
|
||||
cd BigDataTool
|
||||
|
||||
# 2. 构建并启动(简化版本)
|
||||
make build
|
||||
make run
|
||||
|
||||
# 或者使用Docker Compose直接启动
|
||||
docker-compose -f docker-compose.simple.yml up -d
|
||||
```
|
||||
|
||||
#### 方式3:完整Docker环境(包含缓存和监控)
|
||||
```bash
|
||||
# 启动完整服务栈
|
||||
docker-compose up -d
|
||||
|
||||
# 查看服务状态
|
||||
make ps
|
||||
```
|
||||
|
||||
### 容器化部署详情
|
||||
|
||||
#### 🐳 Docker镜像特性
|
||||
- **基础镜像**: Python 3.9 Alpine(轻量级)
|
||||
- **镜像大小**: < 200MB
|
||||
- **安全性**: 非root用户运行
|
||||
- **健康检查**: 内置应用健康监控
|
||||
- **多架构**: 支持AMD64和ARM64
|
||||
|
||||
#### 🚀 一键部署命令
|
||||
```bash
|
||||
# 查看所有可用命令
|
||||
make help
|
||||
|
||||
# 构建镜像
|
||||
make build
|
||||
|
||||
# 启动服务(简化版本)
|
||||
make run
|
||||
|
||||
# 启动完整服务(包含Redis缓存)
|
||||
make run-full
|
||||
|
||||
# 启动生产环境(包含Nginx反向代理)
|
||||
make run-prod
|
||||
|
||||
# 查看服务日志
|
||||
make logs
|
||||
|
||||
# 进入容器调试
|
||||
make shell
|
||||
|
||||
# 健康检查
|
||||
make health
|
||||
|
||||
# 停止服务
|
||||
make stop
|
||||
|
||||
# 清理资源
|
||||
make clean
|
||||
```
|
||||
|
||||
#### 🔧 环境变量配置
|
||||
```bash
|
||||
# 设置应用密钥(生产环境必须设置)
|
||||
export SECRET_KEY="your-super-secret-key-change-in-production"
|
||||
|
||||
# 设置运行环境
|
||||
export FLASK_ENV=production
|
||||
export FLASK_DEBUG=False
|
||||
|
||||
# 数据库配置
|
||||
export DATABASE_URL="sqlite:///config_groups.db"
|
||||
|
||||
# 安全配置
|
||||
export FORCE_HTTPS=true
|
||||
```
|
||||
|
||||
#### 📊 服务端点
|
||||
启动后可访问以下地址:
|
||||
|
||||
**简化部署**:
|
||||
- 主应用: http://localhost:5000
|
||||
|
||||
**完整部署**:
|
||||
- 主应用: http://localhost:5000
|
||||
- Redis缓存: localhost:6379
|
||||
- Prometheus监控: http://localhost:9090
|
||||
|
||||
**生产环境**:
|
||||
- HTTP: http://localhost
|
||||
- HTTPS: https://localhost
|
||||
|
||||
### 传统部署方式
|
||||
|
||||
#### Python虚拟环境部署
|
||||
```bash
|
||||
# 创建虚拟环境
|
||||
python -m venv venv
|
||||
source venv/bin/activate # Linux/Mac
|
||||
# 或 venv\Scripts\activate # Windows
|
||||
|
||||
# 安装依赖
|
||||
pip install -r requirements.txt
|
||||
|
||||
# 启动应用
|
||||
python app.py
|
||||
```
|
||||
|
||||
#### 生产环境部署(Gunicorn)
|
||||
```bash
|
||||
# 安装Gunicorn
|
||||
pip install gunicorn
|
||||
|
||||
# 启动生产服务
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 app:app
|
||||
```
|
||||
|
||||
## 🎯 快速开始
|
||||
|
||||
@@ -88,7 +206,51 @@ python app.py
|
||||
4. 设置查询参数
|
||||
5. 执行比对分析
|
||||
|
||||
## 📊 功能特性
|
||||
## 🏗️ 系统架构
|
||||
|
||||
BigDataTool采用模块化分层架构设计:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ 前端界面层 │
|
||||
│ (HTML + JavaScript + Bootstrap) │
|
||||
└─────────────┬───────────────────────────┘
|
||||
│
|
||||
┌─────────────▼───────────────────────────┐
|
||||
│ API路由层 │
|
||||
│ (Flask Routes) │
|
||||
└─────────────┬───────────────────────────┘
|
||||
│
|
||||
┌─────────────▼───────────────────────────┐
|
||||
│ 业务逻辑层 │
|
||||
│ ┌─────────────┬─────────────────┐ │
|
||||
│ │ 查询引擎 │ 比对引擎 │ │
|
||||
│ │Query Engine │ Comparison │ │
|
||||
│ └─────────────┴─────────────────┘ │
|
||||
└─────────────┬───────────────────────────┘
|
||||
│
|
||||
┌─────────────▼───────────────────────────┐
|
||||
│ 数据访问层 │
|
||||
│ ┌─────────────┬─────────────────┐ │
|
||||
│ │ Cassandra │ Redis │ │
|
||||
│ │ Client │ Client │ │
|
||||
│ └─────────────┴─────────────────┘ │
|
||||
└─────────────┬───────────────────────────┘
|
||||
│
|
||||
┌─────────────▼───────────────────────────┐
|
||||
│ 数据存储层 │
|
||||
│ ┌──────┬──────┬─────────────────┐ │
|
||||
│ │SQLite│Cassandra│ Redis │ │
|
||||
│ │(配置) │ (生产) │ (缓存) │ │
|
||||
│ └──────┴──────┴─────────────────┘ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 核心组件
|
||||
- **查询引擎**: 负责Cassandra和Redis的查询执行
|
||||
- **比对引擎**: 实现智能数据比对算法
|
||||
- **配置管理**: SQLite存储的配置持久化
|
||||
- **日志系统**: 实时查询日志收集和展示
|
||||
|
||||
### 数据比对引擎
|
||||
- **智能JSON比较**:自动处理JSON格式差异和嵌套结构
|
||||
@@ -113,24 +275,24 @@ python app.py
|
||||
### Cassandra配置
|
||||
```json
|
||||
{
|
||||
"cluster_name": "生产集群",
|
||||
"hosts": ["192.168.1.100", "192.168.1.101"],
|
||||
"cluster_name": "示例集群",
|
||||
"hosts": ["127.0.0.1", "127.0.0.2"],
|
||||
"port": 9042,
|
||||
"datacenter": "dc1",
|
||||
"username": "cassandra",
|
||||
"password": "password",
|
||||
"keyspace": "my_keyspace",
|
||||
"table": "my_table"
|
||||
"keyspace": "example_keyspace",
|
||||
"table": "example_table"
|
||||
}
|
||||
```
|
||||
|
||||
### Redis配置
|
||||
```json
|
||||
{
|
||||
"name": "生产Redis",
|
||||
"name": "示例Redis",
|
||||
"nodes": [
|
||||
{"host": "192.168.1.200", "port": 7000},
|
||||
{"host": "192.168.1.201", "port": 7001}
|
||||
{"host": "127.0.0.1", "port": 6379},
|
||||
{"host": "127.0.0.2", "port": 6379}
|
||||
],
|
||||
"password": "redis_password",
|
||||
"socket_timeout": 3,
|
||||
@@ -139,13 +301,40 @@ python app.py
|
||||
}
|
||||
```
|
||||
|
||||
## 📈 性能优化
|
||||
## 📈 性能指标
|
||||
|
||||
- **连接池管理**:优化的数据库连接复用
|
||||
- **批量查询**:支持大批量Key的高效查询
|
||||
- **内存管理**:大结果集的内存友好处理
|
||||
- **并行处理**:多表并行查询和数据比对
|
||||
- **缓存机制**:查询结果和配置的智能缓存
|
||||
### 响应时间
|
||||
- 单表查询(100条记录):< 10秒
|
||||
- 分表查询(100条记录):< 15秒
|
||||
- Redis查询(100个Key):< 10秒
|
||||
- 页面加载时间:< 3秒
|
||||
|
||||
### 系统容量
|
||||
- 最大并发查询数:10个
|
||||
- 单次最大查询记录:10,000条
|
||||
- 支持的数据库连接数:无限制
|
||||
- 内存使用峰值:< 1GB
|
||||
|
||||
### 数据处理能力
|
||||
- Cassandra分表自动计算准确率:> 95%
|
||||
- JSON深度比较支持嵌套层级:无限制
|
||||
- Redis全数据类型支持:100%
|
||||
- 查询历史存储容量:无限制
|
||||
|
||||
## 🔄 版本更新
|
||||
|
||||
### v2.0 (2024-08)
|
||||
- ✨ 新增Redis集群比对功能
|
||||
- ✨ 支持多主键复合查询
|
||||
- ✨ 智能数据类型检测和比对
|
||||
- 🚀 性能优化和UI改进
|
||||
- 📚 完整文档体系建设
|
||||
|
||||
### v1.0 (2024-07)
|
||||
- 🎉 基础Cassandra数据比对功能
|
||||
- 🎉 TWCS分表查询支持
|
||||
- 🎉 配置管理和查询历史
|
||||
- 🎉 Web界面和API接口
|
||||
|
||||
## 🔍 故障排查
|
||||
|
||||
@@ -166,6 +355,17 @@ python app.py
|
||||
- 检查数据库服务器负载
|
||||
- 优化查询条件和索引
|
||||
|
||||
4. **内存使用过高**
|
||||
- 减少单次查询的记录数量
|
||||
- 使用分批查询处理大数据集
|
||||
- 定期清理查询历史和日志
|
||||
|
||||
5. **分表查询失败**
|
||||
- 检查Key中是否包含有效时间戳
|
||||
- 确认分表参数配置正确
|
||||
- 验证目标分表是否存在
|
||||
|
||||
|
||||
## 📝 API文档
|
||||
|
||||
### 主要API端点
|
||||
@@ -178,35 +378,120 @@ python app.py
|
||||
- `GET /api/query-history` - 获取查询历史
|
||||
- `GET /api/query-logs` - 获取查询日志
|
||||
|
||||
详细API文档请参考 [API.md](docs/API.md)
|
||||
|
||||
## 📚 文档目录
|
||||
|
||||
- [API文档](docs/API.md) - 完整的API接口说明
|
||||
- [使用指南](docs/USER_GUIDE.md) - 详细的功能使用说明
|
||||
- [架构设计](docs/ARCHITECTURE.md) - 系统架构和设计原理
|
||||
- [部署指南](docs/DEPLOYMENT.md) - 生产环境部署说明
|
||||
|
||||
## 🤝 贡献指南
|
||||
|
||||
1. Fork 项目
|
||||
2. 创建功能分支 (`git checkout -b feature/AmazingFeature`)
|
||||
3. 提交更改 (`git commit -m 'Add some AmazingFeature'`)
|
||||
4. 推送到分支 (`git push origin feature/AmazingFeature`)
|
||||
5. 开启 Pull Request
|
||||
我们欢迎所有形式的贡献!请遵循以下步骤:
|
||||
|
||||
### 基本流程
|
||||
1. **Fork项目**
|
||||
```bash
|
||||
git clone https://github.com/your-username/BigDataTool.git
|
||||
cd BigDataTool
|
||||
```
|
||||
|
||||
2. **创建功能分支**
|
||||
```bash
|
||||
git checkout -b feature/amazing-feature
|
||||
```
|
||||
|
||||
3. **遵循代码规范**
|
||||
- 查看 [代码规范](docs/coding-standards.md)
|
||||
- 使用PEP 8 Python风格
|
||||
- 添加必要的测试用例
|
||||
- 更新相关文档
|
||||
|
||||
4. **提交更改**
|
||||
```bash
|
||||
git commit -m 'feat: Add some AmazingFeature'
|
||||
```
|
||||
|
||||
5. **推送到分支**
|
||||
```bash
|
||||
git push origin feature/amazing-feature
|
||||
```
|
||||
|
||||
6. **创建Pull Request**
|
||||
- 描述变更内容和原因
|
||||
- 确保所有测试通过
|
||||
- 添加必要的截图或演示
|
||||
|
||||
### 贡献类型
|
||||
- 🐛 Bug修复
|
||||
- ✨ 新功能开发
|
||||
- 📚 文档改进
|
||||
- 🎨 界面优化
|
||||
- 🚀 性能优化
|
||||
- 🔧 配置和工具
|
||||
|
||||
### 代码审查
|
||||
所有贡献都将经过代码审查,包括:
|
||||
- 功能正确性验证
|
||||
- 代码质量检查
|
||||
- 安全性评估
|
||||
- 文档完整性确认
|
||||
|
||||
详细开发指南请参考 [开发者文档](docs/developer-guide.md)
|
||||
|
||||
## 🛡️ 安全声明
|
||||
|
||||
BigDataTool致力于数据安全:
|
||||
|
||||
- 🔒 **传输加密**: 支持HTTPS/TLS加密传输
|
||||
- 🔐 **认证机制**: 预留身份认证和权限控制接口
|
||||
- 🔍 **输入验证**: 严格的输入参数验证和过滤
|
||||
- 📝 **审计日志**: 完整的操作日志和安全事件记录
|
||||
- 🛡️ **数据保护**: 敏感信息不明文存储
|
||||
|
||||
如发现安全漏洞,请发送邮件至安全团队或创建私密Issue。
|
||||
|
||||
详细安全规范请参考 [安全指南](docs/security-guidelines.md)
|
||||
|
||||
## 📄 许可证
|
||||
|
||||
本项目采用 MIT 许可证 - 查看 [LICENSE](LICENSE) 文件了解详情
|
||||
|
||||
## 👥 作者
|
||||
## 👥 项目团队
|
||||
|
||||
BigDataTool项目组
|
||||
### 核心开发者
|
||||
- **项目负责人**: BigDataTool项目组
|
||||
- **架构师**: 系统架构设计团队
|
||||
- **前端开发**: UI/UX开发团队
|
||||
- **后端开发**: 数据处理引擎团队
|
||||
- **测试工程师**: 质量保证团队
|
||||
|
||||
### 贡献者统计
|
||||
感谢所有为项目做出贡献的开发者!
|
||||
|
||||
## 📞 支持与反馈
|
||||
|
||||
### 问题报告
|
||||
- 🐛 [Bug报告](https://github.com/your-org/BigDataTool/issues/new?template=bug_report.md)
|
||||
- ✨ [功能请求](https://github.com/your-org/BigDataTool/issues/new?template=feature_request.md)
|
||||
- ❓ [问题讨论](https://github.com/your-org/BigDataTool/discussions)
|
||||
|
||||
### 社区支持
|
||||
- 📚 查看 [用户手册](docs/user-manual.md) 获取详细使用说明
|
||||
- 🔧 查看 [故障排查指南](docs/operations.md) 解决常见问题
|
||||
- 💬 加入社区讨论组获取实时帮助
|
||||
|
||||
## 🙏 致谢
|
||||
|
||||
感谢所有为这个项目做出贡献的开发者和用户。
|
||||
感谢以下开源项目和技术社区的支持:
|
||||
|
||||
- **[Flask](https://flask.palletsprojects.com/)** - 轻量级Web框架
|
||||
- **[Cassandra](https://cassandra.apache.org/)** - 分布式NoSQL数据库
|
||||
- **[Redis](https://redis.io/)** - 高性能键值存储
|
||||
- **[Bootstrap](https://getbootstrap.com/)** - 前端UI框架
|
||||
- **[jQuery](https://jquery.com/)** - JavaScript库
|
||||
|
||||
特别感谢所有提供反馈、bug报告和功能建议的用户!
|
||||
|
||||
---
|
||||
|
||||
**注意**:使用前请确保已正确配置数据库连接信息,并在生产环境中谨慎使用。
|
||||
## 📊 项目状态
|
||||
|
||||
**最后更新**: 2024年8月6日
|
||||
**当前版本**: v2.0
|
||||
**开发状态**: 持续维护中
|
||||
|
||||
> ⚠️ **重要提示**: 本工具主要用于开发测试环境的数据比对,生产环境使用请谨慎评估并做好安全防护。建议在使用前详细阅读 [安全指南](docs/security-guidelines.md)。
|
||||
|
13
app.py
13
app.py
@@ -76,6 +76,15 @@ if __name__ == '__main__':
|
||||
logger.info("=== BigDataTool 启动 ===")
|
||||
logger.info("应用架构:模块化")
|
||||
logger.info("支持功能:单表查询、分表查询、多主键查询、配置管理、查询历史")
|
||||
logger.info("访问地址:http://localhost:5000")
|
||||
|
||||
# 从环境变量获取配置,支持Docker部署
|
||||
import os
|
||||
host = os.getenv('FLASK_HOST', '0.0.0.0')
|
||||
port = int(os.getenv('FLASK_PORT', 5000))
|
||||
debug = os.getenv('FLASK_DEBUG', 'True').lower() == 'true'
|
||||
|
||||
logger.info(f"访问地址:http://{host}:{port}")
|
||||
logger.info("API文档:/api/* 路径下的所有端点")
|
||||
app.run(debug=True, port=5000)
|
||||
logger.info(f"配置信息 - 主机: {host}, 端口: {port}, 调试: {debug}")
|
||||
|
||||
app.run(debug=debug, host=host, port=port)
|
29
docker-compose.simple.yml
Normal file
29
docker-compose.simple.yml
Normal file
@@ -0,0 +1,29 @@
|
||||
# BigDataTool 简化版本 Docker Compose 配置
|
||||
# 适用于快速开发和测试
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
bigdatatool:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: bigdatatool
|
||||
ports:
|
||||
- "8080:5000"
|
||||
environment:
|
||||
- FLASK_ENV=production
|
||||
- FLASK_DEBUG=False
|
||||
- FLASK_HOST=0.0.0.0
|
||||
- FLASK_PORT=5000
|
||||
# volumes:
|
||||
# # 持久化数据库
|
||||
# - ./data:/app/data
|
||||
# # 持久化日志
|
||||
# - ./logs:/app/logs
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:5000/api/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 40s
|
237
docker-entrypoint.sh
Executable file
237
docker-entrypoint.sh
Executable file
@@ -0,0 +1,237 @@
|
||||
#!/bin/bash
|
||||
|
||||
# BigDataTool Docker 启动脚本
|
||||
# 用于容器化部署的入口脚本
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色定义
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 日志函数
|
||||
log_info() {
|
||||
echo -e "${BLUE}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo -e "${YELLOW}[WARN]${NC} $1"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[SUCCESS]${NC} $1"
|
||||
}
|
||||
|
||||
# 显示启动信息
|
||||
show_banner() {
|
||||
echo "======================================"
|
||||
echo " BigDataTool Container Startup"
|
||||
echo "======================================"
|
||||
echo "Version: 2.0"
|
||||
echo "Python: $(python --version)"
|
||||
echo "Working Directory: $(pwd)"
|
||||
echo "User: $(whoami)"
|
||||
echo "======================================"
|
||||
}
|
||||
|
||||
# 检查环境变量
|
||||
check_environment() {
|
||||
log_info "检查环境变量..."
|
||||
|
||||
# 设置默认值
|
||||
export FLASK_ENV=${FLASK_ENV:-production}
|
||||
export FLASK_DEBUG=${FLASK_DEBUG:-False}
|
||||
export SECRET_KEY=${SECRET_KEY:-$(python -c "import secrets; print(secrets.token_hex(32))")}
|
||||
export DATABASE_URL=${DATABASE_URL:-sqlite:///config_groups.db}
|
||||
|
||||
log_info "FLASK_ENV: $FLASK_ENV"
|
||||
log_info "FLASK_DEBUG: $FLASK_DEBUG"
|
||||
log_info "数据库URL: $DATABASE_URL"
|
||||
|
||||
# 检查必要的环境变量
|
||||
if [ -z "$SECRET_KEY" ]; then
|
||||
log_warn "SECRET_KEY 未设置,使用随机生成的密钥"
|
||||
fi
|
||||
}
|
||||
|
||||
# 初始化数据库
|
||||
initialize_database() {
|
||||
log_info "初始化数据库..."
|
||||
|
||||
# 检查数据库文件是否存在
|
||||
if [ ! -f "config_groups.db" ]; then
|
||||
log_info "数据库文件不存在,将自动创建"
|
||||
python -c "
|
||||
from modules.database import ensure_database
|
||||
if ensure_database():
|
||||
print('数据库初始化成功')
|
||||
else:
|
||||
print('数据库初始化失败')
|
||||
exit(1)
|
||||
"
|
||||
if [ $? -eq 0 ]; then
|
||||
log_success "数据库初始化完成"
|
||||
else
|
||||
log_error "数据库初始化失败"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log_info "数据库文件已存在"
|
||||
fi
|
||||
}
|
||||
|
||||
# 创建必要目录
|
||||
create_directories() {
|
||||
log_info "创建必要目录..."
|
||||
|
||||
# 创建日志目录
|
||||
if [ ! -d "logs" ]; then
|
||||
mkdir -p logs
|
||||
log_info "创建日志目录: logs"
|
||||
fi
|
||||
|
||||
# 创建配置目录
|
||||
if [ ! -d "config" ]; then
|
||||
mkdir -p config
|
||||
log_info "创建配置目录: config"
|
||||
fi
|
||||
|
||||
# 设置权限
|
||||
chmod -R 755 logs config || true
|
||||
}
|
||||
|
||||
# 健康检查函数
|
||||
health_check() {
|
||||
log_info "执行健康检查..."
|
||||
|
||||
local max_attempts=30
|
||||
local attempt=1
|
||||
|
||||
while [ $attempt -le $max_attempts ]; do
|
||||
if wget --no-verbose --tries=1 --spider http://localhost:5000/api/health >/dev/null 2>&1; then
|
||||
log_success "应用健康检查通过"
|
||||
return 0
|
||||
fi
|
||||
|
||||
log_info "等待应用启动... ($attempt/$max_attempts)"
|
||||
sleep 2
|
||||
((attempt++))
|
||||
done
|
||||
|
||||
log_error "健康检查失败,应用可能未正常启动"
|
||||
return 1
|
||||
}
|
||||
|
||||
# 信号处理函数
|
||||
cleanup() {
|
||||
log_warn "收到退出信号,正在清理..."
|
||||
|
||||
# 这里可以添加清理逻辑
|
||||
# 例如:保存缓存、关闭数据库连接等
|
||||
|
||||
log_info "清理完成,退出应用"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# 设置信号处理
|
||||
trap cleanup SIGTERM SIGINT
|
||||
|
||||
# 主启动函数
|
||||
start_application() {
|
||||
log_info "启动BigDataTool应用..."
|
||||
|
||||
# 根据环境变量选择启动方式
|
||||
if [ "$FLASK_ENV" = "development" ]; then
|
||||
log_info "以开发模式启动"
|
||||
python app.py
|
||||
else
|
||||
log_info "以生产模式启动"
|
||||
|
||||
# 检查是否安装了gunicorn
|
||||
if command -v gunicorn >/dev/null 2>&1; then
|
||||
log_info "使用Gunicorn启动应用"
|
||||
exec gunicorn \
|
||||
--bind 0.0.0.0:5000 \
|
||||
--workers 4 \
|
||||
--worker-class sync \
|
||||
--worker-connections 1000 \
|
||||
--max-requests 1000 \
|
||||
--max-requests-jitter 50 \
|
||||
--timeout 120 \
|
||||
--keep-alive 5 \
|
||||
--log-level info \
|
||||
--access-logfile - \
|
||||
--error-logfile - \
|
||||
app:app
|
||||
else
|
||||
log_warn "Gunicorn未安装,使用Flask开发服务器"
|
||||
python app.py
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# 显示帮助信息
|
||||
show_help() {
|
||||
echo "BigDataTool Docker 启动脚本"
|
||||
echo ""
|
||||
echo "用法: $0 [选项]"
|
||||
echo ""
|
||||
echo "选项:"
|
||||
echo " start 启动应用(默认)"
|
||||
echo " health-check 执行健康检查"
|
||||
echo " init-db 仅初始化数据库"
|
||||
echo " shell 进入交互式shell"
|
||||
echo " help 显示此帮助信息"
|
||||
echo ""
|
||||
echo "环境变量:"
|
||||
echo " FLASK_ENV Flask运行环境 (development/production)"
|
||||
echo " FLASK_DEBUG 是否启用调试模式 (True/False)"
|
||||
echo " SECRET_KEY 应用密钥"
|
||||
echo " DATABASE_URL 数据库连接URL"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# 主逻辑
|
||||
main() {
|
||||
case "${1:-start}" in
|
||||
start)
|
||||
show_banner
|
||||
check_environment
|
||||
create_directories
|
||||
initialize_database
|
||||
start_application
|
||||
;;
|
||||
health-check)
|
||||
health_check
|
||||
;;
|
||||
init-db)
|
||||
log_info "仅初始化数据库模式"
|
||||
check_environment
|
||||
create_directories
|
||||
initialize_database
|
||||
log_success "数据库初始化完成"
|
||||
;;
|
||||
shell)
|
||||
log_info "进入交互式shell"
|
||||
exec /bin/bash
|
||||
;;
|
||||
help|--help|-h)
|
||||
show_help
|
||||
;;
|
||||
*)
|
||||
log_error "未知选项: $1"
|
||||
show_help
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# 执行主函数
|
||||
main "$@"
|
@@ -20,7 +20,8 @@ from .config_manager import (
|
||||
save_redis_query_history, get_redis_query_history,
|
||||
get_redis_query_history_by_id, delete_redis_query_history,
|
||||
batch_delete_redis_query_history,
|
||||
parse_redis_config_from_yaml
|
||||
parse_redis_config_from_yaml,
|
||||
convert_bytes_to_str # 添加bytes转换函数
|
||||
)
|
||||
from .cassandra_client import create_connection
|
||||
from .query_engine import execute_query, execute_mixed_query
|
||||
@@ -48,6 +49,12 @@ def setup_routes(app, query_log_collector):
|
||||
@app.route('/db-compare')
|
||||
def db_compare():
|
||||
return render_template('db_compare.html')
|
||||
|
||||
# 新增:更语义化的路由别名
|
||||
@app.route('/cassandra-compare')
|
||||
def cassandra_compare():
|
||||
"""Cassandra数据比对工具 - 语义化路由"""
|
||||
return render_template('db_compare.html')
|
||||
|
||||
@app.route('/redis-compare')
|
||||
def redis_compare():
|
||||
@@ -62,6 +69,32 @@ def setup_routes(app, query_log_collector):
|
||||
return render_template('redis_test.html')
|
||||
|
||||
# 基础API
|
||||
@app.route('/api/health')
|
||||
def health_check():
|
||||
"""健康检查端点,用于Docker健康检查和服务监控"""
|
||||
try:
|
||||
# 检查应用基本状态
|
||||
current_time = datetime.now().isoformat()
|
||||
|
||||
# 简单的数据库连接检查(可选)
|
||||
from .database import ensure_database
|
||||
db_status = ensure_database()
|
||||
|
||||
return jsonify({
|
||||
'status': 'healthy',
|
||||
'timestamp': current_time,
|
||||
'service': 'BigDataTool',
|
||||
'version': '2.0',
|
||||
'database': 'ok' if db_status else 'warning'
|
||||
})
|
||||
except Exception as e:
|
||||
logger.error(f"健康检查失败: {str(e)}")
|
||||
return jsonify({
|
||||
'status': 'unhealthy',
|
||||
'error': str(e),
|
||||
'timestamp': datetime.now().isoformat()
|
||||
}), 503
|
||||
|
||||
@app.route('/api/default-config')
|
||||
def get_default_config():
|
||||
return jsonify(DEFAULT_CONFIG)
|
||||
@@ -234,6 +267,8 @@ def setup_routes(app, query_log_collector):
|
||||
|
||||
# 结束查询批次
|
||||
query_log_collector.end_current_batch()
|
||||
# 转换result中可能包含的bytes类型数据
|
||||
result = convert_bytes_to_str(result)
|
||||
return jsonify(result)
|
||||
|
||||
except Exception as e:
|
||||
@@ -389,6 +424,8 @@ def setup_routes(app, query_log_collector):
|
||||
|
||||
# 结束查询批次
|
||||
query_log_collector.end_current_batch()
|
||||
# 转换result中可能包含的bytes类型数据
|
||||
result = convert_bytes_to_str(result)
|
||||
return jsonify(result)
|
||||
|
||||
except Exception as e:
|
||||
@@ -1185,7 +1222,7 @@ def setup_routes(app, query_log_collector):
|
||||
# 过滤每个批次中的日志,只保留Redis相关的
|
||||
redis_logs = [
|
||||
log for log in logs
|
||||
if log.get('query_type') == 'redis' or
|
||||
if log.get('query_type', '').lower() == 'redis' or
|
||||
(log.get('message') and 'redis' in log.get('message', '').lower())
|
||||
]
|
||||
if redis_logs: # 只有当批次中有Redis日志时才添加
|
||||
@@ -1207,7 +1244,7 @@ def setup_routes(app, query_log_collector):
|
||||
# 过滤Redis相关的日志
|
||||
redis_logs = [
|
||||
log for log in logs
|
||||
if log.get('query_type') == 'redis' or
|
||||
if log.get('query_type', '').lower() == 'redis' or
|
||||
(log.get('message') and 'redis' in log.get('message', '').lower())
|
||||
]
|
||||
|
||||
|
@@ -83,16 +83,16 @@ def create_connection(config):
|
||||
|
||||
使用示例:
|
||||
config = {
|
||||
'hosts': ['192.168.1.100', '192.168.1.101'],
|
||||
'hosts': ['127.0.0.1'],
|
||||
'port': 9042,
|
||||
'username': 'cassandra',
|
||||
'password': 'password',
|
||||
'keyspace': 'my_keyspace',
|
||||
'keyspace': 'example_keyspace',
|
||||
'datacenter': 'dc1'
|
||||
}
|
||||
cluster, session = create_connection(config)
|
||||
if session:
|
||||
result = session.execute("SELECT * FROM my_table LIMIT 10")
|
||||
result = session.execute("SELECT * FROM example_table LIMIT 10")
|
||||
cluster.shutdown()
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
@@ -39,6 +39,31 @@ from .database import ensure_database, get_db_connection
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def convert_bytes_to_str(obj):
|
||||
"""递归转换对象中的bytes类型为字符串,用于JSON序列化
|
||||
|
||||
Args:
|
||||
obj: 需要转换的对象(可以是dict, list或其他类型)
|
||||
|
||||
Returns:
|
||||
转换后的对象,所有bytes类型都被转换为hex字符串
|
||||
"""
|
||||
if isinstance(obj, bytes):
|
||||
# 将bytes转换为十六进制字符串
|
||||
return obj.hex()
|
||||
elif isinstance(obj, dict):
|
||||
# 递归处理字典
|
||||
return {key: convert_bytes_to_str(value) for key, value in obj.items()}
|
||||
elif isinstance(obj, list):
|
||||
# 递归处理列表
|
||||
return [convert_bytes_to_str(item) for item in obj]
|
||||
elif isinstance(obj, tuple):
|
||||
# 递归处理元组
|
||||
return tuple(convert_bytes_to_str(item) for item in obj)
|
||||
else:
|
||||
# 其他类型直接返回
|
||||
return obj
|
||||
|
||||
# Cassandra数据库默认配置模板
|
||||
# 注意:此配置不包含敏感信息,仅作为UI表单的初始模板使用
|
||||
DEFAULT_CONFIG = {
|
||||
@@ -233,6 +258,9 @@ def save_redis_query_history(name, description, cluster1_config, cluster2_config
|
||||
cursor = conn.cursor()
|
||||
|
||||
try:
|
||||
# 转换可能包含bytes类型的数据
|
||||
raw_results = convert_bytes_to_str(raw_results) if raw_results else None
|
||||
|
||||
cursor.execute('''
|
||||
INSERT INTO redis_query_history
|
||||
(name, description, cluster1_config, cluster2_config, query_options, query_keys,
|
||||
@@ -600,6 +628,11 @@ def save_query_history(name, description, pro_config, test_config, query_config,
|
||||
cursor = conn.cursor()
|
||||
|
||||
try:
|
||||
# 转换可能包含bytes类型的数据
|
||||
raw_results = convert_bytes_to_str(raw_results) if raw_results else None
|
||||
differences_data = convert_bytes_to_str(differences_data) if differences_data else None
|
||||
identical_data = convert_bytes_to_str(identical_data) if identical_data else None
|
||||
|
||||
cursor.execute('''
|
||||
INSERT INTO query_history
|
||||
(name, description, pro_config, test_config, query_config, query_keys,
|
||||
|
@@ -86,8 +86,8 @@ def compare_results(pro_data, test_data, keys, fields_to_compare, exclude_fields
|
||||
value_pro = getattr(row_pro, column)
|
||||
value_test = getattr(row_test, column)
|
||||
|
||||
# 使用智能比较函数
|
||||
if not compare_values(value_pro, value_test):
|
||||
# 使用智能比较函数,传递字段名用于标签字段判断
|
||||
if not compare_values(value_pro, value_test, column):
|
||||
has_difference = True
|
||||
# 格式化显示值
|
||||
formatted_pro_value = format_json_for_display(value_pro)
|
||||
@@ -99,7 +99,8 @@ def compare_results(pro_data, test_data, keys, fields_to_compare, exclude_fields
|
||||
'pro_value': formatted_pro_value,
|
||||
'test_value': formatted_test_value,
|
||||
'is_json': is_json_field(value_pro) or is_json_field(value_test),
|
||||
'is_array': is_json_array_field(value_pro) or is_json_array_field(value_test)
|
||||
'is_array': is_json_array_field(value_pro) or is_json_array_field(value_test),
|
||||
'is_tag': is_tag_field(column, value_pro) or is_tag_field(column, value_test)
|
||||
})
|
||||
|
||||
# 统计字段差异次数
|
||||
@@ -279,6 +280,11 @@ def compare_json_arrays(array1, array2):
|
||||
|
||||
def format_json_for_display(value):
|
||||
"""格式化JSON用于显示"""
|
||||
# 处理None值
|
||||
if value is None:
|
||||
return "null"
|
||||
|
||||
# 处理非字符串类型
|
||||
if not isinstance(value, str):
|
||||
return str(value)
|
||||
|
||||
@@ -302,9 +308,54 @@ def is_json_field(value):
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return False
|
||||
|
||||
def compare_values(value1, value2):
|
||||
"""智能比较两个值,支持JSON标准化和数组比较"""
|
||||
# 首先检查是否为数组类型
|
||||
def is_tag_field(field_name, value):
|
||||
"""判断是否为标签类字段(空格分隔的标签列表)
|
||||
|
||||
标签字段特征:
|
||||
1. 字段名包含 'tag' 关键字
|
||||
2. 值是字符串类型
|
||||
3. 包含空格分隔的多个元素
|
||||
"""
|
||||
if not isinstance(value, str):
|
||||
return False
|
||||
|
||||
# 检查字段名是否包含tag
|
||||
if field_name and 'tag' in field_name.lower():
|
||||
# 检查是否包含空格分隔的多个元素
|
||||
elements = value.strip().split()
|
||||
if len(elements) > 1:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def compare_tag_values(value1, value2):
|
||||
"""比较标签类字段的值(忽略顺序)
|
||||
|
||||
将空格分隔的标签字符串拆分成集合进行比较
|
||||
"""
|
||||
if not isinstance(value1, str) or not isinstance(value2, str):
|
||||
return value1 == value2
|
||||
|
||||
# 将标签字符串拆分成集合
|
||||
tags1 = set(value1.strip().split())
|
||||
tags2 = set(value2.strip().split())
|
||||
|
||||
# 比较集合是否相等(忽略顺序)
|
||||
return tags1 == tags2
|
||||
|
||||
def compare_values(value1, value2, field_name=None):
|
||||
"""智能比较两个值,支持JSON标准化、数组比较和标签比较
|
||||
|
||||
Args:
|
||||
value1: 第一个值
|
||||
value2: 第二个值
|
||||
field_name: 字段名(可选,用于判断是否为标签字段)
|
||||
"""
|
||||
# 检查是否为标签字段
|
||||
if field_name and (is_tag_field(field_name, value1) or is_tag_field(field_name, value2)):
|
||||
return compare_tag_values(value1, value2)
|
||||
|
||||
# 检查是否为数组类型
|
||||
if is_json_array_field(value1) or is_json_array_field(value2):
|
||||
return compare_array_values(value1, value2)
|
||||
|
||||
|
@@ -65,7 +65,7 @@ def get_redis_value_with_type(redis_client, key):
|
||||
- 数据异常:记录警告并提供基本信息
|
||||
|
||||
示例:
|
||||
>>> result = get_redis_value_with_type(client, "user:1001")
|
||||
>>> result = get_redis_value_with_type(client, "user:example")
|
||||
>>> print(result['type']) # 'string'
|
||||
>>> print(result['value']) # 'John Doe'
|
||||
>>> print(result['exists']) # True
|
||||
|
779
static/js/app.js
779
static/js/app.js
@@ -908,35 +908,141 @@ function displayStats(results) {
|
||||
function displayDifferences() {
|
||||
const differencesContainer = document.getElementById('differences');
|
||||
|
||||
if (!filteredDifferenceResults.length) {
|
||||
differencesContainer.innerHTML = '<p class="text-success"><i class="fas fa-check"></i> 未发现差异</p>';
|
||||
// 保存当前搜索框的值
|
||||
const currentSearchValue = document.getElementById('differenceSearch')?.value || '';
|
||||
|
||||
if (!filteredDifferenceResults || !filteredDifferenceResults.length) {
|
||||
// 显示搜索和控制界面,即使没有结果
|
||||
let html = `
|
||||
<!-- 字段筛选按钮和操作按钮 -->
|
||||
<div class="mb-3">
|
||||
<div class="row">
|
||||
<div class="col-md-8">
|
||||
<div class="btn-group btn-group-sm flex-wrap" role="group">
|
||||
<button type="button" class="btn btn-outline-primary active" onclick="filterByField('')">
|
||||
全部 (0)
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<div class="d-flex justify-content-end">
|
||||
<button type="button" class="btn btn-outline-primary btn-sm me-2" onclick="toggleAllDifferenceCollapse()">
|
||||
<i class="fas fa-expand-alt"></i> <span id="toggleAllText">全部收起</span>
|
||||
</button>
|
||||
<button type="button" class="btn btn-success btn-sm" onclick="copyDifferenceKeys()">
|
||||
<i class="fas fa-copy"></i> 复制差异主键
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 搜索框 -->
|
||||
<div class="mb-3">
|
||||
<div class="input-group">
|
||||
<span class="input-group-text"><i class="fas fa-search"></i></span>
|
||||
<input type="text" class="form-control" placeholder="搜索主键、字段名或值内容..."
|
||||
onkeypress="if(event.key === 'Enter') searchDifferenceResults(this.value)"
|
||||
id="differenceSearch"
|
||||
value="${currentSearchValue}">
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="searchDifferenceResults(document.getElementById('differenceSearch').value)">
|
||||
<i class="fas fa-search"></i> 搜索
|
||||
</button>
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="clearDifferenceSearch()">
|
||||
<i class="fas fa-times"></i> 清除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 无结果提示 -->
|
||||
<div class="alert alert-info text-center">
|
||||
<i class="fas fa-search"></i>
|
||||
${currentSearchValue ? `没有找到包含"${escapeHtml(currentSearchValue)}"的差异记录` : '未发现差异'}
|
||||
${currentSearchValue ? '<br><small class="text-muted">请尝试其他搜索条件或点击"清除"按钮查看所有结果</small>' : ''}
|
||||
</div>
|
||||
`;
|
||||
|
||||
differencesContainer.innerHTML = html;
|
||||
|
||||
// 恢复搜索框的焦点和光标位置
|
||||
if (currentSearchValue) {
|
||||
const searchInput = document.getElementById('differenceSearch');
|
||||
if (searchInput) {
|
||||
searchInput.focus();
|
||||
searchInput.setSelectionRange(searchInput.value.length, searchInput.value.length);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// 计算分页
|
||||
const totalPages = Math.ceil(filteredDifferenceResults.length / differencePageSize);
|
||||
// 调试日志:检查差异数据
|
||||
console.log('差异数据:', filteredDifferenceResults);
|
||||
|
||||
// 按主键分组差异
|
||||
const groupedDifferences = groupDifferencesByKey(filteredDifferenceResults);
|
||||
const totalGroups = Object.keys(groupedDifferences).length;
|
||||
|
||||
// 计算分页(基于主键组)
|
||||
const groupKeys = Object.keys(groupedDifferences);
|
||||
const totalPages = Math.ceil(groupKeys.length / differencePageSize);
|
||||
const startIndex = (currentDifferencePage - 1) * differencePageSize;
|
||||
const endIndex = startIndex + differencePageSize;
|
||||
const currentPageData = filteredDifferenceResults.slice(startIndex, endIndex);
|
||||
const currentPageKeys = groupKeys.slice(startIndex, endIndex);
|
||||
|
||||
// 统计字段差异类型
|
||||
const fieldStats = {};
|
||||
filteredDifferenceResults.forEach(diff => {
|
||||
if (diff.field) {
|
||||
fieldStats[diff.field] = (fieldStats[diff.field] || 0) + 1;
|
||||
}
|
||||
});
|
||||
|
||||
let html = `
|
||||
<!-- 字段筛选按钮和操作按钮 -->
|
||||
<div class="mb-3">
|
||||
<div class="row">
|
||||
<div class="col-md-8">
|
||||
<div class="btn-group btn-group-sm flex-wrap" role="group">
|
||||
<button type="button" class="btn btn-outline-primary active" onclick="filterByField('')">
|
||||
全部 (${filteredDifferenceResults.length})
|
||||
</button>
|
||||
${Object.entries(fieldStats).map(([field, count]) => `
|
||||
<button type="button" class="btn btn-outline-secondary" onclick="filterByField('${field}')">
|
||||
${field} (${count})
|
||||
</button>
|
||||
`).join('')}
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<div class="d-flex justify-content-end">
|
||||
<button type="button" class="btn btn-outline-primary btn-sm me-2" onclick="toggleAllDifferenceCollapse()">
|
||||
<i class="fas fa-expand-alt"></i> <span id="toggleAllText">全部收起</span>
|
||||
</button>
|
||||
<button type="button" class="btn btn-success btn-sm" onclick="copyDifferenceKeys()">
|
||||
<i class="fas fa-copy"></i> 复制差异主键
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 分页控制 -->
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-6">
|
||||
<div class="d-flex align-items-center">
|
||||
<label class="form-label me-2 mb-0">每页显示:</label>
|
||||
<select class="form-select form-select-sm me-2" style="width: auto;" onchange="changeDifferencePageSize(this.value)">
|
||||
<option value="5" ${differencePageSize == 5 ? 'selected' : ''}>5条</option>
|
||||
<option value="10" ${differencePageSize == 10 ? 'selected' : ''}>10条</option>
|
||||
<option value="20" ${differencePageSize == 20 ? 'selected' : ''}>20条</option>
|
||||
<option value="50" ${differencePageSize == 50 ? 'selected' : ''}>50条</option>
|
||||
<option value="100" ${differencePageSize == 100 ? 'selected' : ''}>100条</option>
|
||||
<option value="5" ${differencePageSize == 5 ? 'selected' : ''}>5组</option>
|
||||
<option value="10" ${differencePageSize == 10 ? 'selected' : ''}>10组</option>
|
||||
<option value="20" ${differencePageSize == 20 ? 'selected' : ''}>20组</option>
|
||||
<option value="50" ${differencePageSize == 50 ? 'selected' : ''}>50组</option>
|
||||
<option value="100" ${differencePageSize == 100 ? 'selected' : ''}>100组</option>
|
||||
<option value="custom_diff">自定义</option>
|
||||
</select>
|
||||
<input type="number" class="form-control form-control-sm me-2" style="width: 80px; display: none;"
|
||||
id="customDiffPageSize" placeholder="数量" min="1" max="1000"
|
||||
onchange="setCustomDifferencePageSize(this.value)" onkeypress="handleCustomDiffPageSizeEnter(event)">
|
||||
<span class="ms-2 text-muted">共 ${filteredDifferenceResults.length} 条差异记录</span>
|
||||
<span class="ms-2 text-muted">共 ${totalGroups} 个主键组,${filteredDifferenceResults.length} 条差异记录</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6">
|
||||
@@ -951,94 +1057,124 @@ function displayDifferences() {
|
||||
<div class="input-group">
|
||||
<span class="input-group-text"><i class="fas fa-search"></i></span>
|
||||
<input type="text" class="form-control" placeholder="搜索主键、字段名或值内容..."
|
||||
onkeyup="searchDifferenceResults(this.value)" id="differenceSearch">
|
||||
onkeypress="if(event.key === 'Enter') searchDifferenceResults(this.value)"
|
||||
id="differenceSearch"
|
||||
value="${currentSearchValue}">
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="searchDifferenceResults(document.getElementById('differenceSearch').value)">
|
||||
<i class="fas fa-search"></i> 搜索
|
||||
</button>
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="clearDifferenceSearch()">
|
||||
<i class="fas fa-times"></i> 清除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// 显示当前页数据
|
||||
currentPageData.forEach((diff, index) => {
|
||||
const globalIndex = startIndex + index + 1;
|
||||
if (diff.message) {
|
||||
// 记录不存在的情况
|
||||
html += `
|
||||
<div class="difference-item card mb-3 border-warning">
|
||||
<div class="card-header bg-light">
|
||||
<div class="row align-items-center">
|
||||
<div class="col">
|
||||
<strong>差异 #${globalIndex}</strong>
|
||||
<span class="badge bg-warning ms-2">记录缺失</span>
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<button class="btn btn-sm btn-outline-info" onclick='showDifferenceRawData(${JSON.stringify(JSON.stringify(diff.key))})'>
|
||||
<i class="fas fa-code"></i> 原生数据
|
||||
</button>
|
||||
</div>
|
||||
// 显示当前页的主键组
|
||||
currentPageKeys.forEach((keyStr, groupIndex) => {
|
||||
const diffs = groupedDifferences[keyStr];
|
||||
const globalIndex = startIndex + groupIndex + 1;
|
||||
|
||||
// 将字符串化的key解析回对象
|
||||
let keyObj;
|
||||
try {
|
||||
keyObj = JSON.parse(keyStr);
|
||||
} catch (e) {
|
||||
// 如果解析失败,假设它本身就是一个简单值
|
||||
keyObj = keyStr;
|
||||
}
|
||||
|
||||
html += `
|
||||
<div class="card mb-3 border-primary">
|
||||
<div class="card-header bg-primary bg-opacity-10">
|
||||
<div class="row align-items-center">
|
||||
<div class="col">
|
||||
<strong>主键组 #${globalIndex}</strong>
|
||||
<span class="badge bg-primary ms-2">${diffs.length} 个差异字段</span>
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<button class="btn btn-sm btn-outline-info" onclick='showDifferenceRawData(${JSON.stringify(JSON.stringify(keyObj))})'>
|
||||
<i class="fas fa-code"></i> 原生数据
|
||||
</button>
|
||||
</div>
|
||||
<p class="mb-0 mt-2"><strong>主键:</strong> ${formatCompositeKey(diff.key)}</p>
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<p class="text-warning"><i class="fas fa-exclamation-triangle"></i> ${diff.message}</p>
|
||||
</div>
|
||||
<p class="mb-0 mt-2"><strong>主键:</strong> ${formatCompositeKey(keyObj)}</p>
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
// 字段值差异的情况
|
||||
const isJson = diff.is_json;
|
||||
const isArray = diff.is_array;
|
||||
const jsonClass = isJson ? 'json-field' : '';
|
||||
|
||||
html += `
|
||||
<div class="difference-item card mb-3 border-danger">
|
||||
<div class="card-header bg-light">
|
||||
<div class="row align-items-center">
|
||||
<div class="col">
|
||||
<strong>差异 #${globalIndex}</strong>
|
||||
<span class="badge bg-danger ms-2">字段差异</span>
|
||||
${isJson ? '<span class="badge bg-info ms-2">JSON字段</span>' : ''}
|
||||
${isArray ? '<span class="badge bg-warning ms-2">数组字段</span>' : ''}
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<button class="btn btn-sm btn-outline-secondary me-2" type="button"
|
||||
data-bs-toggle="collapse" data-bs-target="#diffCollapse${globalIndex}">
|
||||
<i class="fas fa-eye"></i> 查看详情
|
||||
</button>
|
||||
<button class="btn btn-sm btn-outline-info" onclick='showDifferenceRawData(${JSON.stringify(JSON.stringify(diff.key))})'>
|
||||
<i class="fas fa-code"></i> 原生数据
|
||||
</button>
|
||||
<div class="collapse show" id="keyGroup${globalIndex}">
|
||||
<div class="card-body">
|
||||
`;
|
||||
|
||||
// 显示该主键下的所有差异字段
|
||||
diffs.forEach((diff, diffIndex) => {
|
||||
if (diff.message) {
|
||||
// 记录不存在的情况
|
||||
html += `
|
||||
<div class="alert alert-warning mb-2">
|
||||
<i class="fas fa-exclamation-triangle"></i> ${diff.message}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
// 字段值差异的情况
|
||||
const isJson = diff.is_json;
|
||||
const isArray = diff.is_array;
|
||||
const jsonClass = isJson ? 'json-field' : '';
|
||||
const fieldId = `field_${globalIndex}_${diffIndex}`;
|
||||
|
||||
// 调试:打印差异数据
|
||||
console.log(`差异 ${diff.field}:`, {
|
||||
pro_value: diff.pro_value,
|
||||
test_value: diff.test_value,
|
||||
is_json: diff.is_json,
|
||||
is_array: diff.is_array
|
||||
});
|
||||
|
||||
// 确保值存在
|
||||
const proValue = diff.pro_value !== undefined && diff.pro_value !== null ? diff.pro_value : '无数据';
|
||||
const testValue = diff.test_value !== undefined && diff.test_value !== null ? diff.test_value : '无数据';
|
||||
|
||||
html += `
|
||||
<div class="mb-3 border-start border-3 border-danger ps-3">
|
||||
<div class="d-flex justify-content-between align-items-start mb-2">
|
||||
<div>
|
||||
<strong class="text-danger">${diff.field}</strong>
|
||||
${isJson ? '<span class="badge bg-info ms-2">JSON</span>' : ''}
|
||||
${isArray ? '<span class="badge bg-warning ms-2">数组</span>' : ''}
|
||||
</div>
|
||||
</div>
|
||||
<p class="mb-0 mt-2"><strong>主键:</strong> ${formatCompositeKey(diff.key)}</p>
|
||||
<p class="mb-0"><strong>差异字段:</strong> ${diff.field}</p>
|
||||
</div>
|
||||
<div class="collapse" id="diffCollapse${globalIndex}">
|
||||
<div class="card-body">
|
||||
<div class="mb-4">
|
||||
<div class="field-header mb-2">
|
||||
<i class="fas fa-tag text-primary"></i>
|
||||
<strong>${diff.field}</strong>
|
||||
<div class="collapse show" id="${fieldId}">
|
||||
<div class="row">
|
||||
<div class="col-md-6">
|
||||
<div class="card">
|
||||
<div class="card-header bg-success bg-opacity-10">
|
||||
<small class="text-success fw-bold">生产环境</small>
|
||||
</div>
|
||||
<div class="card-body p-2">
|
||||
<pre class="field-value mb-0 ${jsonClass}" style="max-height: 200px; overflow-y: auto;">${escapeHtml(proValue)}</pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 环境对比数据行 -->
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
<div class="field-container position-relative">
|
||||
<pre class="field-value bg-light p-2 rounded mb-0 ${jsonClass}" style="max-height: 400px; overflow-y: auto; margin: 0;">${escapeHtml(diff.pro_value)}
|
||||
${escapeHtml(diff.test_value)}</pre>
|
||||
<button class="btn btn-sm btn-outline-secondary copy-btn"
|
||||
onclick="copyToClipboard('${escapeForJs(diff.pro_value)}\n${escapeForJs(diff.test_value)}', this)"
|
||||
title="复制全部内容">
|
||||
<i class="fas fa-copy"></i>
|
||||
</button>
|
||||
<div class="col-md-6">
|
||||
<div class="card">
|
||||
<div class="card-header bg-info bg-opacity-10">
|
||||
<small class="text-info fw-bold">测试环境</small>
|
||||
</div>
|
||||
<div class="card-body p-2">
|
||||
<pre class="field-value mb-0 ${jsonClass}" style="max-height: 200px; overflow-y: auto;">${escapeHtml(testValue)}</pre>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
});
|
||||
|
||||
html += `
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
</div>
|
||||
`;
|
||||
});
|
||||
|
||||
// 底部分页
|
||||
@@ -1055,12 +1191,41 @@ ${escapeHtml(diff.test_value)}</pre>
|
||||
}
|
||||
|
||||
differencesContainer.innerHTML = html;
|
||||
|
||||
// 初始化全部展开/收起按钮的状态
|
||||
const toggleButton = document.getElementById('toggleAllText');
|
||||
if (toggleButton) {
|
||||
// 由于默认展开,按钮文本应该是"全部收起"
|
||||
toggleButton.innerHTML = '全部收起';
|
||||
}
|
||||
|
||||
// 恢复搜索框的焦点和光标位置
|
||||
if (currentSearchValue) {
|
||||
const searchInput = document.getElementById('differenceSearch');
|
||||
if (searchInput) {
|
||||
searchInput.focus();
|
||||
searchInput.setSelectionRange(searchInput.value.length, searchInput.value.length);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// HTML转义函数,防止XSS
|
||||
function escapeHtml(text) {
|
||||
// 处理undefined和null值
|
||||
if (text === undefined || text === null) {
|
||||
return '<span class="text-muted">无数据</span>';
|
||||
}
|
||||
|
||||
// 确保是字符串
|
||||
const str = String(text);
|
||||
|
||||
// 如果是空字符串
|
||||
if (str === '') {
|
||||
return '<span class="text-muted">空值</span>';
|
||||
}
|
||||
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
@@ -1068,8 +1233,73 @@ function escapeHtml(text) {
|
||||
function displayIdenticalResults() {
|
||||
const identicalContainer = document.getElementById('identical-results');
|
||||
|
||||
// 保存当前搜索框的值
|
||||
const currentSearchValue = document.getElementById('identicalSearch')?.value || '';
|
||||
|
||||
if (!filteredIdenticalResults.length) {
|
||||
identicalContainer.innerHTML = '<p class="text-muted"><i class="fas fa-info-circle"></i> 没有完全相同的记录</p>';
|
||||
// 显示搜索和控制界面,即使没有结果
|
||||
let html = `
|
||||
<!-- 分页控制 -->
|
||||
<div class="row mb-3">
|
||||
<div class="col-md-6">
|
||||
<div class="d-flex align-items-center">
|
||||
<label class="form-label me-2 mb-0">每页显示:</label>
|
||||
<select class="form-select form-select-sm me-2" style="width: auto;" onchange="changePageSize(this.value)">
|
||||
<option value="5" ${identicalPageSize == 10 ? 'selected' : ''}>10条</option>
|
||||
<option value="10" ${identicalPageSize == 50 ? 'selected' : ''}>50条</option>
|
||||
<option value="20" ${identicalPageSize == 100 ? 'selected' : ''}>100条</option>
|
||||
<option value="50" ${identicalPageSize == 200 ? 'selected' : ''}>200条</option>
|
||||
<option value="100" ${identicalPageSize == 500 ? 'selected' : ''}>500条</option>
|
||||
<option value="custom">自定义</option>
|
||||
</select>
|
||||
<input type="number" class="form-control form-control-sm me-2" style="width: 80px; display: none;"
|
||||
id="customPageSize" placeholder="数量" min="1" max="1000"
|
||||
onchange="setCustomPageSize(this.value)" onkeypress="handleCustomPageSizeEnter(event)">
|
||||
<span class="ms-2 text-muted">共 0 条记录</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6">
|
||||
<div class="d-flex justify-content-end align-items-center">
|
||||
<!-- 无分页控制 -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 搜索框 -->
|
||||
<div class="mb-3">
|
||||
<div class="input-group">
|
||||
<span class="input-group-text"><i class="fas fa-search"></i></span>
|
||||
<input type="text" class="form-control" placeholder="搜索主键或字段内容..."
|
||||
onkeypress="if(event.key === 'Enter') searchIdenticalResults(this.value)"
|
||||
id="identicalSearch"
|
||||
value="${currentIdenticalSearchTerm || ''}">
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="searchIdenticalResults(document.getElementById('identicalSearch').value)">
|
||||
<i class="fas fa-search"></i> 搜索
|
||||
</button>
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="clearIdenticalSearch()">
|
||||
<i class="fas fa-times"></i> 清除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 无结果提示 -->
|
||||
<div class="alert alert-info text-center">
|
||||
<i class="fas fa-search"></i>
|
||||
${(currentIdenticalSearchTerm || currentSearchValue) ? `没有找到包含"${escapeHtml(currentIdenticalSearchTerm || currentSearchValue)}"的相同记录` : '没有完全相同的记录'}
|
||||
${(currentIdenticalSearchTerm || currentSearchValue) ? '<br><small class="text-muted">请尝试其他搜索条件或点击"清除"按钮查看所有结果</small>' : ''}
|
||||
</div>
|
||||
`;
|
||||
|
||||
identicalContainer.innerHTML = html;
|
||||
|
||||
// 恢复搜索框的焦点和光标位置
|
||||
if (currentSearchValue) {
|
||||
const searchInput = document.getElementById('identicalSearch');
|
||||
if (searchInput) {
|
||||
searchInput.focus();
|
||||
searchInput.setSelectionRange(searchInput.value.length, searchInput.value.length);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1111,7 +1341,15 @@ function displayIdenticalResults() {
|
||||
<div class="input-group">
|
||||
<span class="input-group-text"><i class="fas fa-search"></i></span>
|
||||
<input type="text" class="form-control" placeholder="搜索主键或字段内容..."
|
||||
onkeyup="searchIdenticalResults(this.value)" id="identicalSearch">
|
||||
onkeypress="if(event.key === 'Enter') searchIdenticalResults(this.value)"
|
||||
id="identicalSearch"
|
||||
value="${currentIdenticalSearchTerm || ''}">
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="searchIdenticalResults(document.getElementById('identicalSearch').value)">
|
||||
<i class="fas fa-search"></i> 搜索
|
||||
</button>
|
||||
<button class="btn btn-outline-secondary" type="button" onclick="clearIdenticalSearch()">
|
||||
<i class="fas fa-times"></i> 清除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
@@ -1128,10 +1366,6 @@ function displayIdenticalResults() {
|
||||
<span class="badge bg-success ms-2">完全匹配</span>
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<button class="btn btn-sm btn-outline-secondary me-2" type="button"
|
||||
data-bs-toggle="collapse" data-bs-target="#collapse${globalIndex}">
|
||||
<i class="fas fa-eye"></i> 查看详情
|
||||
</button>
|
||||
<button class="btn btn-sm btn-outline-info" onclick='showRawData(${JSON.stringify(JSON.stringify(result.key))})'>
|
||||
<i class="fas fa-code"></i> 原生数据
|
||||
</button>
|
||||
@@ -1139,7 +1373,7 @@ function displayIdenticalResults() {
|
||||
</div>
|
||||
<p class="mb-0 mt-2"><strong>主键:</strong> ${formatCompositeKey(result.key)}</p>
|
||||
</div>
|
||||
<div class="collapse" id="collapse${globalIndex}">
|
||||
<div class="collapse show" id="collapse${globalIndex}">
|
||||
<div class="card-body">
|
||||
`;
|
||||
|
||||
@@ -1219,6 +1453,15 @@ ${escapeHtml(String(testValue))}</pre>
|
||||
}
|
||||
|
||||
identicalContainer.innerHTML = html;
|
||||
|
||||
// 恢复搜索框的焦点和光标位置
|
||||
if (currentSearchValue) {
|
||||
const searchInput = document.getElementById('identicalSearch');
|
||||
if (searchInput) {
|
||||
searchInput.focus();
|
||||
searchInput.setSelectionRange(searchInput.value.length, searchInput.value.length);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 生成分页导航
|
||||
@@ -1334,9 +1577,14 @@ function handleCustomPageSizeEnter(event) {
|
||||
}
|
||||
|
||||
// 搜索相同结果
|
||||
let currentIdenticalSearchTerm = '';
|
||||
|
||||
function searchIdenticalResults(searchTerm) {
|
||||
if (!currentResults) return;
|
||||
|
||||
// 保存搜索词
|
||||
currentIdenticalSearchTerm = searchTerm;
|
||||
|
||||
if (!searchTerm.trim()) {
|
||||
filteredIdenticalResults = currentResults.identical_results;
|
||||
} else {
|
||||
@@ -1361,6 +1609,15 @@ function searchIdenticalResults(searchTerm) {
|
||||
displayIdenticalResults();
|
||||
}
|
||||
|
||||
// 清除相同记录搜索
|
||||
function clearIdenticalSearch() {
|
||||
currentIdenticalSearchTerm = '';
|
||||
document.getElementById('identicalSearch').value = '';
|
||||
filteredIdenticalResults = currentResults.identical_results;
|
||||
currentIdenticalPage = 1;
|
||||
displayIdenticalResults();
|
||||
}
|
||||
|
||||
// 生成差异分页导航
|
||||
function generateDifferencePagination(currentPage, totalPages) {
|
||||
if (totalPages <= 1) return '';
|
||||
@@ -1474,14 +1731,28 @@ function handleCustomDiffPageSizeEnter(event) {
|
||||
}
|
||||
|
||||
// 搜索差异结果
|
||||
let currentSearchTerm = '';
|
||||
|
||||
function searchDifferenceResults(searchTerm) {
|
||||
if (!currentResults) return;
|
||||
|
||||
// 保存搜索词
|
||||
currentSearchTerm = searchTerm;
|
||||
|
||||
if (!searchTerm.trim()) {
|
||||
filteredDifferenceResults = currentResults.differences;
|
||||
// 如果有字段筛选,应用字段筛选;否则显示全部
|
||||
if (currentFieldFilter) {
|
||||
filteredDifferenceResults = currentResults.differences.filter(diff => diff.field === currentFieldFilter);
|
||||
} else {
|
||||
filteredDifferenceResults = currentResults.differences;
|
||||
}
|
||||
} else {
|
||||
const term = searchTerm.toLowerCase();
|
||||
filteredDifferenceResults = currentResults.differences.filter(diff => {
|
||||
let baseResults = currentFieldFilter ?
|
||||
currentResults.differences.filter(diff => diff.field === currentFieldFilter) :
|
||||
currentResults.differences;
|
||||
|
||||
filteredDifferenceResults = baseResults.filter(diff => {
|
||||
// 搜索主键
|
||||
const keyStr = JSON.stringify(diff.key).toLowerCase();
|
||||
if (keyStr.includes(term)) return true;
|
||||
@@ -1504,6 +1775,22 @@ function searchDifferenceResults(searchTerm) {
|
||||
displayDifferences();
|
||||
}
|
||||
|
||||
// 清除差异搜索
|
||||
function clearDifferenceSearch() {
|
||||
currentSearchTerm = '';
|
||||
document.getElementById('differenceSearch').value = '';
|
||||
|
||||
// 重新应用字段筛选
|
||||
if (currentFieldFilter) {
|
||||
filteredDifferenceResults = currentResults.differences.filter(diff => diff.field === currentFieldFilter);
|
||||
} else {
|
||||
filteredDifferenceResults = currentResults.differences;
|
||||
}
|
||||
|
||||
currentDifferencePage = 1;
|
||||
displayDifferences();
|
||||
}
|
||||
|
||||
// 复制到剪贴板
|
||||
function copyToClipboard(text, button) {
|
||||
navigator.clipboard.writeText(text).then(function() {
|
||||
@@ -1551,6 +1838,7 @@ function showRawData(keyStr) {
|
||||
try {
|
||||
// 解析key
|
||||
const key = JSON.parse(keyStr);
|
||||
console.log('查找原始数据,key:', key);
|
||||
|
||||
// 在原生数据中查找对应的记录
|
||||
let proData = null;
|
||||
@@ -1562,15 +1850,40 @@ function showRawData(keyStr) {
|
||||
// 支持复合主键比较
|
||||
if (typeof key === 'object' && !Array.isArray(key)) {
|
||||
// 复合主键情况:比较所有主键字段
|
||||
return Object.keys(key).every(keyField =>
|
||||
JSON.stringify(item[keyField]) === JSON.stringify(key[keyField])
|
||||
);
|
||||
const matches = Object.keys(key).every(keyField => {
|
||||
const itemValue = item[keyField];
|
||||
const keyValue = key[keyField];
|
||||
|
||||
// 如果都是undefined或null,认为匹配
|
||||
if (itemValue == null && keyValue == null) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// 转换为字符串进行比较
|
||||
const itemStr = String(itemValue);
|
||||
const keyStr = String(keyValue);
|
||||
|
||||
// 直接比较字符串
|
||||
return itemStr === keyStr;
|
||||
});
|
||||
|
||||
if (matches) {
|
||||
console.log('找到生产环境数据:', item);
|
||||
}
|
||||
return matches;
|
||||
} else {
|
||||
// 单主键情况:保持原有逻辑
|
||||
const keyField = Object.keys(key)[0];
|
||||
return JSON.stringify(item[keyField]) === JSON.stringify(key[keyField]);
|
||||
// 单主键情况(兼容旧代码)
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
// 如果没找到,输出调试信息
|
||||
if (!proData) {
|
||||
console.log('未找到生产数据,查找的key:', key);
|
||||
console.log('可用的数据:', currentResults.raw_pro_data.map(item => ({
|
||||
statusid: item.statusid
|
||||
})));
|
||||
}
|
||||
}
|
||||
|
||||
// 查找测试环境数据
|
||||
@@ -1579,17 +1892,44 @@ function showRawData(keyStr) {
|
||||
// 支持复合主键比较
|
||||
if (typeof key === 'object' && !Array.isArray(key)) {
|
||||
// 复合主键情况:比较所有主键字段
|
||||
return Object.keys(key).every(keyField =>
|
||||
JSON.stringify(item[keyField]) === JSON.stringify(key[keyField])
|
||||
);
|
||||
const matches = Object.keys(key).every(keyField => {
|
||||
const itemValue = item[keyField];
|
||||
const keyValue = key[keyField];
|
||||
|
||||
// 如果都是undefined或null,认为匹配
|
||||
if (itemValue == null && keyValue == null) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// 转换为字符串进行比较
|
||||
const itemStr = String(itemValue);
|
||||
const keyStr = String(keyValue);
|
||||
|
||||
// 直接比较字符串
|
||||
return itemStr === keyStr;
|
||||
});
|
||||
|
||||
if (matches) {
|
||||
console.log('找到测试环境数据:', item);
|
||||
}
|
||||
return matches;
|
||||
} else {
|
||||
// 单主键情况:保持原有逻辑
|
||||
const keyField = Object.keys(key)[0];
|
||||
return JSON.stringify(item[keyField]) === JSON.stringify(key[keyField]);
|
||||
// 单主键情况(兼容旧代码)
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
// 如果没找到,输出调试信息
|
||||
if (!testData) {
|
||||
console.log('未找到测试数据,查找的key:', key);
|
||||
console.log('可用的数据:', currentResults.raw_test_data.map(item => ({
|
||||
statusid: item.statusid
|
||||
})));
|
||||
}
|
||||
}
|
||||
|
||||
console.log('查找结果 - 生产数据:', proData, '测试数据:', testData);
|
||||
|
||||
// 创建模态框内容
|
||||
const modalContent = `
|
||||
<div class="modal fade" id="rawDataModal" tabindex="-1">
|
||||
@@ -1713,14 +2053,47 @@ function showRawData(keyStr) {
|
||||
|
||||
// 延迟渲染对比视图和树形视图
|
||||
setTimeout(() => {
|
||||
renderDiffView(proData, testData);
|
||||
renderTreeView('proTreeView', proData);
|
||||
renderTreeView('testTreeView', testData);
|
||||
console.log('开始渲染视图,生产数据:', proData, '测试数据:', testData);
|
||||
|
||||
// 渲染对比视图
|
||||
try {
|
||||
renderDiffView(proData, testData);
|
||||
console.log('对比视图渲染完成');
|
||||
} catch (e) {
|
||||
console.error('对比视图渲染失败:', e);
|
||||
}
|
||||
|
||||
// 渲染树形视图
|
||||
try {
|
||||
renderTreeView('proTreeView', proData);
|
||||
renderTreeView('testTreeView', testData);
|
||||
console.log('树形视图渲染完成');
|
||||
} catch (e) {
|
||||
console.error('树形视图渲染失败:', e);
|
||||
}
|
||||
|
||||
// 为格式化视图添加同步滚动
|
||||
setupSyncScroll('formatted');
|
||||
// 为树形视图添加同步滚动
|
||||
setupSyncScroll('tree');
|
||||
|
||||
// 添加标签页切换事件监听器,确保切换时重新渲染
|
||||
const tabButtons = document.querySelectorAll('#rawDataTabs button[data-bs-toggle="tab"]');
|
||||
tabButtons.forEach(button => {
|
||||
button.addEventListener('shown.bs.tab', (event) => {
|
||||
const targetId = event.target.getAttribute('data-bs-target');
|
||||
console.log('切换到标签页:', targetId);
|
||||
|
||||
if (targetId === '#diff') {
|
||||
// 重新渲染对比视图
|
||||
renderDiffView(proData, testData);
|
||||
} else if (targetId === '#tree') {
|
||||
// 重新渲染树形视图
|
||||
renderTreeView('proTreeView', proData);
|
||||
renderTreeView('testTreeView', testData);
|
||||
}
|
||||
});
|
||||
});
|
||||
}, 100);
|
||||
|
||||
} catch (error) {
|
||||
@@ -1746,6 +2119,101 @@ function copyRawData() {
|
||||
}
|
||||
}
|
||||
|
||||
// 复制差异主键
|
||||
function copyDifferenceKeys() {
|
||||
if (!filteredDifferenceResults || filteredDifferenceResults.length === 0) {
|
||||
showAlert('warning', '无差异数据可复制');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// 收集所有差异记录的主键
|
||||
const differenceKeys = [];
|
||||
const uniqueKeys = new Set();
|
||||
|
||||
filteredDifferenceResults.forEach(diff => {
|
||||
if (diff.key) {
|
||||
let keyText = '';
|
||||
|
||||
if (typeof diff.key === 'object' && !Array.isArray(diff.key)) {
|
||||
// 复合主键:转换为逗号分隔格式
|
||||
keyText = Object.values(diff.key).join(',');
|
||||
} else {
|
||||
// 单主键或其他格式
|
||||
keyText = String(diff.key);
|
||||
}
|
||||
|
||||
// 避免重复主键
|
||||
if (!uniqueKeys.has(keyText)) {
|
||||
uniqueKeys.add(keyText);
|
||||
differenceKeys.push(keyText);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (differenceKeys.length === 0) {
|
||||
showAlert('warning', '未找到有效的主键数据');
|
||||
return;
|
||||
}
|
||||
|
||||
// 将主键列表转换为文本格式(每行一个)
|
||||
const keyText = differenceKeys.join('\n');
|
||||
|
||||
// 复制到剪贴板
|
||||
navigator.clipboard.writeText(keyText).then(() => {
|
||||
showAlert('success', `已复制 ${differenceKeys.length} 个差异主键到剪贴板`);
|
||||
}).catch(err => {
|
||||
console.error('复制失败:', err);
|
||||
showAlert('danger', '复制失败,请手动选择复制');
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('处理差异主键失败:', error);
|
||||
showAlert('danger', '处理差异主键失败: ' + error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// 切换所有差异详情的展开/收起状态
|
||||
function toggleAllDifferenceCollapse() {
|
||||
// 获取当前页面的所有差异展开区域
|
||||
const collapseElements = document.querySelectorAll('#differences [id^="keyGroup"]');
|
||||
const toggleButton = document.getElementById('toggleAllText');
|
||||
|
||||
if (collapseElements.length === 0) {
|
||||
showAlert('warning', '没有找到差异详情');
|
||||
return;
|
||||
}
|
||||
|
||||
// 检查当前状态 - 如果大部分是展开的,就全部收起;否则全部展开
|
||||
let expandedCount = 0;
|
||||
collapseElements.forEach(element => {
|
||||
if (element.classList.contains('show')) {
|
||||
expandedCount++;
|
||||
}
|
||||
});
|
||||
|
||||
const shouldExpand = expandedCount < collapseElements.length / 2;
|
||||
|
||||
collapseElements.forEach(element => {
|
||||
if (shouldExpand) {
|
||||
// 展开
|
||||
element.classList.add('show');
|
||||
} else {
|
||||
// 收起
|
||||
element.classList.remove('show');
|
||||
}
|
||||
});
|
||||
|
||||
// 更新按钮文本
|
||||
if (shouldExpand) {
|
||||
toggleButton.innerHTML = '全部收起';
|
||||
showAlert('success', `已展开 ${collapseElements.length} 个差异详情`);
|
||||
} else {
|
||||
toggleButton.innerHTML = '全部展开';
|
||||
showAlert('success', `已收起 ${collapseElements.length} 个差异详情`);
|
||||
}
|
||||
}
|
||||
|
||||
// 显示差异数据的原生数据
|
||||
function showDifferenceRawData(keyStr) {
|
||||
showRawData(keyStr); // 复用相同的原生数据显示逻辑
|
||||
@@ -1755,6 +2223,12 @@ function showDifferenceRawData(keyStr) {
|
||||
function renderDiffView(proData, testData) {
|
||||
const diffViewContainer = document.getElementById('diffView');
|
||||
|
||||
// 确保容器存在
|
||||
if (!diffViewContainer) {
|
||||
console.error('对比视图容器不存在');
|
||||
return;
|
||||
}
|
||||
|
||||
if (!proData && !testData) {
|
||||
diffViewContainer.innerHTML = '<p class="text-muted text-center">无数据可对比</p>';
|
||||
return;
|
||||
@@ -2136,6 +2610,12 @@ function normalizeValue(value) {
|
||||
function renderTreeView(containerId, data) {
|
||||
const container = document.getElementById(containerId);
|
||||
|
||||
// 确保容器存在
|
||||
if (!container) {
|
||||
console.error('树形视图容器不存在:', containerId);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!data) {
|
||||
container.innerHTML = '<p class="text-muted">无数据</p>';
|
||||
return;
|
||||
@@ -3101,14 +3581,14 @@ function showImportDialog(env) {
|
||||
<p class="text-muted mb-3">请粘贴配置数据,支持以下格式:</p>
|
||||
<div class="mb-3">
|
||||
<small class="text-muted">示例格式:</small>
|
||||
<pre class="bg-light p-2 rounded small">clusterName: "Hot Cluster"
|
||||
clusterNodes: "10.20.2.22,10.20.2.23"
|
||||
<pre class="bg-light p-2 rounded small">clusterName: "Example Cluster"
|
||||
clusterNodes: "127.0.0.1,127.0.0.2"
|
||||
port: 9042
|
||||
datacenter: "cs01"
|
||||
username: "cbase"
|
||||
password: "antducbaseadmin@2022"
|
||||
keyspace: "yuqing_skinny"
|
||||
table: "status_test"</pre>
|
||||
datacenter: "dc1"
|
||||
username: "cassandra"
|
||||
password: "example_password"
|
||||
keyspace: "example_keyspace"
|
||||
table: "example_table"</pre>
|
||||
</div>
|
||||
<div class="mb-3">
|
||||
<label class="form-label">配置数据:</label>
|
||||
@@ -3954,10 +4434,6 @@ function renderRawDataContent() {
|
||||
<span class="badge ${envBadgeClass} ms-2">${item.displayName}</span>
|
||||
</div>
|
||||
<div class="col-auto">
|
||||
<button class="btn btn-sm btn-outline-secondary me-2" type="button"
|
||||
data-bs-toggle="collapse" data-bs-target="#rawCollapse${globalIndex}">
|
||||
<i class="fas fa-eye"></i> 查看详情
|
||||
</button>
|
||||
<button class="btn btn-sm btn-outline-info" onclick='copyRawRecord(${JSON.stringify(JSON.stringify(item.data))})'>
|
||||
<i class="fas fa-copy"></i> 复制
|
||||
</button>
|
||||
@@ -3965,7 +4441,7 @@ function renderRawDataContent() {
|
||||
</div>
|
||||
<p class="mb-0 mt-2"><strong>主键:</strong> ${keyValue}</p>
|
||||
</div>
|
||||
<div class="collapse" id="rawCollapse${globalIndex}">
|
||||
<div class="collapse show" id="rawCollapse${globalIndex}">
|
||||
<div class="card-body">
|
||||
<pre class="bg-light p-3 rounded" style="max-height: 500px; overflow-y: auto; font-size: 0.9em; line-height: 1.4;">${escapeHtml(jsonData)}</pre>
|
||||
</div>
|
||||
@@ -4271,4 +4747,69 @@ async function batchDeleteQueryHistory() {
|
||||
console.error('批量删除Cassandra查询历史记录失败:', error);
|
||||
showAlert('danger', `批量删除失败: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// 按主键分组差异数据
|
||||
function groupDifferencesByKey(differences) {
|
||||
const grouped = {};
|
||||
differences.forEach(diff => {
|
||||
const keyStr = typeof diff.key === 'object' ? JSON.stringify(diff.key) : diff.key;
|
||||
if (!grouped[keyStr]) {
|
||||
grouped[keyStr] = [];
|
||||
}
|
||||
grouped[keyStr].push(diff);
|
||||
});
|
||||
return grouped;
|
||||
}
|
||||
|
||||
// 按字段筛选差异
|
||||
let currentFieldFilter = '';
|
||||
|
||||
function filterByField(field) {
|
||||
currentFieldFilter = field;
|
||||
|
||||
// 更新按钮状态
|
||||
const buttons = document.querySelectorAll('.btn-group button');
|
||||
buttons.forEach(btn => {
|
||||
btn.classList.remove('active', 'btn-outline-primary');
|
||||
btn.classList.add('btn-outline-secondary');
|
||||
});
|
||||
|
||||
// 设置当前选中的按钮
|
||||
if (field === '') {
|
||||
buttons[0].classList.remove('btn-outline-secondary');
|
||||
buttons[0].classList.add('btn-outline-primary', 'active');
|
||||
} else {
|
||||
buttons.forEach(btn => {
|
||||
if (btn.textContent.includes(field + ' (')) {
|
||||
btn.classList.remove('btn-outline-secondary');
|
||||
btn.classList.add('btn-outline-primary', 'active');
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 筛选差异数据(考虑搜索条件)
|
||||
let baseResults = field === '' ?
|
||||
currentResults.differences :
|
||||
currentResults.differences.filter(diff => diff.field === field);
|
||||
|
||||
// 如果有搜索条件,应用搜索过滤
|
||||
if (currentSearchTerm && currentSearchTerm.trim()) {
|
||||
const term = currentSearchTerm.toLowerCase();
|
||||
filteredDifferenceResults = baseResults.filter(diff => {
|
||||
const keyStr = JSON.stringify(diff.key).toLowerCase();
|
||||
if (keyStr.includes(term)) return true;
|
||||
if (diff.field && diff.field.toLowerCase().includes(term)) return true;
|
||||
if (diff.pro_value && String(diff.pro_value).toLowerCase().includes(term)) return true;
|
||||
if (diff.test_value && String(diff.test_value).toLowerCase().includes(term)) return true;
|
||||
if (diff.message && diff.message.toLowerCase().includes(term)) return true;
|
||||
return false;
|
||||
});
|
||||
} else {
|
||||
filteredDifferenceResults = baseResults;
|
||||
}
|
||||
|
||||
// 重置页码并重新显示
|
||||
currentDifferencePage = 1;
|
||||
displayDifferences();
|
||||
}
|
@@ -900,9 +900,9 @@ async function loadRedisConfigGroupsForManagement() {
|
||||
const response = await fetch('/api/redis/config-groups');
|
||||
const result = await response.json();
|
||||
|
||||
const container = document.getElementById('redisConfigGroupList');
|
||||
const container = document.getElementById('redisConfigGroupsList');
|
||||
if (!container) {
|
||||
console.error('Redis配置组列表容器未找到: redisConfigGroupList');
|
||||
console.error('Redis配置组列表容器未找到: redisConfigGroupsList');
|
||||
showAlert('Redis配置组列表容器未找到,请检查页面结构', 'danger');
|
||||
return;
|
||||
}
|
||||
@@ -938,7 +938,7 @@ async function loadRedisConfigGroupsForManagement() {
|
||||
console.log('没有找到Redis配置组数据');
|
||||
}
|
||||
} catch (error) {
|
||||
const container = document.getElementById('redisConfigGroupList');
|
||||
const container = document.getElementById('redisConfigGroupsList');
|
||||
if (container) {
|
||||
container.innerHTML = '<div class="alert alert-danger">加载失败: ' + error.message + '</div>';
|
||||
}
|
||||
@@ -1665,30 +1665,30 @@ function updateConfigTemplate() {
|
||||
if (format === 'yaml') {
|
||||
templateElement.textContent = `# Redis集群配置示例 (YAML格式)
|
||||
cluster1:
|
||||
clusterName: "redis-production"
|
||||
clusterAddress: "10.20.2.109:6470"
|
||||
clusterName: "redis-example1"
|
||||
clusterAddress: "127.0.0.1:6379"
|
||||
clusterPassword: ""
|
||||
cachePrefix: "message.status.Reader."
|
||||
cacheTtl: 2000
|
||||
async: true
|
||||
nodes:
|
||||
- host: "10.20.2.109"
|
||||
port: 6470
|
||||
- host: "10.20.2.110"
|
||||
port: 6470
|
||||
- host: "127.0.0.1"
|
||||
port: 6379
|
||||
- host: "127.0.0.2"
|
||||
port: 6379
|
||||
|
||||
cluster2:
|
||||
clusterName: "redis-test"
|
||||
clusterAddress: "10.20.2.109:6471"
|
||||
clusterName: "redis-example2"
|
||||
clusterAddress: "127.0.0.1:6380"
|
||||
clusterPassword: ""
|
||||
cachePrefix: "message.status.Reader."
|
||||
cacheTtl: 2000
|
||||
async: true
|
||||
nodes:
|
||||
- host: "10.20.2.109"
|
||||
port: 6471
|
||||
- host: "10.20.2.110"
|
||||
port: 6471
|
||||
- host: "127.0.0.1"
|
||||
port: 6380
|
||||
- host: "127.0.0.2"
|
||||
port: 6380
|
||||
|
||||
queryOptions:
|
||||
mode: "random" # random 或 specified
|
||||
@@ -1697,32 +1697,32 @@ queryOptions:
|
||||
sourceCluster: "cluster2"
|
||||
# 指定Key模式下的键值列表
|
||||
keys:
|
||||
- "user:1001"
|
||||
- "user:1002"`;
|
||||
- "user:example1"
|
||||
- "user:example2"`;
|
||||
} else {
|
||||
templateElement.textContent = `{
|
||||
"cluster1": {
|
||||
"clusterName": "redis-production",
|
||||
"clusterAddress": "10.20.2.109:6470",
|
||||
"clusterName": "redis-example1",
|
||||
"clusterAddress": "127.0.0.1:6379",
|
||||
"clusterPassword": "",
|
||||
"cachePrefix": "message.status.Reader.",
|
||||
"cacheTtl": 2000,
|
||||
"async": true,
|
||||
"nodes": [
|
||||
{"host": "10.20.2.109", "port": 6470},
|
||||
{"host": "10.20.2.110", "port": 6470}
|
||||
{"host": "127.0.0.1", "port": 6379},
|
||||
{"host": "127.0.0.2", "port": 6379}
|
||||
]
|
||||
},
|
||||
"cluster2": {
|
||||
"clusterName": "redis-test",
|
||||
"clusterAddress": "10.20.2.109:6471",
|
||||
"clusterName": "redis-example2",
|
||||
"clusterAddress": "127.0.0.1:6380",
|
||||
"clusterPassword": "",
|
||||
"cachePrefix": "message.status.Reader.",
|
||||
"cacheTtl": 2000,
|
||||
"async": true,
|
||||
"nodes": [
|
||||
{"host": "10.20.2.109", "port": 6471},
|
||||
{"host": "10.20.2.110", "port": 6471}
|
||||
{"host": "127.0.0.1", "port": 6380},
|
||||
{"host": "127.0.0.2", "port": 6380}
|
||||
]
|
||||
},
|
||||
"queryOptions": {
|
||||
@@ -1730,7 +1730,7 @@ queryOptions:
|
||||
"count": 100,
|
||||
"pattern": "*",
|
||||
"sourceCluster": "cluster2",
|
||||
"keys": ["user:1001", "user:1002"]
|
||||
"keys": ["user:example1", "user:example2"]
|
||||
}
|
||||
}`;
|
||||
}
|
||||
@@ -2195,80 +2195,6 @@ async function saveRedisConfigGroup() {
|
||||
}
|
||||
}
|
||||
|
||||
// 显示管理Redis配置组对话框
|
||||
function showManageRedisConfigDialog() {
|
||||
loadRedisConfigGroupsForManagement();
|
||||
new bootstrap.Modal(document.getElementById('manageRedisConfigModal')).show();
|
||||
}
|
||||
|
||||
// 为管理界面加载Redis配置组
|
||||
async function loadRedisConfigGroupsForManagement() {
|
||||
try {
|
||||
const response = await fetch('/api/redis/config-groups');
|
||||
const result = await response.json();
|
||||
|
||||
if (result.success) {
|
||||
displayRedisConfigGroupsForManagement(result.data);
|
||||
} else {
|
||||
showAlert(`加载配置组失败: ${result.error}`, 'danger');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('加载Redis配置组失败:', error);
|
||||
showAlert(`加载失败: ${error.message}`, 'danger');
|
||||
}
|
||||
}
|
||||
|
||||
// 显示Redis配置组管理列表
|
||||
function displayRedisConfigGroupsForManagement(configGroups) {
|
||||
const container = document.getElementById('redisConfigGroupsList');
|
||||
|
||||
if (!configGroups || configGroups.length === 0) {
|
||||
container.innerHTML = '<div class="text-center text-muted py-4">暂无保存的配置组</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
let html = '';
|
||||
configGroups.forEach(config => {
|
||||
html += `
|
||||
<div class="card mb-3">
|
||||
<div class="card-body">
|
||||
<div class="d-flex justify-content-between align-items-start">
|
||||
<div class="flex-grow-1">
|
||||
<h6 class="card-title mb-2">
|
||||
<i class="fas fa-cog redis-logo me-2"></i>${config.name}
|
||||
</h6>
|
||||
<p class="card-text text-muted small mb-2">${config.description || '无描述'}</p>
|
||||
<div class="small text-muted">
|
||||
<i class="fas fa-clock me-1"></i>创建时间: ${config.created_at}
|
||||
${config.updated_at !== config.created_at ? `<br><i class="fas fa-edit me-1"></i>更新时间: ${config.updated_at}` : ''}
|
||||
</div>
|
||||
</div>
|
||||
<div class="btn-group-vertical btn-group-sm">
|
||||
<button class="btn btn-outline-primary btn-sm mb-1" onclick="loadRedisConfigGroup(${config.id})" title="加载配置">
|
||||
<i class="fas fa-download"></i> 加载
|
||||
</button>
|
||||
<button class="btn btn-outline-success btn-sm mb-1" onclick="exportRedisConfigGroup(${config.id}, '${config.name}')" title="导出YAML">
|
||||
<i class="fas fa-file-export"></i> 导出
|
||||
</button>
|
||||
<button class="btn btn-outline-info btn-sm mb-1" onclick="copyRedisConfigGroup(${config.id}, '${config.name}')" title="复制配置">
|
||||
<i class="fas fa-copy"></i> 复制
|
||||
</button>
|
||||
<button class="btn btn-outline-warning btn-sm mb-1" onclick="testRedisConfigConnection(${config.id})" title="测试连接">
|
||||
<i class="fas fa-plug"></i> 测试
|
||||
</button>
|
||||
<button class="btn btn-outline-danger btn-sm" onclick="deleteRedisConfigGroup(${config.id}, '${config.name}')" title="删除配置">
|
||||
<i class="fas fa-trash"></i> 删除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
});
|
||||
|
||||
container.innerHTML = html;
|
||||
}
|
||||
|
||||
// 刷新Redis配置组列表
|
||||
function refreshRedisConfigGroups() {
|
||||
loadRedisConfigGroupsForManagement();
|
||||
|
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>数据库查询比对工具 - 支持分表查询</title>
|
||||
<title>Cassandra数据比对工具 - DataTools Pro</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
|
||||
<style>
|
||||
@@ -255,6 +255,16 @@
|
||||
border-color: #ffb3b3;
|
||||
color: #d63384;
|
||||
}
|
||||
|
||||
/* 面包屑导航样式 */
|
||||
.breadcrumb {
|
||||
background: none;
|
||||
padding: 0;
|
||||
}
|
||||
.breadcrumb-item a {
|
||||
color: #007bff;
|
||||
text-decoration: none;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -262,7 +272,7 @@
|
||||
<nav class="navbar navbar-expand-lg navbar-dark bg-primary">
|
||||
<div class="container">
|
||||
<a class="navbar-brand" href="/">
|
||||
<i class="fas fa-tools"></i> 大数据工具集合
|
||||
<i class="fas fa-database me-2"></i> DataTools Pro
|
||||
</a>
|
||||
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
@@ -270,10 +280,19 @@
|
||||
<div class="collapse navbar-collapse" id="navbarNav">
|
||||
<ul class="navbar-nav ms-auto">
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="/">首页</a>
|
||||
<a class="nav-link" href="/">
|
||||
<i class="fas fa-home"></i> 首页
|
||||
</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link active" href="/db-compare">单表查询</a>
|
||||
<a class="nav-link active" href="/cassandra-compare">
|
||||
<i class="fas fa-database"></i> Cassandra比对
|
||||
</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="/redis-compare">
|
||||
<i class="fab fa-redis"></i> Redis比对
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
@@ -281,10 +300,18 @@
|
||||
</nav>
|
||||
|
||||
<div class="container-fluid py-4">
|
||||
<!-- 面包屑导航 -->
|
||||
<nav aria-label="breadcrumb">
|
||||
<ol class="breadcrumb">
|
||||
<li class="breadcrumb-item"><a href="/">首页</a></li>
|
||||
<li class="breadcrumb-item active" aria-current="page">Cassandra数据比对工具</li>
|
||||
</ol>
|
||||
</nav>
|
||||
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
<h1 class="text-center mb-4">
|
||||
<i class="fas fa-database"></i> 数据库查询比对工具
|
||||
<i class="fas fa-database"></i> Cassandra数据比对工具
|
||||
<small class="text-muted d-block fs-6 mt-2">支持单表查询和分表查询两种模式</small>
|
||||
</h1>
|
||||
</div>
|
||||
@@ -448,7 +475,7 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-8">
|
||||
<label class="form-label">集群节点 (逗号分隔)</label>
|
||||
<input type="text" class="form-control form-control-sm" id="pro_hosts" placeholder="10.20.2.22,10.20.2.23">
|
||||
<input type="text" class="form-control form-control-sm" id="pro_hosts" placeholder="127.0.0.1,127.0.0.2">
|
||||
</div>
|
||||
<div class="col-4">
|
||||
<label class="form-label">端口</label>
|
||||
@@ -458,7 +485,7 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-6">
|
||||
<label class="form-label">用户名</label>
|
||||
<input type="text" class="form-control form-control-sm" id="pro_username" placeholder="cbase">
|
||||
<input type="text" class="form-control form-control-sm" id="pro_username" placeholder="username">
|
||||
</div>
|
||||
<div class="col-6">
|
||||
<label class="form-label">密码</label>
|
||||
@@ -468,11 +495,11 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-6">
|
||||
<label class="form-label">Keyspace</label>
|
||||
<input type="text" class="form-control form-control-sm" id="pro_keyspace" placeholder="yuqing_skinny">
|
||||
<input type="text" class="form-control form-control-sm" id="pro_keyspace" placeholder="keyspace">
|
||||
</div>
|
||||
<div class="col-6">
|
||||
<label class="form-label">表名</label>
|
||||
<input type="text" class="form-control form-control-sm" id="pro_table" placeholder="document">
|
||||
<input type="text" class="form-control form-control-sm" id="pro_table" placeholder="tablename">
|
||||
<small class="form-text text-muted" id="pro_table_hint">完整表名或基础表名(分表时)</small>
|
||||
</div>
|
||||
</div>
|
||||
@@ -501,7 +528,7 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-8">
|
||||
<label class="form-label">集群节点 (逗号分隔)</label>
|
||||
<input type="text" class="form-control form-control-sm" id="test_hosts" placeholder="10.20.2.22,10.20.2.23">
|
||||
<input type="text" class="form-control form-control-sm" id="test_hosts" placeholder="127.0.0.1,127.0.0.2">
|
||||
</div>
|
||||
<div class="col-4">
|
||||
<label class="form-label">端口</label>
|
||||
@@ -511,7 +538,7 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-6">
|
||||
<label class="form-label">用户名</label>
|
||||
<input type="text" class="form-control form-control-sm" id="test_username" placeholder="cbase">
|
||||
<input type="text" class="form-control form-control-sm" id="test_username" placeholder="username">
|
||||
</div>
|
||||
<div class="col-6">
|
||||
<label class="form-label">密码</label>
|
||||
@@ -521,11 +548,11 @@
|
||||
<div class="row mt-2">
|
||||
<div class="col-6">
|
||||
<label class="form-label">Keyspace</label>
|
||||
<input type="text" class="form-control form-control-sm" id="test_keyspace" placeholder="yuqing_skinny">
|
||||
<input type="text" class="form-control form-control-sm" id="test_keyspace" placeholder="keyspace">
|
||||
</div>
|
||||
<div class="col-6">
|
||||
<label class="form-label">表名</label>
|
||||
<input type="text" class="form-control form-control-sm" id="test_table" placeholder="document_test">
|
||||
<input type="text" class="form-control form-control-sm" id="test_table" placeholder="tablename">
|
||||
<small class="form-text text-muted" id="test_table_hint">完整表名或基础表名(分表时)</small>
|
||||
</div>
|
||||
</div>
|
||||
|
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>大数据工具集合</title>
|
||||
<title>DataTools Pro - 专业数据处理工具平台</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
|
||||
<style>
|
||||
@@ -152,7 +152,7 @@
|
||||
<nav class="navbar navbar-expand-lg navbar-dark" style="background: rgba(0,0,0,0.1);">
|
||||
<div class="container">
|
||||
<a class="navbar-brand" href="/">
|
||||
<i class="fas fa-tools"></i> 大数据工具集合
|
||||
<i class="fas fa-code-branch"></i> DataTools Pro
|
||||
</a>
|
||||
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
@@ -163,7 +163,10 @@
|
||||
<a class="nav-link active" href="/">首页</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="/db-compare">数据库比对</a>
|
||||
<a class="nav-link" href="/cassandra-compare">Cassandra比对</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="/redis-compare">Redis比对</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
@@ -174,11 +177,11 @@
|
||||
<div class="hero-section">
|
||||
<div class="container">
|
||||
<h1 class="hero-title">
|
||||
<i class="fas fa-database"></i> 大数据工具集合
|
||||
<i class="fas fa-rocket"></i> DataTools Pro
|
||||
</h1>
|
||||
<p class="hero-subtitle">
|
||||
专业的数据处理、分析和比对工具平台<br>
|
||||
提升数据工作效率,简化复杂操作
|
||||
企业级数据处理与比对工具平台<br>
|
||||
高效、精准、可视化的数据分析解决方案
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
@@ -189,20 +192,20 @@
|
||||
<div class="row">
|
||||
<div class="col-md-4">
|
||||
<div class="stat-item">
|
||||
<span class="stat-number">1</span>
|
||||
<span class="stat-label">可用工具</span>
|
||||
<span class="stat-number">2</span>
|
||||
<span class="stat-label">核心工具</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<div class="stat-item">
|
||||
<span class="stat-number">100%</span>
|
||||
<span class="stat-label">可视化操作</span>
|
||||
<span class="stat-number">∞</span>
|
||||
<span class="stat-label">数据处理能力</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-4">
|
||||
<div class="stat-item">
|
||||
<span class="stat-number">0</span>
|
||||
<span class="stat-label">学习成本</span>
|
||||
<span class="stat-number">24/7</span>
|
||||
<span class="stat-label">稳定运行</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -215,70 +218,72 @@
|
||||
<div class="row justify-content-center">
|
||||
<div class="col-lg-8">
|
||||
<div class="text-center mb-5">
|
||||
<h2 class="mb-3">可用工具</h2>
|
||||
<p class="text-muted">选择合适的工具来处理您的数据任务</p>
|
||||
<h2 class="mb-3">核心工具模块</h2>
|
||||
<p class="text-muted">选择适合的工具来解决您的数据处理挑战</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row">
|
||||
<!-- 数据库比对工具 -->
|
||||
<!-- Cassandra数据库比对工具 -->
|
||||
<div class="col-lg-6 col-md-12">
|
||||
<div class="tool-card">
|
||||
<div class="text-center">
|
||||
<div class="feature-badge">可用</div>
|
||||
<div class="feature-badge">生产就绪</div>
|
||||
<div class="tool-icon">
|
||||
<i class="fas fa-exchange-alt"></i>
|
||||
<i class="fas fa-database"></i>
|
||||
</div>
|
||||
<h3 class="tool-title">数据库查询比对工具</h3>
|
||||
<h3 class="tool-title">Cassandra数据比对工具</h3>
|
||||
<p class="tool-description">
|
||||
专业的Cassandra数据库比对工具,支持生产环境与测试环境数据差异分析,
|
||||
提供批量查询、字段级比对和详细统计报告。
|
||||
企业级Cassandra数据库比对平台,支持生产环境与测试环境数据差异分析,
|
||||
提供批量查询、分表查询、多主键查询和详细统计报告。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="tool-features">
|
||||
<h5><i class="fas fa-star text-warning"></i> 核心功能:</h5>
|
||||
<h5><i class="fas fa-star text-warning"></i> 核心特性:</h5>
|
||||
<ul>
|
||||
<li><i class="fas fa-check text-success"></i> 支持多环境数据库配置管理</li>
|
||||
<li><i class="fas fa-check text-success"></i> 批量Key查询和数据比对</li>
|
||||
<li><i class="fas fa-check text-success"></i> 自定义比较字段和排除字段</li>
|
||||
<li><i class="fas fa-check text-success"></i> 可视化差异展示和统计</li>
|
||||
<li><i class="fas fa-check text-success"></i> 配置和结果导出功能</li>
|
||||
<li><i class="fas fa-check text-success"></i> 多环境数据库配置管理</li>
|
||||
<li><i class="fas fa-check text-success"></i> 分表查询(TWCS时间分表)</li>
|
||||
<li><i class="fas fa-check text-success"></i> 多主键查询和复合主键支持</li>
|
||||
<li><i class="fas fa-check text-success"></i> 可视化差异展示和智能统计</li>
|
||||
<li><i class="fas fa-check text-success"></i> 查询历史管理和结果导出</li>
|
||||
<li><i class="fas fa-check text-success"></i> 详细的查询日志和性能监控</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div class="text-center">
|
||||
<a href="/db-compare" class="tool-btn">
|
||||
<a href="/cassandra-compare" class="tool-btn">
|
||||
<i class="fas fa-rocket"></i> 立即使用
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Redis比对工具 -->
|
||||
<!-- Redis集群比对工具 -->
|
||||
<div class="col-lg-6 col-md-12">
|
||||
<div class="tool-card">
|
||||
<div class="text-center">
|
||||
<div class="feature-badge">可用</div>
|
||||
<div class="feature-badge">生产就绪</div>
|
||||
<div class="tool-icon">
|
||||
<i class="fab fa-redis"></i>
|
||||
</div>
|
||||
<h3 class="tool-title">Redis集群比对工具</h3>
|
||||
<p class="tool-description">
|
||||
专业的Redis集群数据比对工具,支持生产环境与测试环境Redis数据差异分析,
|
||||
提供随机采样和指定Key查询两种模式。
|
||||
专业的Redis集群数据比对平台,支持多种数据类型比对分析,
|
||||
提供随机采样、指定Key查询和全面的性能监控。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="tool-features">
|
||||
<h5><i class="fas fa-star text-warning"></i> 核心功能:</h5>
|
||||
<h5><i class="fas fa-star text-warning"></i> 核心特性:</h5>
|
||||
<ul>
|
||||
<li><i class="fas fa-check text-success"></i> 支持Redis集群连接配置</li>
|
||||
<li><i class="fas fa-check text-success"></i> 随机采样和指定Key查询</li>
|
||||
<li><i class="fas fa-check text-success"></i> 智能数据比对和差异分析</li>
|
||||
<li><i class="fas fa-check text-success"></i> 详细的性能统计报告</li>
|
||||
<li><i class="fas fa-check text-success"></i> 历史记录和结果导出</li>
|
||||
<li><i class="fas fa-check text-success"></i> Redis集群连接和配置管理</li>
|
||||
<li><i class="fas fa-check text-success"></i> 智能随机采样和指定Key查询</li>
|
||||
<li><i class="fas fa-check text-success"></i> 全数据类型支持(String/Hash/List/Set/ZSet)</li>
|
||||
<li><i class="fas fa-check text-success"></i> 实时性能统计和详细报告</li>
|
||||
<li><i class="fas fa-check text-success"></i> 批量操作和历史记录管理</li>
|
||||
<li><i class="fas fa-check text-success"></i> 可视化数据展示和导出功能</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
@@ -290,85 +295,70 @@
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 第二行工具 -->
|
||||
<div class="row mt-4">
|
||||
<div class="col-lg-6 col-md-12">
|
||||
<div class="tool-card">
|
||||
<div class="text-center">
|
||||
<div class="feature-badge coming-soon">即将推出</div>
|
||||
<div class="tool-icon">
|
||||
<i class="fas fa-file-import"></i>
|
||||
</div>
|
||||
<h3 class="tool-title">数据导入导出工具</h3>
|
||||
<p class="tool-description">
|
||||
高效的数据迁移工具,支持多种格式和数据库类型之间的数据传输,
|
||||
提供批量处理和进度监控功能。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="tool-features">
|
||||
<h5><i class="fas fa-star text-warning"></i> 计划功能:</h5>
|
||||
<ul>
|
||||
<li><i class="fas fa-clock text-muted"></i> 多格式数据支持</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 批量数据处理</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 实时进度监控</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 数据映射配置</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 错误处理和日志</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div class="text-center">
|
||||
<button class="tool-btn" disabled style="opacity: 0.6;">
|
||||
<i class="fas fa-hourglass-half"></i> 开发中
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-lg-6 col-md-12">
|
||||
<div class="tool-card">
|
||||
<div class="text-center">
|
||||
<div class="feature-badge coming-soon">即将推出</div>
|
||||
<div class="tool-icon">
|
||||
<i class="fas fa-shield-alt"></i>
|
||||
</div>
|
||||
<h3 class="tool-title">数据质量检测工具</h3>
|
||||
<p class="tool-description">
|
||||
专业的数据质量评估工具,自动检测数据完整性、一致性和准确性问题,
|
||||
生成详细的质量报告和改进建议。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="tool-features">
|
||||
<h5><i class="fas fa-star text-warning"></i> 计划功能:</h5>
|
||||
<ul>
|
||||
<li><i class="fas fa-clock text-muted"></i> 数据完整性检查</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 重复数据检测</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 数据格式验证</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 质量评分系统</li>
|
||||
<li><i class="fas fa-clock text-muted"></i> 自动化修复建议</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div class="text-center">
|
||||
<button class="tool-btn" disabled style="opacity: 0.6;">
|
||||
<i class="fas fa-hourglass-half"></i> 开发中
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 页脚 -->
|
||||
<div class="footer">
|
||||
<div class="container">
|
||||
<p>© 2024 大数据工具集合. 专注于提供高效的数据处理解决方案.</p>
|
||||
<p>© 2024 DataTools Pro. 企业级数据处理与比对解决方案.</p>
|
||||
<p class="mb-0">
|
||||
<small class="text-muted">
|
||||
Version 2.0 | Powered by Flask & Bootstrap |
|
||||
<i class="fas fa-heart text-danger"></i> Made with passion for data professionals
|
||||
</small>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
|
||||
<!-- 新的模块化JS结构 -->
|
||||
<script type="module" src="/static/js/app-main.js"></script>
|
||||
|
||||
<!-- 页面特定脚本 -->
|
||||
<script type="module">
|
||||
// 首页特定的功能
|
||||
import app from '/static/js/app-main.js';
|
||||
|
||||
// 首页初始化完成后的操作
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
// 为工具卡片添加悬停效果
|
||||
document.querySelectorAll('.tool-card').forEach(card => {
|
||||
card.addEventListener('mouseenter', function() {
|
||||
this.style.transform = 'translateY(-5px)';
|
||||
});
|
||||
|
||||
card.addEventListener('mouseleave', function() {
|
||||
this.style.transform = 'translateY(0)';
|
||||
});
|
||||
});
|
||||
|
||||
// 统计数字动画效果
|
||||
function animateNumbers() {
|
||||
const stats = document.querySelectorAll('.stat-number');
|
||||
stats.forEach(stat => {
|
||||
const target = stat.textContent;
|
||||
if (target === '∞' || target === '24/7') return;
|
||||
|
||||
const num = parseInt(target);
|
||||
if (isNaN(num)) return;
|
||||
|
||||
let current = 0;
|
||||
const increment = num / 20;
|
||||
const timer = setInterval(() => {
|
||||
current += increment;
|
||||
if (current >= num) {
|
||||
current = num;
|
||||
clearInterval(timer);
|
||||
}
|
||||
stat.textContent = Math.floor(current);
|
||||
}, 50);
|
||||
});
|
||||
}
|
||||
|
||||
// 页面加载后延迟执行动画
|
||||
setTimeout(animateNumbers, 500);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Redis集群比对工具</title>
|
||||
<title>Redis集群比对工具 - DataTools Pro</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
|
||||
<style>
|
||||
@@ -228,15 +228,15 @@
|
||||
<nav class="navbar navbar-expand-lg navbar-dark">
|
||||
<div class="container">
|
||||
<a class="navbar-brand" href="/">
|
||||
<i class="fas fa-database me-2"></i>
|
||||
大数据工具集合
|
||||
<i class="fab fa-redis me-2 redis-logo"></i>
|
||||
DataTools Pro
|
||||
</a>
|
||||
<div class="navbar-nav ms-auto">
|
||||
<a class="nav-link" href="/">
|
||||
<i class="fas fa-home"></i> 首页
|
||||
</a>
|
||||
<a class="nav-link" href="/db-compare">
|
||||
<i class="fas fa-exchange-alt"></i> 数据库比对
|
||||
<a class="nav-link" href="/cassandra-compare">
|
||||
<i class="fas fa-database"></i> Cassandra比对
|
||||
</a>
|
||||
<a class="nav-link active" href="/redis-compare">
|
||||
<i class="fab fa-redis"></i> Redis比对
|
||||
@@ -495,7 +495,7 @@
|
||||
</div>
|
||||
<div id="specifiedOptions" class="mt-2" style="display: none;">
|
||||
<label class="form-label">Key列表 (每行一个)</label>
|
||||
<textarea class="form-control query-keys" id="specifiedKeys" rows="6" placeholder="输入要查询的Key,每行一个 例如: user:1001 user:1002 session:abc123"></textarea>
|
||||
<textarea class="form-control query-keys" id="specifiedKeys" rows="6" placeholder="输入要查询的Key,每行一个 例如: user:example1 user:example2 session:abc123"></textarea>
|
||||
<small class="form-text text-muted">支持大批量Key查询,建议单次不超过1000个</small>
|
||||
</div>
|
||||
</div>
|
||||
@@ -674,22 +674,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 配置组管理对话框 -->
|
||||
<div class="modal fade" id="manageRedisConfigModal" tabindex="-1" aria-hidden="true">
|
||||
<div class="modal-dialog modal-lg">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h5 class="modal-title">管理Redis配置组</h5>
|
||||
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div id="redisConfigGroupList">
|
||||
<!-- 配置组列表将在这里动态生成 -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 导入配置模态框 -->
|
||||
<div class="modal fade" id="importConfigModal" tabindex="-1" aria-labelledby="importConfigModalLabel" aria-hidden="true">
|
||||
@@ -720,7 +704,7 @@
|
||||
<div id="textImportSection">
|
||||
<div class="mb-3">
|
||||
<label for="configYamlText" class="form-label">YAML配置内容</label>
|
||||
<textarea class="form-control" id="configYamlText" rows="8" placeholder="请粘贴YAML格式的配置内容,例如: clusterName: "redis-test" clusterAddress: "10.20.2.109:6470,10.20.2.109:6570" clusterPassword: """></textarea>
|
||||
<textarea class="form-control" id="configYamlText" rows="8" placeholder="请粘贴YAML格式的配置内容,例如: clusterName: "redis-example" clusterAddress: "127.0.0.1:6379" clusterPassword: """></textarea>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
Reference in New Issue
Block a user