Compare commits

...

7 Commits

Author SHA1 Message Date
14ad709b53 gitgnore 2025-12-08 12:51:50 +01:00
ccfb40a2b3 cleanup add training bell 2025-12-08 12:26:34 +01:00
Philipp
036f3b178a que and training status 2025-12-02 12:50:22 +01:00
Philipp
5bfe82fc26 addet training info 2025-12-02 10:51:20 +01:00
Philipp
de5a8d2028 add multi trainings +fix path for coco retraining 2025-12-02 10:16:18 +01:00
Philipp
0e31237b79 training fix.add global settings 2025-12-02 09:55:50 +01:00
c3c7e042bb training fix. add global settings 2025-12-02 09:31:52 +01:00
3079 changed files with 679039 additions and 7307 deletions

2
.gitignore vendored Normal file → Executable file
View File

@@ -3,3 +3,5 @@ backend/uploads/*.pth
*.pth
backend/node_modules/
backend/.venv/
key
key.pub

View File

@@ -1,217 +0,0 @@
# Transfer Learning Base Configuration Feature
## Overview
This feature implements automatic loading of base configurations when "Train on COCO" transfer learning is selected. Base parameters are loaded from `backend/data/` based on the selected YOLOX model, and these protected fields are displayed as greyed out and non-editable in the frontend.
## Components Modified/Created
### Backend
#### 1. Base Configuration Files (`backend/data/`)
- **`yolox_s.py`** - Base config for YOLOX-Small (depth=0.33, width=0.50)
- **`yolox_m.py`** - Base config for YOLOX-Medium (depth=0.67, width=0.75)
- **`yolox_l.py`** - Base config for YOLOX-Large (depth=1.0, width=1.0)
- **`yolox_x.py`** - Base config for YOLOX-XLarge (depth=1.33, width=1.25)
Each file contains a `BaseExp` class with protected parameters:
- Model architecture (depth, width, activation)
- Training hyperparameters (max_epoch, warmup_epochs, scheduler, etc.)
- Optimizer settings (momentum, weight_decay)
- Augmentation probabilities (mosaic_prob, mixup_prob, etc.)
- Input/output sizes
#### 2. Services (`backend/services/generate_yolox_exp.py`)
**New functions:**
- `load_base_config(selected_model)` - Dynamically loads base config using importlib
- Modified `generate_yolox_inference_exp()` to support `use_base_config` parameter
- Base config merging logic: base → user overrides → defaults
**Behavior:**
- `transfer_learning='coco'` → loads base config + applies user overrides
- `transfer_learning='sketch'` → uses only user-defined values
- Protected parameters from base config are preserved unless explicitly overridden
#### 3. API Routes (`backend/routes/api.py`)
**New endpoint:**
```python
@api_bp.route('/base-config/<model_name>', methods=['GET'])
def get_base_config(model_name):
```
Returns the base configuration JSON for a specific YOLOX model.
### Frontend
#### 1. HTML (`edit-training.html`)
**Added:**
- Info banner to indicate when base config is active
- CSS styles for disabled input fields (grey background, not-allowed cursor)
- Visual feedback showing which model's base config is loaded
**Banner HTML:**
```html
<div id="base-config-info" style="display:none; ...">
🔒 Base Configuration Active
Protected parameters are loaded from [model] base config
</div>
```
**CSS for disabled fields:**
```css
.setting-row input[type="number"]:disabled,
.setting-row input[type="text"]:disabled,
.setting-row input[type="checkbox"]:disabled {
background: #d3d3d3 !important;
color: #666 !important;
cursor: not-allowed !important;
border: 1px solid #999 !important;
}
```
#### 2. JavaScript (`js/start-training.js`)
**New functionality:**
1. **Base Config Loading:**
```javascript
function loadBaseConfig(modelName)
```
Fetches base config from `/api/base-config/<model>`
2. **Apply Base Config:**
```javascript
function applyBaseConfig(config, isCocoMode)
```
- Applies config values to form fields
- Disables and greys out protected fields
- Shows/hides info banner
- Adds tooltips to disabled fields
3. **Update Transfer Learning Mode:**
```javascript
function updateTransferLearningMode()
```
- Monitors changes to "Transfer Learning" dropdown
- Monitors changes to "Select Model" dropdown
- Loads appropriate base config when COCO mode is selected
- Clears base config when sketch mode is selected
4. **Form Submission Enhancement:**
- Temporarily enables disabled fields before submission
- Ensures protected parameters are included in form data
- Re-disables fields after collection
**Protected Fields List:**
```javascript
const protectedFields = [
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
];
```
## User Flow
### 1. Normal Custom Training (Train from sketch)
- User selects model: e.g., "YOLOX-s"
- User selects "Train from sketch"
- All fields are editable (white background)
- User can customize all parameters
- Submission uses user-defined values only
### 2. COCO Transfer Learning (Train on COCO)
- User selects model: e.g., "YOLOX-s"
- User selects "Train on coco"
- **Automatic actions:**
1. Frontend calls `/api/base-config/YOLOX-s`
2. Base config is loaded and applied
3. Protected fields become greyed out and disabled
4. Green info banner appears: "🔒 Base Configuration Active"
5. Tooltip on hover: "Protected by base config for YOLOX-s. Switch to 'Train from sketch' to customize."
- User can still edit non-protected fields
- On submit: both base config values AND user overrides are sent to backend
- Backend generates exp.py with merged settings
### 3. Switching Models
- User changes from "YOLOX-s" to "YOLOX-l" (while in COCO mode)
- Frontend automatically:
1. Fetches new base config for YOLOX-l
2. Updates field values (depth=1.0, width=1.0, etc.)
3. Updates banner text to show "YOLOX-l"
- Protected parameters update to match new model's architecture
## Testing
### Manual Test Steps:
1. **Test Base Config Loading:**
```bash
cd backend/data
python test_base_configs.py
```
Should display all parameters for yolox-s, yolox-m, yolox-l, yolox-x
2. **Test API Endpoint:**
```bash
# Start Flask server
cd backend
python app.py
# In another terminal:
curl http://localhost:3000/api/base-config/YOLOX-s
```
Should return JSON with depth, width, activation, etc.
3. **Test Frontend:**
- Open `edit-training.html?id=<project_id>` in browser
- Select "YOLOX-s" model
- Select "Train on coco" → fields should grey out
- Select "Train from sketch" → fields should become editable
- Switch to "YOLOX-l" (in COCO mode) → values should update
- Open browser console and check for: `Applied base config. Protected fields: depth, width, ...`
4. **Test Form Submission:**
- With COCO mode active (fields greyed out)
- Click "Save Parameters"
- Check browser Network tab → POST to `/api/yolox-settings`
- Verify payload includes protected parameters (depth, width, etc.)
- Check Flask logs for successful save
### Expected Behaviors:
✅ **COCO mode + YOLOX-s:**
- depth: 0.33 (greyed out)
- width: 0.50 (greyed out)
- activation: silu (greyed out)
- Info banner visible
✅ **COCO mode + YOLOX-l:**
- depth: 1.0 (greyed out)
- width: 1.0 (greyed out)
- activation: silu (greyed out)
✅ **Sketch mode:**
- All fields white/editable
- No info banner
- User can set any values
## Documentation
- **`backend/data/README.md`** - Complete guide on base config system
- **`backend/data/test_base_configs.py`** - Test script for base configs
## Benefits
1. **Proven defaults:** Users start with battle-tested COCO pretraining settings
2. **Prevents mistakes:** Can't accidentally break model architecture by changing depth/width
3. **Easy customization:** Can still override specific parameters if needed
4. **Visual feedback:** Clear indication of which fields are protected
5. **Model-specific:** Each model (s/m/l/x) has appropriate architecture defaults
6. **Flexible:** Can easily add new models by creating new base config files
## Future Enhancements
- Add "Override" button next to protected fields to unlock individual parameters
- Show diff comparison between base config and user overrides
- Add validation warnings if user tries values far from base config ranges
- Export final merged config as preview before training

190
add-project.html Normal file → Executable file
View File

@@ -29,10 +29,19 @@
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" class="button-red">Add Training Project</button>
<button id="Add Dataset" class="button">Add Dataset</button>
<button id="Import Dataset" class="button">Refresh Label-Studio</button>
<button id="seed-db-btn" class="button">Seed Database</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
@@ -169,6 +178,187 @@
</script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
</div>
</div>
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
</div>
</div>
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>

0
backend/.gitignore vendored Normal file → Executable file
View File

View File

@@ -1,28 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_27_train.json"
self.val_ann = "coco_project_27_valid.json"
self.test_ann = "coco_project_27_test.json"
self.num_classes = 80
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.random_size = (10, 20)
self.test_size = (640.0, 640.0)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

View File

@@ -1,148 +0,0 @@
# Backend Conversion Summary
## ✅ Conversion Complete
Your Node.js backend has been successfully converted to Python using Flask and SQLAlchemy.
## 📁 New Python Files Created
### Core Application
- **app.py** - Main Flask application (replaces server.js)
- **start.py** - Startup script for easy launching
- **requirements.txt** - Python dependencies (replaces package.json)
### Database Layer
- **database/database.py** - SQLAlchemy database configuration (replaces database.js)
### Models (Sequelize → SQLAlchemy)
- **models/TrainingProject.py**
- **models/TrainingProjectDetails.py**
- **models/training.py**
- **models/LabelStudioProject.py**
- **models/Images.py**
- **models/Annotation.py**
- **models/__init__.py**
### API Routes
- **routes/api.py** - All API endpoints converted to Flask blueprints (replaces api.js)
- **routes/__init__.py**
### Services
- **services/fetch_labelstudio.py** - Label Studio API integration
- **services/seed_label_studio.py** - Database seeding logic
- **services/generate_json_yolox.py** - COCO JSON generation
- **services/generate_yolox_exp.py** - YOLOX experiment file generation
- **services/push_yolox_exp.py** - Save training settings to DB
- **services/__init__.py**
### Documentation
- **README.md** - Comprehensive documentation
- **QUICKSTART.md** - Quick setup guide
- **.gitignore** - Python-specific ignore patterns
## 🔄 Key Changes
### Technology Stack
| Component | Node.js | Python |
|-----------|---------|--------|
| Framework | Express.js | Flask |
| ORM | Sequelize | SQLAlchemy |
| HTTP Client | node-fetch | requests |
| Package Manager | npm | pip |
| Runtime | Node.js | Python 3.8+ |
### API Compatibility
✅ All endpoints preserved with same URLs
✅ Request/response formats maintained
✅ Same database schema
✅ Same business logic
### Converted Features
- ✅ Training project management
- ✅ Label Studio integration
- ✅ YOLOX configuration and training
- ✅ File upload handling
- ✅ Image and annotation management
- ✅ COCO JSON generation
- ✅ Training logs
## 🚀 Getting Started
1. **Install dependencies:**
```bash
cd backend
python -m venv venv
.\venv\Scripts\Activate.ps1 # Windows
pip install -r requirements.txt
```
2. **Run the server:**
```bash
python start.py
```
3. **Server runs at:** `http://0.0.0.0:3000`
## 📦 Dependencies Installed
- Flask 3.0.0 - Web framework
- Flask-CORS 4.0.0 - Cross-origin resource sharing
- Flask-SQLAlchemy 3.1.1 - ORM integration
- SQLAlchemy 2.0.23 - Database ORM
- PyMySQL 1.1.0 - MySQL driver
- requests 2.31.0 - HTTP client
- Pillow 10.1.0 - Image processing
## ⚠️ Important Notes
1. **Virtual Environment**: Always activate the virtual environment before running
2. **Database**: MySQL must be running with the `myapp` database created
3. **Credentials**: Update database credentials in `app.py` if needed
4. **Python Version**: Requires Python 3.8 or higher
## 🧪 Testing
Test the conversion:
```bash
# Get all training projects
curl http://localhost:3000/api/training-projects
# Get Label Studio projects
curl http://localhost:3000/api/label-studio-projects
```
## 📝 Original Files
Your original Node.js files remain untouched:
- server.js
- package.json
- routes/api.js
- models/*.js (JavaScript)
- services/*.js (JavaScript)
You can keep them as backup or remove them once you verify the Python version works correctly.
## 🔍 What to Verify
1. ✅ Database connection works
2. ✅ All API endpoints respond correctly
3. ✅ File uploads work
4. ✅ Label Studio integration works
5. ✅ YOLOX training can be triggered
6. ✅ COCO JSON generation works
## 🐛 Troubleshooting
See **QUICKSTART.md** for common issues and solutions.
## 📚 Further Documentation
- **README.md** - Complete project documentation
- **QUICKSTART.md** - Setup guide
- **API Documentation** - All endpoints documented in README.md
---
**Conversion completed successfully!** 🎉
Your backend is now running on Python with Flask and SQLAlchemy.

View File

@@ -1,113 +0,0 @@
# Quick Start Guide - Python Backend
## Step-by-Step Setup
### 1. Install Python
Make sure you have Python 3.8 or higher installed:
```bash
python --version
```
### 2. Create Virtual Environment
```bash
cd backend
python -m venv venv
```
### 3. Activate Virtual Environment
**Windows:**
```powershell
.\venv\Scripts\Activate.ps1
```
**Linux/Mac:**
```bash
source venv/bin/activate
```
### 4. Install Dependencies
```bash
pip install -r requirements.txt
```
### 5. Verify Database Connection
Make sure MySQL is running and the database `myapp` exists:
```sql
CREATE DATABASE IF NOT EXISTS myapp;
```
### 6. Run the Server
```bash
python start.py
```
Or:
```bash
python app.py
```
The server should now be running at `http://0.0.0.0:3000`
## Testing the API
Test if the server is working:
```bash
curl http://localhost:3000/api/training-projects
```
## Common Issues
### ModuleNotFoundError
If you get import errors, make sure you've activated the virtual environment and installed all dependencies.
### Database Connection Error
Check that:
- MySQL is running
- Database credentials in `app.py` are correct
- Database `myapp` exists
### Port Already in Use
If port 3000 is already in use, modify the port in `app.py`:
```python
app.run(host='0.0.0.0', port=3001, debug=True)
```
## What Changed from Node.js
1. **Server Framework**: Express.js → Flask
2. **ORM**: Sequelize → SQLAlchemy
3. **HTTP Client**: node-fetch → requests
4. **Package Manager**: npm → pip
5. **Dependencies**: package.json → requirements.txt
6. **Startup**: `node server.js``python app.py`
## Next Steps
1. Test all API endpoints
2. Update frontend to point to the new Python backend (if needed)
3. Migrate any remaining Node.js-specific logic
4. Test file uploads and downloads
5. Test YOLOX training functionality
## File Structure Comparison
**Before (Node.js):**
```
backend/
├── server.js
├── package.json
├── routes/api.js
├── models/*.js
└── services/*.js
```
**After (Python):**
```
backend/
├── app.py
├── requirements.txt
├── routes/api.py
├── models/*.py
└── services/*.py
```

0
backend/README.md Normal file → Executable file
View File

9
backend/app.py Normal file → Executable file
View File

@@ -7,7 +7,7 @@ app = Flask(__name__, static_folder='..', static_url_path='')
CORS(app)
# Configure database
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp'
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp2'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
# Initialize database
@@ -37,7 +37,12 @@ if __name__ == '__main__':
# Create tables if they don't exist
db.create_all()
# Initialize default settings
from services.settings_service import initialize_default_settings
initialize_default_settings()
print('Settings initialized.')
# Start server
app.run(host='0.0.0.0', port=3000, debug=True)
app.run(host='0.0.0.0', port=4000, debug=True)
except Exception as err:
print(f'Failed to start: {err}')

View File

@@ -1,25 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_5_train.json"
self.val_ann = "coco_project_5_valid.json"
self.test_ann = "coco_project_5_test.json"
self.num_classes = 4
self.depth = 1.0
self.width = 1.0
self.input_size = (640, 640)
self.mosaic_scale = (0.1, 2)
self.random_size = (10, 20)
self.test_size = (640, 640)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

2
backend/check_db.py Normal file → Executable file
View File

@@ -1,6 +1,6 @@
import pymysql
conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp')
conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp2')
cursor = conn.cursor()
cursor.execute('DESCRIBE image')
rows = cursor.fetchall()

0
backend/data/README.md Normal file → Executable file
View File

0
backend/data/__init__.py Normal file → Executable file
View File

0
backend/data/test_base_configs.py Normal file → Executable file
View File

0
backend/data/yolox_l.py Normal file → Executable file
View File

0
backend/data/yolox_m.py Normal file → Executable file
View File

0
backend/data/yolox_s.py Normal file → Executable file
View File

0
backend/data/yolox_x.py Normal file → Executable file
View File

0
backend/database/__init__.py Normal file → Executable file
View File

View File

@@ -1,10 +0,0 @@
// database.js
const { Sequelize } = require('sequelize');
const sequelize = new Sequelize('myapp', 'root', 'root', {
host: 'localhost',
dialect: 'mysql',
logging: false,
});
module.exports = sequelize;

0
backend/database/database.py Normal file → Executable file
View File

View File

@@ -1,12 +0,0 @@
-- Migration: Add width and height columns to image table
-- Date: 2025-11-27
USE myapp;
-- Add width and height columns to image table
ALTER TABLE `image`
ADD COLUMN `width` FLOAT NULL AFTER `image_path`,
ADD COLUMN `height` FLOAT NULL AFTER `width`;
-- Verify the changes
DESCRIBE `image`;

File diff suppressed because one or more lines are too long

View File

@@ -1,40 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Annotation = sequelize.define('Annotation', {
annotation_id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
},
image_id: {
type: DataTypes.INTEGER,
allowNull: false,
},
x: {
type: DataTypes.FLOAT,
allowNull: false,
},
y: {
type: DataTypes.FLOAT,
allowNull: false,
},
height: {
type: DataTypes.FLOAT,
allowNull: false,
},
width: {
type: DataTypes.FLOAT,
allowNull: false,
},
Label: {
type: DataTypes.STRING,
allowNull: false,
},
}, {
tableName: 'annotation',
timestamps: false,
});
module.exports = Annotation;

2
backend/models/Annotation.py Normal file → Executable file
View File

@@ -4,7 +4,7 @@ class Annotation(db.Model):
__tablename__ = 'annotation'
annotation_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_id = db.Column(db.Integer, nullable=False)
image_id = db.Column(db.Integer, db.ForeignKey('image.image_id', ondelete='CASCADE'), nullable=False)
x = db.Column(db.Float, nullable=False)
y = db.Column(db.Float, nullable=False)
height = db.Column(db.Float, nullable=False)

View File

@@ -0,0 +1,21 @@
from database.database import db
class AnnotationProjectMapping(db.Model):
"""Mapping between training project details and label studio projects (3NF)"""
__tablename__ = 'annotation_project_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
# Unique constraint: each label studio project can only be mapped once per training project details
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', name='uq_annotation_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id
}

25
backend/models/ClassMapping.py Executable file
View File

@@ -0,0 +1,25 @@
from database.database import db
class ClassMapping(db.Model):
"""Class name mappings for training project details (3NF)"""
__tablename__ = 'class_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
source_class = db.Column(db.String(255), nullable=False)
target_class = db.Column(db.String(255), nullable=False)
# Unique constraint: each source class can only be mapped once per project details AND label studio project
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', 'source_class', name='uq_class_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id,
'source_class': self.source_class,
'target_class': self.target_class
}

View File

@@ -1,35 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Image = sequelize.define('Image', {
image_id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
},
image_path: {
type: DataTypes.STRING,
allowNull: false,
},
project_id: {
type: DataTypes.INTEGER,
allowNull: false,
},
width: {
type: DataTypes.FLOAT,
allowNull: true,
},
height: {
type: DataTypes.FLOAT,
allowNull: true,
},
}, {
tableName: 'image',
timestamps: false,
});
module.exports = Image;

6
backend/models/Images.py Normal file → Executable file
View File

@@ -5,9 +5,9 @@ class Image(db.Model):
image_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_path = db.Column(db.String(500), nullable=False)
project_id = db.Column(db.Integer, nullable=False)
width = db.Column(db.Float)
height = db.Column(db.Float)
project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
width = db.Column(db.Integer)
height = db.Column(db.Integer)
def to_dict(self):
return {

View File

@@ -1,24 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Label_studio_project = sequelize.define('LabelStudioProject', {
project_id: {
type: DataTypes.INTEGER,
primaryKey: true,
unique: true,
allowNull: false,
},
title:{
type: DataTypes.STRING,
allowNull: false,
}
}, {
tableName: 'label_studio_project',
timestamps: false,
});
module.exports = Label_studio_project;

0
backend/models/LabelStudioProject.py Normal file → Executable file
View File

23
backend/models/ProjectClass.py Executable file
View File

@@ -0,0 +1,23 @@
from database.database import db
class ProjectClass(db.Model):
"""Class definitions for training projects (3NF)"""
__tablename__ = 'project_class'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False)
class_name = db.Column(db.String(255), nullable=False)
display_order = db.Column(db.Integer, default=0)
# Unique constraint: one class name per project
__table_args__ = (
db.UniqueConstraint('project_id', 'class_name', name='uq_project_class'),
)
def to_dict(self):
return {
'id': self.id,
'project_id': self.project_id,
'class_name': self.class_name,
'display_order': self.display_order
}

21
backend/models/Settings.py Executable file
View File

@@ -0,0 +1,21 @@
from database.database import db
class Settings(db.Model):
__tablename__ = 'settings'
id = db.Column(db.Integer, primary_key=True)
key = db.Column(db.String(255), unique=True, nullable=False)
value = db.Column(db.Text, nullable=True)
description = db.Column(db.String(500), nullable=True)
created_at = db.Column(db.DateTime, server_default=db.func.now())
updated_at = db.Column(db.DateTime, server_default=db.func.now(), onupdate=db.func.now())
def to_dict(self):
return {
'id': self.id,
'key': self.key,
'value': self.value,
'description': self.description,
'created_at': self.created_at.isoformat() if self.created_at else None,
'updated_at': self.updated_at.isoformat() if self.updated_at else None
}

View File

@@ -1,38 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Training_Project = sequelize.define('LabelStudioProject', {
project_id: {
type: DataTypes.INTEGER,
primaryKey: true,
unique: true,
allowNull: false,
autoIncrement: true,
},
title:{
type: DataTypes.STRING,
allowNull: false,
},
description: {
type: DataTypes.STRING,
},
classes: {
type: DataTypes.JSON,
allowNull: false,
},
project_image: {
type: DataTypes.BLOB,
},
project_image_type: {
type: DataTypes.STRING,
allowNull: true,
}
}, {
tableName: 'training_project',
timestamps: false,
});
module.exports = Training_Project;

14
backend/models/TrainingProject.py Normal file → Executable file
View File

@@ -6,18 +6,26 @@ class TrainingProject(db.Model):
project_id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
title = db.Column(db.String(255), nullable=False)
description = db.Column(db.String(500))
classes = db.Column(db.JSON, nullable=False)
project_image = db.Column(db.LargeBinary)
project_image_type = db.Column(db.String(100))
def to_dict(self):
# Relationship to classes (3NF)
classes_relation = db.relationship('ProjectClass', backref='project', lazy=True, cascade='all, delete-orphan')
def to_dict(self, include_classes=True):
result = {
'project_id': self.project_id,
'title': self.title,
'description': self.description,
'classes': self.classes,
'project_image_type': self.project_image_type
}
# Include classes as array for backwards compatibility
if include_classes:
from models.ProjectClass import ProjectClass
classes = ProjectClass.query.filter_by(project_id=self.project_id).order_by(ProjectClass.display_order).all()
result['classes'] = [c.class_name for c in classes]
if self.project_image:
import base64
base64_data = base64.b64encode(self.project_image).decode('utf-8')

View File

@@ -1,33 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const TrainingProjectDetails = sequelize.define('TrainingProjectDetails', {
id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
unique: true,
},
project_id: {
type: DataTypes.INTEGER,
allowNull: false,
unique: true,
},
annotation_projects: {
type: DataTypes.JSON,
allowNull: false,
},
class_map: {
type: DataTypes.JSON,
allowNull: true,
},
description: {
type: DataTypes.JSON,
allowNull: true,
}
}, {
tableName: 'training_project_details',
timestamps: false,
});
module.exports = TrainingProjectDetails;

34
backend/models/TrainingProjectDetails.py Normal file → Executable file
View File

@@ -4,16 +4,32 @@ class TrainingProjectDetails(db.Model):
__tablename__ = 'training_project_details'
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
project_id = db.Column(db.Integer, nullable=False, unique=True)
annotation_projects = db.Column(db.JSON, nullable=False)
class_map = db.Column(db.JSON)
description = db.Column(db.JSON)
project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False, unique=True)
description_text = db.Column(db.Text) # Renamed from 'description' JSON to plain text
def to_dict(self):
return {
# Relationships (3NF)
annotation_mappings = db.relationship('AnnotationProjectMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
class_mappings = db.relationship('ClassMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
def to_dict(self, include_mappings=True):
result = {
'id': self.id,
'project_id': self.project_id,
'annotation_projects': self.annotation_projects,
'class_map': self.class_map,
'description': self.description
'description': self.description_text
}
# Include mappings for backwards compatibility
if include_mappings:
from models.AnnotationProjectMapping import AnnotationProjectMapping
from models.ClassMapping import ClassMapping
# Get annotation projects as array
mappings = AnnotationProjectMapping.query.filter_by(project_details_id=self.id).all()
result['annotation_projects'] = [m.label_studio_project_id for m in mappings]
# Get class map as dictionary (grouped by label_studio_project_id for backwards compatibility)
# Return format: {source: target} (flattened across all projects)
class_maps = ClassMapping.query.filter_by(project_details_id=self.id).all()
result['class_map'] = {cm.source_class: cm.target_class for cm in class_maps}
return result

25
backend/models/TrainingSize.py Executable file
View File

@@ -0,0 +1,25 @@
from database.database import db
class TrainingSize(db.Model):
"""Size configurations for training (3NF - replaces JSON arrays)"""
__tablename__ = 'training_size'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
training_id = db.Column(db.Integer, db.ForeignKey('training.id', ondelete='CASCADE'), nullable=False)
size_type = db.Column(db.String(50), nullable=False) # 'input_size', 'test_size', 'mosaic_scale', 'mixup_scale'
value_order = db.Column(db.Integer, nullable=False, default=0) # Order in array (0=first, 1=second)
value = db.Column(db.Float, nullable=False)
# Composite key ensures proper ordering
__table_args__ = (
db.UniqueConstraint('training_id', 'size_type', 'value_order', name='uq_training_size'),
)
def to_dict(self):
return {
'id': self.id,
'training_id': self.training_id,
'size_type': self.size_type,
'value_order': self.value_order,
'value': self.value
}

12
backend/models/__init__.py Normal file → Executable file
View File

@@ -5,6 +5,11 @@ from models.training import Training
from models.LabelStudioProject import LabelStudioProject
from models.Images import Image
from models.Annotation import Annotation
from models.Settings import Settings
from models.ProjectClass import ProjectClass
from models.AnnotationProjectMapping import AnnotationProjectMapping
from models.ClassMapping import ClassMapping
from models.TrainingSize import TrainingSize
__all__ = [
'TrainingProject',
@@ -12,5 +17,10 @@ __all__ = [
'Training',
'LabelStudioProject',
'Image',
'Annotation'
'Annotation',
'Settings',
'ProjectClass',
'AnnotationProjectMapping',
'ClassMapping',
'TrainingSize'
]

View File

@@ -1,30 +0,0 @@
const LabelStudioProject = require('./LabelStudioProject.js');
const Annotation = require('./Annotation.js');
const Image = require('./Images.js');
const sequelize = require('../database/database.js');
const TrainingProjectDetails = require('./TrainingProjectDetails.js');
const TrainingProject = require('./TrainingProject.js');
const Training = require('./training.js');
const Project = LabelStudioProject;
const Img = Image;
const Ann = Annotation;
// Associations
Project.hasMany(Img, { foreignKey: 'project_id' });
Img.belongsTo(Project, { foreignKey: 'project_id' });
Img.hasMany(Ann, { foreignKey: 'image_id' });
Ann.belongsTo(Img, { foreignKey: 'image_id' });
// TrainingProjectDetails <-> TrainingProject
TrainingProjectDetails.belongsTo(TrainingProject, { foreignKey: 'project_id' });
TrainingProject.hasOne(TrainingProjectDetails, { foreignKey: 'project_id' });
// Training <-> TrainingProjectDetails
Training.belongsTo(TrainingProjectDetails, { foreignKey: 'project_details_id' });
TrainingProjectDetails.hasMany(Training, { foreignKey: 'project_details_id' });
module.exports = { Project, Img, Ann, TrainingProjectDetails, TrainingProject, Training };

View File

@@ -1,140 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Training = sequelize.define('training', {
id: {
type: DataTypes.INTEGER,
autoIncrement: true,
unique: true,
primaryKey: true
},
exp_name: {
type: DataTypes.STRING(255)
},
max_epoch: {
type: DataTypes.INTEGER
},
depth: {
type: DataTypes.FLOAT
},
width: {
type: DataTypes.FLOAT
},
activation: {
type: DataTypes.STRING(255)
},
warmup_epochs: {
type: DataTypes.INTEGER
},
warmup_lr: {
type: DataTypes.FLOAT
},
basic_lr_per_img: {
type: DataTypes.FLOAT
},
scheduler: {
type: DataTypes.STRING(255)
},
no_aug_epochs: {
type: DataTypes.INTEGER
},
min_lr_ratio: {
type: DataTypes.FLOAT
},
ema: {
type: DataTypes.BOOLEAN
},
weight_decay: {
type: DataTypes.FLOAT
},
momentum: {
type: DataTypes.FLOAT
},
input_size: {
type: DataTypes.JSON
},
print_interval: {
type: DataTypes.INTEGER
},
eval_interval: {
type: DataTypes.INTEGER
},
save_history_ckpt: {
type: DataTypes.BOOLEAN
},
test_size: {
type: DataTypes.JSON
},
test_conf: {
type: DataTypes.FLOAT
},
nms_thre: {
type: DataTypes.FLOAT
},
multiscale_range: {
type: DataTypes.INTEGER
},
enable_mixup: {
type: DataTypes.BOOLEAN
},
mosaic_prob: {
type: DataTypes.FLOAT
},
mixup_prob: {
type: DataTypes.FLOAT
},
hsv_prob: {
type: DataTypes.FLOAT
},
flip_prob: {
type: DataTypes.FLOAT
},
degrees: {
type: DataTypes.FLOAT
},
mosaic_scale: {
type: DataTypes.JSON
},
mixup_scale: {
type: DataTypes.JSON
},
translate: {
type: DataTypes.FLOAT
},
shear: {
type: DataTypes.FLOAT
},
training_name: {
type: DataTypes.STRING(255)
},
project_details_id: {
type: DataTypes.INTEGER,
allowNull: false
},
seed: {
type: DataTypes.INTEGER
},
train: {
type: DataTypes.INTEGER
},
valid: {
type: DataTypes.INTEGER
},
test: {
type: DataTypes.INTEGER
},
selected_model: {
type: DataTypes.STRING(255)
},
transfer_learning: {
type: DataTypes.STRING(255)
},
model_upload: {
type: DataTypes.BLOB
}
}, {
tableName: 'training',
timestamps: false
});
module.exports = Training;

39
backend/models/training.py Normal file → Executable file
View File

@@ -18,11 +18,11 @@ class Training(db.Model):
ema = db.Column(db.Boolean)
weight_decay = db.Column(db.Float)
momentum = db.Column(db.Float)
input_size = db.Column(db.JSON)
# input_size moved to TrainingSize table
print_interval = db.Column(db.Integer)
eval_interval = db.Column(db.Integer)
save_history_ckpt = db.Column(db.Boolean)
test_size = db.Column(db.JSON)
# test_size moved to TrainingSize table
test_conf = db.Column(db.Float)
nms_thre = db.Column(db.Float)
multiscale_range = db.Column(db.Integer)
@@ -32,12 +32,12 @@ class Training(db.Model):
hsv_prob = db.Column(db.Float)
flip_prob = db.Column(db.Float)
degrees = db.Column(db.Float)
mosaic_scale = db.Column(db.JSON)
mixup_scale = db.Column(db.JSON)
# mosaic_scale moved to TrainingSize table
# mixup_scale moved to TrainingSize table
translate = db.Column(db.Float)
shear = db.Column(db.Float)
training_name = db.Column(db.String(255))
project_details_id = db.Column(db.Integer, nullable=False)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
seed = db.Column(db.Integer)
train = db.Column(db.Integer)
valid = db.Column(db.Integer)
@@ -46,8 +46,11 @@ class Training(db.Model):
transfer_learning = db.Column(db.String(255))
model_upload = db.Column(db.LargeBinary)
def to_dict(self):
return {
# Relationship to size configurations (3NF)
size_configs = db.relationship('TrainingSize', backref='training', lazy=True, cascade='all, delete-orphan')
def to_dict(self, include_sizes=True):
result = {
'id': self.id,
'exp_name': self.exp_name,
'max_epoch': self.max_epoch,
@@ -63,11 +66,9 @@ class Training(db.Model):
'ema': self.ema,
'weight_decay': self.weight_decay,
'momentum': self.momentum,
'input_size': self.input_size,
'print_interval': self.print_interval,
'eval_interval': self.eval_interval,
'save_history_ckpt': self.save_history_ckpt,
'test_size': self.test_size,
'test_conf': self.test_conf,
'nms_thre': self.nms_thre,
'multiscale_range': self.multiscale_range,
@@ -77,8 +78,6 @@ class Training(db.Model):
'hsv_prob': self.hsv_prob,
'flip_prob': self.flip_prob,
'degrees': self.degrees,
'mosaic_scale': self.mosaic_scale,
'mixup_scale': self.mixup_scale,
'translate': self.translate,
'shear': self.shear,
'training_name': self.training_name,
@@ -90,3 +89,21 @@ class Training(db.Model):
'selected_model': self.selected_model,
'transfer_learning': self.transfer_learning
}
# Include size arrays for backwards compatibility
if include_sizes:
from models.TrainingSize import TrainingSize
def get_size_array(size_type):
sizes = TrainingSize.query.filter_by(
training_id=self.id,
size_type=size_type
).order_by(TrainingSize.value_order).all()
return [s.value for s in sizes] if sizes else None
result['input_size'] = get_size_array('input_size')
result['test_size'] = get_size_array('test_size')
result['mosaic_scale'] = get_size_array('mosaic_scale')
result['mixup_scale'] = get_size_array('mixup_scale')
return result

0
backend/node Normal file → Executable file
View File

0
backend/package-lock.json generated Normal file → Executable file
View File

0
backend/package.json Normal file → Executable file
View File

0
backend/requirements.txt Normal file → Executable file
View File

0
backend/routes/__init__.py Normal file → Executable file
View File

View File

@@ -1,496 +0,0 @@
const express = require('express');
const multer = require('multer');
const upload = multer();
const TrainingProject = require('../models/TrainingProject.js');
const LabelStudioProject = require('../models/LabelStudioProject.js')
const { seedLabelStudio, updateStatus } = require('../services/seed-label-studio.js');
const fs = require('fs');
const path = require('path');
const {generateTrainingJson} = require('../services/generate-json-yolox.js')
const router = express.Router();
// Ensure JSON bodies are parsed for all routes
router.use(express.json());
router.get('/seed', async (req, res) => {
const result = await seedLabelStudio();
res.json(result);
});
// Trigger generate-json-yolox.js
router.post('/generate-yolox-json', async (req, res) => {
const { project_id } = req.body;
if (!project_id) {
return res.status(400).json({ message: 'Missing project_id in request body' });
}
try {
// Generate COCO JSONs
// Find all TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) {
return res.status(404).json({ message: 'No TrainingProjectDetails found for project ' + project_id });
}
// For each details row, generate coco.jsons and exp.py in projectfolder/project_details_id
const Training = require('../models/training.js');
const { saveYoloxExp } = require('../services/generate-yolox-exp.js');
const TrainingProject = require('../models/TrainingProject.js');
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
for (const details of detailsRows) {
const detailsId = details.id;
await generateTrainingJson(detailsId);
const trainings = await Training.findAll({ where: { project_details_id: detailsId } });
if (trainings.length === 0) continue;
// For each training, save exp.py in projectfolder/project_details_id
const outDir = path.join(__dirname, '..', projectName, String(detailsId));
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
for (const training of trainings) {
const expFilePath = path.join(outDir, 'exp.py');
await saveYoloxExp(training.id, expFilePath);
}
}
// Find all trainings for this project
// ...existing code...
res.json({ message: 'YOLOX JSON and exp.py generated for project ' + project_id });
} catch (err) {
console.error('Error generating YOLOX JSON:', err);
res.status(500).json({ message: 'Failed to generate YOLOX JSON', error: err.message });
}
});
// Start YOLOX training
const { spawn } = require('child_process');
router.post('/start-yolox-training', async (req, res) => {
try {
const { project_id, training_id } = req.body;
// Get project name
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
// Look up training row by id or project_details_id
const Training = require('../models/training.js');
let trainingRow = await Training.findByPk(training_id);
if (!trainingRow) {
trainingRow = await Training.findOne({ where: { project_details_id: training_id } });
}
if (!trainingRow) {
return res.status(404).json({ error: `Training row not found for id or project_details_id ${training_id}` });
}
const project_details_id = trainingRow.project_details_id;
// Use the generated exp.py from the correct project folder
const outDir = path.join(__dirname, '..', projectName, String(project_details_id));
const yoloxMainDir = '/home/kitraining/Yolox/YOLOX-main';
const expSrc = path.join(outDir, 'exp.py');
if (!fs.existsSync(expSrc)) {
return res.status(500).json({ error: `exp.py not found at ${expSrc}` });
}
// Source venv and run YOLOX training in YOLOX-main folder
const yoloxVenv = '/home/kitraining/Yolox/yolox_venv/bin/activate';
// Determine model argument based on selected_model and transfer_learning
let modelArg = '';
let cmd = '';
if (
trainingRow.transfer_learning &&
typeof trainingRow.transfer_learning === 'string' &&
trainingRow.transfer_learning.toLowerCase() === 'coco'
) {
// If transfer_learning is 'coco', add -o and modelArg
modelArg = ` -c /home/kitraining/Yolox/YOLOX-main/pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else if (
trainingRow.selected_model &&
trainingRow.selected_model.toLowerCase() === 'coco' &&
(!trainingRow.transfer_learning || trainingRow.transfer_learning === false)
) {
// If selected_model is 'coco' and not transfer_learning, add modelArg only
modelArg = ` -c /pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else {
// Default: no modelArg
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16' --cache`;
}
console.log(cmd)
const child = spawn(cmd, { shell: true, cwd: yoloxMainDir });
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);
res.json({ message: 'Training started' });
} catch (err) {
res.status(500).json({ error: 'Failed to start training', details: err.message });
}
});
// Get YOLOX training log
router.get('/training-log', async (req, res) => {
try {
const { project_id, training_id } = req.query;
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
const outDir = path.join(__dirname, '..', projectName, String(training_id));
const logPath = path.join(outDir, 'training.log');
if (!fs.existsSync(logPath)) {
return res.status(404).json({ error: 'Log not found' });
}
const logData = fs.readFileSync(logPath, 'utf8');
res.json({ log: logData });
} catch (err) {
res.status(500).json({ error: 'Failed to fetch log', details: err.message });
}
});
router.post('/training-projects', upload.single('project_image'), async (req, res) => {
try {
const { title, description } = req.body;
const classes = JSON.parse(req.body.classes);
const project_image = req.file ? req.file.buffer : null;
const project_image_type = req.file ? req.file.mimetype : null;
await TrainingProject.create({
title,
description,
classes,
project_image,
project_image_type
});
res.json({ message: 'Project created!' });
} catch (error) {
console.error('Error creating project:', error);
res.status(500).json({ message: 'Failed to create project', error: error.message });
}
});
router.get('/training-projects', async (req, res) => {
try {
const projects = await TrainingProject.findAll();
// Convert BLOB to base64 data URL for each project
const serialized = projects.map(project => {
const plain = project.get({ plain: true });
if (plain.project_image) {
const base64 = Buffer.from(plain.project_image).toString('base64');
const mimeType = plain.project_image_type || 'image/png';
plain.project_image = `data:${mimeType};base64,${base64}`;
}
return plain;
});
res.json(serialized);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
}
});
router.get('/update-status', async (req, res) => {
res.json(updateStatus)
})
router.get('/label-studio-projects', async (req, res) => {
try {
const LabelStudioProject = require('../models/LabelStudioProject.js');
const Image = require('../models/Images.js');
const Annotation = require('../models/Annotation.js');
const labelStudioProjects = await LabelStudioProject.findAll();
const projectsWithCounts = await Promise.all(labelStudioProjects.map(async project => {
const plain = project.get({ plain: true });
// Get all images for this project
const images = await Image.findAll({ where: { project_id: plain.project_id } });
let annotationCounts = {};
if (images.length > 0) {
const imageIds = images.map(img => img.image_id);
// Get all annotations for these images
const annotations = await Annotation.findAll({ where: { image_id: imageIds } });
// Count by label
for (const ann of annotations) {
const label = ann.Label;
annotationCounts[label] = (annotationCounts[label] || 0) + 1;
}
}
plain.annotationCounts = annotationCounts;
return plain;
}));
res.json(projectsWithCounts);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
}
})
// POST endpoint to create TrainingProjectDetails with all fields
router.post('/training-project-details', async (req, res) => {
try {
const {
project_id,
annotation_projects,
class_map,
description
} = req.body;
if (!project_id || !annotation_projects) {
return res.status(400).json({ message: 'Missing required fields' });
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const created = await TrainingProjectDetails.create({
project_id,
annotation_projects,
class_map: class_map || null,
description: description || null
});
res.json({ message: 'TrainingProjectDetails created', details: created });
} catch (error) {
res.status(500).json({ message: 'Failed to create TrainingProjectDetails', error: error.message });
}
});
// GET endpoint to fetch all TrainingProjectDetails
router.get('/training-project-details', async (req, res) => {
try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findAll();
res.json(details);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch TrainingProjectDetails', error: error.message });
}
});
// PUT endpoint to update class_map and description in TrainingProjectDetails
router.put('/training-project-details', async (req, res) => {
try {
const { project_id, class_map, description } = req.body;
if (!project_id || !class_map || !description) {
return res.status(400).json({ message: 'Missing required fields' });
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id } });
if (!details) {
return res.status(404).json({ message: 'TrainingProjectDetails not found' });
}
details.class_map = class_map;
details.description = description;
await details.save();
res.json({ message: 'Class map and description updated', details });
} catch (error) {
res.status(500).json({ message: 'Failed to update class map or description', error: error.message });
}
});
// POST endpoint to receive YOLOX settings and save to DB (handles multipart/form-data)
router.post('/yolox-settings', upload.any(), async (req, res) => {
try {
const settings = req.body;
// Debug: Log all received fields and types
console.log('--- YOLOX settings received ---');
console.log('settings:', settings);
if (req.files && req.files.length > 0) {
console.log('Files received:', req.files.map(f => ({ fieldname: f.fieldname, originalname: f.originalname, size: f.size })));
}
// Declare requiredFields once
const requiredFields = ['project_details_id', 'exp_name', 'max_epoch', 'depth', 'width', 'activation', 'train', 'valid', 'test', 'selected_model', 'transfer_learning'];
// Log types of required fields
requiredFields.forEach(field => {
console.log(`Field '${field}': value='${settings[field]}', type='${typeof settings[field]}'`);
});
// Map select_model to selected_model if present
if (settings && settings.select_model && !settings.selected_model) {
settings.selected_model = settings.select_model;
delete settings.select_model;
}
// Lookup project_details_id from project_id
if (!settings.project_id || isNaN(Number(settings.project_id))) {
throw new Error('Missing or invalid project_id in request. Cannot assign training to a project.');
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
let details = await TrainingProjectDetails.findOne({ where: { project_id: settings.project_id } });
if (!details) {
details = await TrainingProjectDetails.create({
project_id: settings.project_id,
annotation_projects: [],
class_map: null,
description: null
});
}
settings.project_details_id = details.id;
// Map 'act' from frontend to 'activation' for DB
if (settings.act !== undefined) {
settings.activation = settings.act;
delete settings.act;
}
// Type conversion for DB compatibility
[
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr', 'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum', 'print_interval', 'eval_interval', 'test_conf', 'nmsthre', 'multiscale_range', 'degrees', 'translate', 'shear', 'train', 'valid', 'test'
].forEach(f => {
if (settings[f] !== undefined) settings[f] = Number(settings[f]);
});
// Improved boolean conversion
['ema', 'enable_mixup', 'save_history_ckpt'].forEach(f => {
if (settings[f] !== undefined) {
if (typeof settings[f] === 'string') {
settings[f] = settings[f].toLowerCase() === 'true';
} else {
settings[f] = Boolean(settings[f]);
}
}
});
// Improved array conversion
['mosaic_scale', 'mixup_scale', 'scale'].forEach(f => {
if (settings[f] && typeof settings[f] === 'string') {
settings[f] = settings[f]
.split(',')
.map(s => Number(s.trim()))
.filter(n => !isNaN(n));
}
});
// Trim all string fields
Object.keys(settings).forEach(f => {
if (typeof settings[f] === 'string') settings[f] = settings[f].trim();
});
// Set default for transfer_learning if missing
if (settings.transfer_learning === undefined) settings.transfer_learning = false;
// Convert empty string seed to null
if ('seed' in settings && (settings.seed === '' || settings.seed === undefined)) {
settings.seed = null;
}
// Validate required fields for training table
for (const field of requiredFields) {
if (settings[field] === undefined || settings[field] === null || settings[field] === '') {
console.error('Missing required field:', field, 'Value:', settings[field]);
throw new Error('Missing required field: ' + field);
}
}
console.log('Received YOLOX settings:', settings);
// Handle uploaded model file (ckpt_upload)
if (req.files && req.files.length > 0) {
const ckptFile = req.files.find(f => f.fieldname === 'ckpt_upload');
if (ckptFile) {
const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = ckptFile.originalname || `uploaded_model_${settings.project_id}.pth`;
const filePath = path.join(uploadDir, filename);
fs.writeFileSync(filePath, ckptFile.buffer);
settings.model_upload = filePath;
}
}
// Save settings to DB only (no file)
const { pushYoloxExpToDb } = require('../services/push-yolox-exp.js');
const training = await pushYoloxExpToDb(settings);
res.json({ message: 'YOLOX settings saved to DB', training });
} catch (error) {
console.error('Error in /api/yolox-settings:', error.stack || error);
res.status(500).json({ message: 'Failed to save YOLOX settings', error: error.message });
}
});
// POST endpoint to receive binary model file and save to disk (not DB)
router.post('/yolox-settings/upload', async (req, res) => {
try {
const projectId = req.query.project_id;
if (!projectId) return res.status(400).json({ message: 'Missing project_id in query' });
// Save file to disk
const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = req.headers['x-upload-filename'] || `uploaded_model_${projectId}.pth`;
const filePath = path.join(uploadDir, filename);
const chunks = [];
req.on('data', chunk => chunks.push(chunk));
req.on('end', async () => {
const buffer = Buffer.concat(chunks);
fs.writeFile(filePath, buffer, async err => {
if (err) {
console.error('Error saving file:', err);
return res.status(500).json({ message: 'Failed to save model file', error: err.message });
}
// Update latest training row for this project with file path
try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
// Find details row for this project
const details = await TrainingProjectDetails.findOne({ where: { project_id: projectId } });
if (!details) return res.status(404).json({ message: 'No TrainingProjectDetails found for project_id' });
// Find latest training for this details row
const training = await Training.findOne({ where: { project_details_id: details.id }, order: [['createdAt', 'DESC']] });
if (!training) return res.status(404).json({ message: 'No training found for project_id' });
// Save file path to model_upload field
training.model_upload = filePath;
await training.save();
res.json({ message: 'Model file uploaded and saved to disk', filename, trainingId: training.id });
} catch (dbErr) {
console.error('Error updating training with file path:', dbErr);
res.status(500).json({ message: 'File saved but failed to update training row', error: dbErr.message });
}
});
});
} catch (error) {
console.error('Error in /api/yolox-settings/upload:', error.stack || error);
res.status(500).json({ message: 'Failed to upload model file', error: error.message });
}
});
// GET endpoint to fetch all trainings (optionally filtered by project_id)
router.get('/trainings', async (req, res) => {
try {
const project_id = req.query.project_id;
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
if (project_id) {
// Find all details rows for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) return res.json([]);
// Get all trainings linked to any details row for this project
const detailsIds = detailsRows.map(d => d.id);
const trainings = await Training.findAll({ where: { project_details_id: detailsIds } });
return res.json(trainings);
} else {
// Return all trainings if no project_id is specified
const trainings = await Training.findAll();
return res.json(trainings);
}
} catch (error) {
res.status(500).json({ message: 'Failed to fetch trainings', error: error.message });
}
});
// DELETE endpoint to remove a training by id
router.delete('/trainings/:id', async (req, res) => {
try {
const Training = require('../models/training.js');
const id = req.params.id;
const deleted = await Training.destroy({ where: { id } });
if (deleted) {
res.json({ message: 'Training deleted' });
} else {
res.status(404).json({ message: 'Training not found' });
}
} catch (error) {
res.status(500).json({ message: 'Failed to delete training', error: error.message });
}
});
// DELETE endpoint to remove a training project and all related entries
router.delete('/training-projects/:id', async (req, res) => {
try {
const projectId = req.params.id;
const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
// Find details row(s) for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id: projectId } });
const detailsIds = detailsRows.map(d => d.id);
// Delete all trainings linked to these details
if (detailsIds.length > 0) {
await Training.destroy({ where: { project_details_id: detailsIds } });
await TrainingProjectDetails.destroy({ where: { project_id: projectId } });
}
// Delete the project itself
const deleted = await TrainingProject.destroy({ where: { project_id: projectId } });
if (deleted) {
res.json({ message: 'Training project and all related entries deleted' });
} else {
res.status(404).json({ message: 'Training project not found' });
}
} catch (error) {
res.status(500).json({ message: 'Failed to delete training project', error: error.message });
}
});
module.exports = router;

479
backend/routes/api.py Normal file → Executable file
View File

@@ -73,63 +73,124 @@ def generate_yolox_json():
@api_bp.route('/start-yolox-training', methods=['POST'])
def start_yolox_training():
"""Start YOLOX training"""
"""Generate JSONs, exp.py, and add training to queue"""
try:
data = request.get_json()
project_id = data.get('project_id')
training_id = data.get('training_id')
# Get project name
if not project_id or not training_id:
return jsonify({'message': 'Missing project_id or training_id'}), 400
# Get training record
training = Training.query.get(training_id)
if not training:
return jsonify({'message': f'Training {training_id} not found'}), 404
details_id = training.project_details_id
# Step 1: Generate COCO JSON files
# IMPORTANT: Pass training_id (not details_id) so JSON files are generated
# in the same location where exp.py expects them
from services.generate_json_yolox import generate_training_json
print(f'Generating COCO JSON for training {training_id}...')
generate_training_json(training_id)
# Step 2: Generate exp.py
from services.generate_yolox_exp import save_yolox_exp
from services.settings_service import get_setting
training_project = TrainingProject.query.get(project_id)
project_name = training_project.title.replace(' ', '_') if training_project.title else f'project_{project_id}'
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{project_id}'
# Look up training row
training_row = Training.query.get(training_id)
if not training_row:
training_row = Training.query.filter_by(project_details_id=training_id).first()
# Use training name + id for folder to support multiple trainings per project
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
training_folder_name = training_folder_name.replace(' ', '_')
if not training_row:
return jsonify({'error': f'Training row not found for id or project_details_id {training_id}'}), 404
output_base_path = get_setting('yolox_output_path', './backend')
out_dir = os.path.join(output_base_path, project_name, training_folder_name)
os.makedirs(out_dir, exist_ok=True)
project_details_id = training_row.project_details_id
exp_file_path = os.path.join(out_dir, 'exp.py')
print(f'Generating exp.py at {exp_file_path}...')
save_yolox_exp(training_id, exp_file_path)
# Path to exp.py
out_dir = os.path.join(os.path.dirname(__file__), '..', project_name, str(project_details_id))
exp_src = os.path.join(out_dir, 'exp.py')
# Step 3: Build training command
print(f'Preparing training command for training {training_id}...')
if not os.path.exists(exp_src):
return jsonify({'error': f'exp.py not found at {exp_src}'}), 500
# Get YOLOX configuration from settings
yolox_main_dir = get_setting('yolox_path', '/home/kitraining/Yolox/YOLOX-main')
yolox_venv = get_setting('yolox_venv_path', '/home/kitraining/Yolox/yolox_venv/bin/activate')
# YOLOX configuration
yolox_main_dir = '/home/kitraining/Yolox/YOLOX-main'
yolox_venv = '/home/kitraining/Yolox/yolox_venv/bin/activate'
# Detect platform and build appropriate command
import platform
is_windows = platform.system() == 'Windows'
# Determine model argument
model_arg = ''
cmd = ''
if (training_row.transfer_learning and
isinstance(training_row.transfer_learning, str) and
training_row.transfer_learning.lower() == 'coco'):
model_arg = f' -c /home/kitraining/Yolox/YOLOX-main/pretrained/{training_row.selected_model}'
cmd = f'bash -c \'source {yolox_venv} && python tools/train.py -f {exp_src} -d 1 -b 8 --fp16 -o {model_arg}.pth --cache\''
elif (training_row.selected_model and
training_row.selected_model.lower() == 'coco' and
(not training_row.transfer_learning or training_row.transfer_learning == False)):
model_arg = f' -c /pretrained/{training_row.selected_model}'
cmd = f'bash -c \'source {yolox_venv} && python tools/train.py -f {exp_src} -d 1 -b 8 --fp16 -o {model_arg}.pth --cache\''
if (training.transfer_learning and
isinstance(training.transfer_learning, str) and
training.transfer_learning.lower() == 'coco'):
# Use yolox_path setting to construct pretrained model path
model_path = os.path.join(yolox_main_dir, 'pretrained', f'{training.selected_model}.pth')
model_path = model_path.replace('\\', '/') # Use forward slashes for command line
model_arg = f'-c {model_path}'
elif (training.selected_model and
training.selected_model.lower() == 'coco' and
(not training.transfer_learning or training.transfer_learning == False)):
# Use yolox_path setting to construct pretrained model path
model_path = os.path.join(yolox_main_dir, 'pretrained', f'{training.selected_model}.pth')
model_path = model_path.replace('\\', '/') # Use forward slashes for command line
model_arg = f'-c {model_path}'
# Build base training arguments
train_args = f'-f {exp_file_path} -d 1 -b 8 --fp16 --cache'
if model_arg:
train_args += f' {model_arg} -o'
# Build platform-specific command
if is_windows:
# Windows: Use call to activate venv, then run python
# If venv path doesn't end with .bat, assume it needs Scripts\activate.bat
if not yolox_venv.endswith('.bat'):
venv_activate = os.path.join(yolox_venv, 'Scripts', 'activate.bat')
else:
cmd = f'bash -c \'source {yolox_venv} && python tools/train.py -f {exp_src} -d 1 -b 8 --fp16 --cache\''
venv_activate = yolox_venv
cmd = f'cmd /c ""{venv_activate}" && python tools\\train.py {train_args}"'
else:
# Linux: Use bash with source to activate the venv
venv_activate = os.path.join(yolox_venv, 'bin', 'activate')
cmd = f'bash -c "source {venv_activate} && python tools/train.py {train_args}"'
print(cmd)
print(f'Training command: {cmd}')
# Start training in background
subprocess.Popen(cmd, shell=True, cwd=yolox_main_dir)
# Step 4: Add to training queue
from services.training_queue import training_queue
training_queue.add_to_queue(training_id, cmd, yolox_main_dir)
return jsonify({'message': 'Training started'})
return jsonify({
'message': f'Training {training_id} added to queue',
'exp_path': exp_file_path
})
except Exception as err:
return jsonify({'error': 'Failed to start training', 'details': str(err)}), 500
print(f'Error starting YOLOX training: {err}')
import traceback
traceback.print_exc()
return jsonify({'message': 'Failed to start training', 'error': str(err)}), 500
@api_bp.route('/training-status', methods=['GET'])
def get_training_status():
"""Get current training queue status"""
try:
from services.training_queue import training_queue
status = training_queue.get_status()
return jsonify(status)
except Exception as err:
print(f'Error getting training status: {err}')
return jsonify({'current': None, 'queue': []}), 500
@api_bp.route('/training-log', methods=['GET'])
def training_log():
@@ -159,6 +220,8 @@ def training_log():
def create_training_project():
"""Create a new training project"""
try:
from models.ProjectClass import ProjectClass
title = request.form.get('title')
description = request.form.get('description')
classes = json.loads(request.form.get('classes', '[]'))
@@ -171,15 +234,26 @@ def create_training_project():
project_image = file.read()
project_image_type = file.content_type
# Create project without classes field
project = TrainingProject(
title=title,
description=description,
classes=classes,
project_image=project_image,
project_image_type=project_image_type
)
db.session.add(project)
db.session.flush() # Get project_id before commit
# Add classes to ProjectClass table
for index, class_name in enumerate(classes):
project_class = ProjectClass(
project_id=project.project_id,
class_name=class_name,
display_order=index
)
db.session.add(project_class)
db.session.commit()
return jsonify({'message': 'Project created!'})
@@ -248,23 +322,49 @@ def get_label_studio_projects():
def create_training_project_details():
"""Create TrainingProjectDetails"""
try:
from models.AnnotationProjectMapping import AnnotationProjectMapping
from models.ClassMapping import ClassMapping
data = request.get_json()
project_id = data.get('project_id')
annotation_projects = data.get('annotation_projects')
class_map = data.get('class_map')
annotation_projects = data.get('annotation_projects') # Array of project IDs
class_map = data.get('class_map') # Dict: {source: target}
description = data.get('description')
if not project_id or annotation_projects is None:
return jsonify({'message': 'Missing required fields'}), 400
# Create TrainingProjectDetails without JSON fields
details = TrainingProjectDetails(
project_id=project_id,
annotation_projects=annotation_projects,
class_map=class_map,
description=description
description_text=description
)
db.session.add(details)
db.session.flush() # Get details.id
# Add annotation project mappings
for ls_project_id in annotation_projects:
mapping = AnnotationProjectMapping(
project_details_id=details.id,
label_studio_project_id=ls_project_id
)
db.session.add(mapping)
# Add class mappings if provided
if class_map:
for source_class, target_class in class_map.items():
# For initial creation, we don't have per-project mappings yet
# Will be replaced when user sets up mappings in UI
mapping = ClassMapping(
project_details_id=details.id,
label_studio_project_id=annotation_projects[0] if annotation_projects else 0,
source_class=source_class,
target_class=target_class
)
db.session.add(mapping)
db.session.commit()
db.session.commit()
return jsonify({'message': 'TrainingProjectDetails created', 'details': details.to_dict()})
@@ -278,21 +378,44 @@ def get_training_project_details():
"""Get all TrainingProjectDetails"""
try:
details = TrainingProjectDetails.query.all()
return jsonify([d.to_dict() for d in details])
result = []
for d in details:
try:
result.append(d.to_dict())
except Exception as e:
print(f'Error serializing detail {d.id}: {e}')
# Return basic info if full serialization fails
result.append({
'id': d.id,
'project_id': d.project_id,
'description': d.description_text,
'annotation_projects': [],
'class_map': {}
})
return jsonify(result)
except Exception as error:
print(f'Error fetching training project details: {error}')
return jsonify({'message': 'Failed to fetch TrainingProjectDetails', 'error': str(error)}), 500
@api_bp.route('/training-project-details', methods=['PUT'])
def update_training_project_details():
"""Update class_map and description in TrainingProjectDetails"""
try:
data = request.get_json()
project_id = data.get('project_id')
class_map = data.get('class_map')
description = data.get('description')
from models.ClassMapping import ClassMapping
if not project_id or not class_map or not description:
data = request.get_json()
print(f'[DEBUG] Received PUT data: {data}')
project_id = data.get('project_id')
class_map_data = data.get('class_map') # Array: [[labelStudioProjectId, [[class, target], ...]], ...]
description_data = data.get('description') # Array: [[projectId, desc], ...]
print(f'[DEBUG] project_id: {project_id}')
print(f'[DEBUG] class_map_data: {class_map_data}')
print(f'[DEBUG] description_data: {description_data}')
if not project_id or class_map_data is None or description_data is None:
return jsonify({'message': 'Missing required fields'}), 400
details = TrainingProjectDetails.query.filter_by(project_id=project_id).first()
@@ -300,14 +423,43 @@ def update_training_project_details():
if not details:
return jsonify({'message': 'TrainingProjectDetails not found'}), 404
details.class_map = class_map
details.description = description
# Update description - combine all descriptions
combined_description = '\n\n'.join([desc[1] for desc in description_data if len(desc) > 1 and desc[1]])
details.description_text = combined_description
# Delete existing class mappings
ClassMapping.query.filter_by(project_details_id=details.id).delete()
# Add new class mappings - iterate through all label studio projects
# class_map_data format: [[labelStudioProjectId, [[class, target], ...]], ...]
for project_mapping in class_map_data:
if len(project_mapping) >= 2:
label_studio_project_id = project_mapping[0]
class_mappings = project_mapping[1] # [[class1, target1], [class2, target2], ...]
for class_pair in class_mappings:
if len(class_pair) >= 2:
source_class = class_pair[0]
target_class = class_pair[1]
# Create mapping with label_studio_project_id
mapping = ClassMapping(
project_details_id=details.id,
label_studio_project_id=label_studio_project_id,
source_class=source_class,
target_class=target_class
)
db.session.add(mapping)
db.session.commit()
return jsonify({'message': 'Class map and description updated', 'details': details.to_dict()})
except Exception as error:
db.session.rollback()
print(f'[ERROR] Failed to update training project details: {error}')
import traceback
traceback.print_exc()
return jsonify({'message': 'Failed to update class map or description', 'error': str(error)}), 500
@api_bp.route('/yolox-settings', methods=['POST'])
@@ -332,14 +484,13 @@ def yolox_settings():
details = TrainingProjectDetails.query.filter_by(project_id=project_id).first()
if not details:
# Create TrainingProjectDetails without JSON fields
details = TrainingProjectDetails(
project_id=project_id,
annotation_projects=[],
class_map=None,
description=None
description_text=None
)
db.session.add(details)
db.session.commit()
db.session.flush() # Get details.id
settings['project_details_id'] = details.id
@@ -348,17 +499,26 @@ def yolox_settings():
settings['activation'] = settings['act']
del settings['act']
# Type conversions
numeric_fields = [
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr',
'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum',
'print_interval', 'eval_interval', 'test_conf', 'nmsthre',
'multiscale_range', 'degrees', 'translate', 'shear',
'train', 'valid', 'test'
# Type conversions - Integer fields
integer_fields = [
'max_epoch', 'warmup_epochs', 'no_aug_epochs', 'print_interval',
'eval_interval', 'multiscale_range', 'data_num_workers', 'num_classes'
]
for field in numeric_fields:
if field in settings:
for field in integer_fields:
if field in settings and settings[field] not in [None, '']:
settings[field] = int(float(settings[field])) # Convert via float to handle "10.0" strings
# Type conversions - Float fields
float_fields = [
'depth', 'width', 'warmup_lr', 'min_lr_ratio', 'weight_decay',
'momentum', 'test_conf', 'nmsthre', 'degrees', 'translate',
'shear', 'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob',
'basic_lr_per_img', 'train', 'valid', 'test'
]
for field in float_fields:
if field in settings and settings[field] not in [None, '']:
settings[field] = float(settings[field])
# Boolean conversions
@@ -488,6 +648,19 @@ def get_trainings():
except Exception as error:
return jsonify({'message': 'Failed to fetch trainings', 'error': str(error)}), 500
@api_bp.route('/trainings/<int:id>', methods=['GET'])
def get_training(id):
"""Get a single training by id"""
try:
training = Training.query.get(id)
if training:
return jsonify(training.to_dict())
else:
return jsonify({'message': 'Training not found'}), 404
except Exception as error:
return jsonify({'message': 'Failed to fetch training', 'error': str(error)}), 500
@api_bp.route('/trainings/<int:id>', methods=['DELETE'])
def delete_training(id):
"""Delete a training by id"""
@@ -539,3 +712,185 @@ def get_base_config(model_name):
return jsonify(config)
except Exception as error:
return jsonify({'message': f'Failed to load base config for {model_name}', 'error': str(error)}), 404
# Settings endpoints
@api_bp.route('/settings', methods=['GET'])
def get_settings():
"""Get all settings"""
from services.settings_service import get_all_settings_detailed
settings = get_all_settings_detailed()
return jsonify(settings)
@api_bp.route('/settings/<key>', methods=['GET'])
def get_setting(key):
"""Get a specific setting"""
from services.settings_service import get_setting as get_setting_value
from models.Settings import Settings
setting = Settings.query.filter_by(key=key).first()
if setting:
return jsonify(setting.to_dict())
else:
return jsonify({'message': f'Setting {key} not found'}), 404
@api_bp.route('/settings', methods=['POST'])
def update_settings():
"""Update multiple settings"""
try:
data = request.get_json()
from services.settings_service import set_setting
for key, value in data.items():
set_setting(key, value)
return jsonify({'message': 'Settings updated successfully'})
except Exception as error:
return jsonify({'message': 'Failed to update settings', 'error': str(error)}), 500
@api_bp.route('/settings/<key>', methods=['PUT'])
def update_setting(key):
"""Update a specific setting"""
try:
data = request.get_json()
value = data.get('value')
description = data.get('description')
from services.settings_service import set_setting
setting = set_setting(key, value, description)
return jsonify(setting.to_dict())
except Exception as error:
return jsonify({'message': f'Failed to update setting {key}', 'error': str(error)}), 500
@api_bp.route('/settings/test/labelstudio', methods=['POST'])
def test_labelstudio_connection():
"""Test Label Studio connection"""
try:
data = request.get_json()
api_url = data.get('api_url')
api_token = data.get('api_token')
if not api_url or not api_token:
return jsonify({'success': False, 'message': 'Missing api_url or api_token'}), 400
import requests
response = requests.get(
f'{api_url}/projects/',
headers={'Authorization': f'Token {api_token}'},
timeout=5
)
if response.ok:
projects = response.json()
return jsonify({
'success': True,
'message': f'Connection successful! Found {len(projects.get("results", projects))} projects.'
})
else:
return jsonify({
'success': False,
'message': f'Connection failed: {response.status_code} {response.reason}'
}), 400
except requests.exceptions.Timeout:
return jsonify({'success': False, 'message': 'Connection timeout'}), 400
except requests.exceptions.ConnectionError:
return jsonify({'success': False, 'message': 'Cannot connect to Label Studio'}), 400
except Exception as error:
return jsonify({'success': False, 'message': str(error)}), 500
@api_bp.route('/settings/test/yolox', methods=['POST'])
def test_yolox_path():
"""Test YOLOX path and venv path validity"""
try:
data = request.get_json()
yolox_path = data.get('yolox_path')
yolox_venv_path = data.get('yolox_venv_path')
if not yolox_path:
return jsonify({'success': False, 'message': 'Missing yolox_path'}), 400
# Check if YOLOX path exists
if not os.path.exists(yolox_path):
return jsonify({'success': False, 'message': 'YOLOX path does not exist'}), 400
# Check for key YOLOX files/directories
required_items = ['yolox', 'exps', 'tools']
found_items = []
missing_items = []
for item in required_items:
item_path = os.path.join(yolox_path, item)
if os.path.exists(item_path):
found_items.append(item)
else:
missing_items.append(item)
if len(found_items) < 2: # At least 2 out of 3 key items required
return jsonify({
'success': False,
'message': f'Invalid YOLOX path. Missing: {", ".join(missing_items)}',
'found': found_items,
'missing': missing_items
}), 400
# Check venv path if provided
venv_message = ''
if yolox_venv_path:
venv_valid = False
venv_details = []
# Normalize path
venv_path_normalized = os.path.normpath(yolox_venv_path)
# Check if it's an activation script (Linux/Mac: bin/activate, Windows: Scripts/activate.bat or Scripts/Activate.ps1)
if os.path.isfile(venv_path_normalized):
# Direct path to activation script
if 'activate' in os.path.basename(venv_path_normalized).lower():
venv_valid = True
venv_details.append(f'Activation script found: {os.path.basename(venv_path_normalized)}')
else:
return jsonify({
'success': False,
'message': 'Venv path points to a file but not an activation script'
}), 400
elif os.path.isdir(venv_path_normalized):
# Check if it's a venv directory
# Look for activation scripts in common locations
possible_activations = [
os.path.join(venv_path_normalized, 'bin', 'activate'), # Linux/Mac
os.path.join(venv_path_normalized, 'Scripts', 'activate.bat'), # Windows CMD
os.path.join(venv_path_normalized, 'Scripts', 'Activate.ps1'), # Windows PowerShell
os.path.join(venv_path_normalized, 'Scripts', 'activate'), # Windows Git Bash
]
found_activations = []
for act_path in possible_activations:
if os.path.exists(act_path):
found_activations.append(os.path.basename(act_path))
venv_valid = True
if venv_valid:
venv_details.append(f'Venv directory valid. Found: {", ".join(found_activations)}')
else:
return jsonify({
'success': False,
'message': 'Venv directory does not contain activation scripts'
}), 400
else:
return jsonify({
'success': False,
'message': 'Venv path does not exist'
}), 400
venv_message = ' ' + '. '.join(venv_details)
return jsonify({
'success': True,
'message': f'Valid YOLOX installation found. Found: {", ".join(found_items)}.{venv_message}',
'found': found_items,
'missing': missing_items
})
except Exception as error:
return jsonify({'success': False, 'message': str(error)}), 500

0
backend/server.js Normal file → Executable file
View File

0
backend/services/__init__.py Normal file → Executable file
View File

View File

@@ -1,92 +0,0 @@
const API_URL = 'http://192.168.1.19:8080/api';
const API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f';
const fetch = require('node-fetch');
async function fetchLableStudioProject(projectid) {
// 1. Trigger export
const exportUrl = `${API_URL}/projects/${projectid}/export?exportType=JSON_MIN`;
const headers = { Authorization: `Token ${API_TOKEN}` };
let res = await fetch(exportUrl, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to trigger export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to trigger export: ${res.status} ${res.statusText}`);
}
let data = await res.json();
// If data is an array, it's ready
if (Array.isArray(data)) return data;
// If not, poll for the export file
let fileUrl = data.download_url || data.url || null;
let tries = 0;
while (!fileUrl && tries < 20) {
await new Promise(r => setTimeout(r, 2000));
res = await fetch(exportUrl, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to poll export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to poll export: ${res.status} ${res.statusText}`);
}
data = await res.json();
fileUrl = data.download_url || data.url || null;
tries++;
}
if (!fileUrl) throw new Error('Label Studio export did not become ready');
// 2. Download the export file
res = await fetch(fileUrl.startsWith('http') ? fileUrl : `${API_URL.replace('/api','')}${fileUrl}`, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to download export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to download export: ${res.status} ${res.statusText}`);
}
return await res.json();
}
async function fetchProjectIdsAndTitles() {
try {
const response = await fetch(`${API_URL}/projects/`, {
headers: {
'Authorization': `Token ${API_TOKEN}`,
'Content-Type': 'application/json'
}
});
if (!response.ok) {
let errorText = await response.text().catch(() => '');
console.error(`Failed to fetch projects: ${response.status} ${response.statusText} - ${errorText}`);
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
if (!data.results || !Array.isArray(data.results)) {
throw new Error('API response does not contain results array');
}
// Extract id and title from each project
const projects = data.results.map(project => ({
id: project.id,
title: project.title
}));
console.log(projects)
return projects;
} catch (error) {
console.error('Failed to fetch projects:', error);
return [];
}
}
module.exports = { fetchLableStudioProject, fetchProjectIdsAndTitles };
//getLableStudioProject(20)
//fetchProjectIdsAndTitles()

12
backend/services/fetch_labelstudio.py Normal file → Executable file
View File

@@ -1,11 +1,17 @@
import requests
import time
from services.settings_service import get_setting
API_URL = 'http://192.168.1.19:8080/api'
API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f'
def get_api_credentials():
"""Get Label Studio API credentials from settings"""
api_url = get_setting('labelstudio_api_url', 'http://192.168.1.19:8080/api')
api_token = get_setting('labelstudio_api_token', 'c1cef980b7c73004f4ee880a42839313b863869f')
return api_url, api_token
def fetch_label_studio_project(project_id):
"""Fetch Label Studio project annotations"""
API_URL, API_TOKEN = get_api_credentials()
export_url = f'{API_URL}/projects/{project_id}/export?exportType=JSON_MIN'
headers = {'Authorization': f'Token {API_TOKEN}'}
@@ -53,6 +59,8 @@ def fetch_label_studio_project(project_id):
def fetch_project_ids_and_titles():
"""Fetch all Label Studio project IDs and titles"""
API_URL, API_TOKEN = get_api_credentials()
try:
response = requests.get(
f'{API_URL}/projects/',

View File

@@ -1,176 +0,0 @@
const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js')
const LabelStudioProject = require('../models/LabelStudioProject.js')
const Annotation = require('../models/Annotation.js')
const Images = require('../models/Images.js')
const fs = require('fs');
async function generateTrainingJson(trainingId){
// trainingId is now project_details_id
const trainingProjectDetails = await TrainingProjectDetails.findByPk(trainingId);
if (!trainingProjectDetails) throw new Error('No TrainingProjectDetails found for project_details_id ' + trainingId);
const detailsObj = trainingProjectDetails.get({ plain: true });
// Get parent project for name
const trainingProject = await TrainingProject.findByPk(detailsObj.project_id);
// Get split percentages (assume they are stored as train_percent, valid_percent, test_percent)
const trainPercent = detailsObj.train_percent || 85;
const validPercent = detailsObj.valid_percent || 10;
const testPercent = detailsObj.test_percent || 5;
let cocoImages = [];
let cocoAnnotations = [];
let cocoCategories = [];
let categoryMap = {};
let categoryId = 0;
let imageid = 0;
let annotationid = 0;
for (const cls of detailsObj.class_map) {
const asgMap = [];
const listAsg = cls[1];
for(const asg of listAsg){
asgMap.push ({ original: asg[0], mapped: asg[1] });
// Build category list and mapping
if (asg[1] && !(asg[1] in categoryMap)) {
categoryMap[asg[1]] = categoryId;
cocoCategories.push({ id: categoryId, name: asg[1], supercategory: '' });
categoryId++;
}
}
const images = await Images.findAll({ where: { project_id: cls[0] } });
for(const image of images){
imageid += 1;
let fileName = image.image_path;
if (fileName.includes('%20')) {
fileName = fileName.replace(/%20/g, ' ');
}
if (fileName && fileName.startsWith('/data/local-files/?d=')) {
fileName = fileName.replace('/data/local-files/?d=', '');
fileName = fileName.replace('/home/kitraining/home/kitraining/', '');
}
if (fileName && fileName.startsWith('home/kitraining/To_Annotate/')) {
fileName = fileName.replace('home/kitraining/To_Annotate/','');
}
// Get annotations for this image
const annotations = await Annotation.findAll({ where: { image_id: image.image_id } });
// Use image.width and image.height from DB (populated from original_width/original_height)
cocoImages.push({
id: imageid,
file_name: fileName,
width: image.width || 0,
height: image.height || 0
});
for (const annotation of annotations) {
// Translate class name using asgMap
let mappedClass = annotation.Label;
for (const mapEntry of asgMap) {
if (annotation.Label === mapEntry.original) {
mappedClass = mapEntry.mapped;
break;
}
}
// Only add annotation if mappedClass is valid
if (mappedClass && mappedClass in categoryMap) {
annotationid += 1;
let area = 0;
if (annotation.width && annotation.height) {
area = annotation.width * annotation.height;
}
cocoAnnotations.push({
id: annotationid,
image_id: imageid,
category_id: categoryMap[mappedClass],
bbox: [annotation.x, annotation.y, annotation.width, annotation.height],
area: area,
iscrowd: annotation.iscrowd || 0
});
}
}
}
}
// Shuffle images for random split using seed
function seededRandom(seed) {
let x = Math.sin(seed++) * 10000;
return x - Math.floor(x);
}
function shuffle(array, seed) {
for (let i = array.length - 1; i > 0; i--) {
const j = Math.floor(seededRandom(seed + i) * (i + 1));
[array[i], array[j]] = [array[j], array[i]];
}
}
// Use seed from detailsObj if present, else default to 42
const splitSeed = detailsObj.seed !== undefined && detailsObj.seed !== null ? Number(detailsObj.seed) : 42;
shuffle(cocoImages, splitSeed);
// Split images
const totalImages = cocoImages.length;
const trainCount = Math.floor(totalImages * trainPercent / 100);
const validCount = Math.floor(totalImages * validPercent / 100);
const testCount = totalImages - trainCount - validCount;
const trainImages = cocoImages.slice(0, trainCount);
const validImages = cocoImages.slice(trainCount, trainCount + validCount);
const testImages = cocoImages.slice(trainCount + validCount);
// Helper to get image ids for each split
const trainImageIds = new Set(trainImages.map(img => img.id));
const validImageIds = new Set(validImages.map(img => img.id));
const testImageIds = new Set(testImages.map(img => img.id));
// Split annotations
const trainAnnotations = cocoAnnotations.filter(ann => trainImageIds.has(ann.image_id));
const validAnnotations = cocoAnnotations.filter(ann => validImageIds.has(ann.image_id));
const testAnnotations = cocoAnnotations.filter(ann => testImageIds.has(ann.image_id));
// Build final COCO JSONs with info section
const buildCocoJson = (images, annotations, categories) => ({
images,
annotations,
categories
});
// Build COCO JSONs with info section
const trainJson = buildCocoJson(trainImages, trainAnnotations, cocoCategories);
const validJson = buildCocoJson(validImages, validAnnotations, cocoCategories);
const testJson = buildCocoJson(testImages, testAnnotations, cocoCategories);
// Create output directory: projectname/trainingid/annotations
const projectName = trainingProject && trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${detailsObj.project_id}`;
const outDir = `${projectName}/${trainingId}`;
const annotationsDir = `/home/kitraining/To_Annotate/annotations`;
if (!fs.existsSync(annotationsDir)) {
fs.mkdirSync(annotationsDir, { recursive: true });
}
// Write to files in the annotations directory
const trainPath = `${annotationsDir}/coco_project_${trainingId}_train.json`;
const validPath = `${annotationsDir}/coco_project_${trainingId}_valid.json`;
const testPath = `${annotationsDir}/coco_project_${trainingId}_test.json`;
fs.writeFileSync(trainPath, JSON.stringify(trainJson, null, 2));
fs.writeFileSync(validPath, JSON.stringify(validJson, null, 2));
fs.writeFileSync(testPath, JSON.stringify(testJson, null, 2));
console.log(`COCO JSON splits written to ${annotationsDir} for trainingId ${trainingId}`);
// Also generate inference exp.py in the same output directory as exp.py (project folder in workspace)
const { generateYoloxInferenceExp } = require('./generate-yolox-exp');
const path = require('path');
const projectFolder = path.join(__dirname, '..', projectName, String(trainingId));
if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true });
}
const inferenceExpPath = path.join(projectFolder, 'exp_infer.py');
generateYoloxInferenceExp(trainingId).then(expContent => {
fs.writeFileSync(inferenceExpPath, expContent);
console.log(`Inference exp.py written to ${inferenceExpPath}`);
}).catch(err => {
console.error('Failed to generate inference exp.py:', err);
});
}
module.exports = {generateTrainingJson};

View File

@@ -1,135 +0,0 @@
const fs = require('fs');
const path = require('path');
const Training = require('../models/training.js');
const TrainingProject = require('../models/TrainingProject.js');
// Remove Python comments and legacy code
const exp_names = [
'YOLOX-s',
'YOLOX-m',
'YOLOX-l',
'YOLOX-x',
'YOLOX-Darknet53', //todo
'YOLOX-Nano',
'YOLOX-Tiny'
]
//TODO: Clean up generation of exp_names.py and remove second exp creation!!!
// Refactored: Accept trainingId, fetch info from DB
async function generateYoloxExp(trainingId) {
// Fetch training row from DB by project_details_id if not found by PK
let training = await Training.findByPk(trainingId);
if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } });
}
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// If transfer_learning is 'coco', just return the path to the default exp.py
if (training.transfer_learning === 'coco') {
const selectedModel = training.selected_model.toLowerCase().replace('-', '_');
const expSourcePath = `/home/kitraining/Yolox/YOLOX-main/exps/default/${selectedModel}.py`;
if (!fs.existsSync(expSourcePath)) {
throw new Error(`Default exp.py not found for model: ${selectedModel} at ${expSourcePath}`);
}
// Copy to project folder (e.g., /home/kitraining/coco_tool/backend/project_XX/YY/exp.py)
const projectDetailsId = training.project_details_id;
const projectFolder = path.resolve(__dirname, `../project_23/${projectDetailsId}`);
if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true });
}
const expDestPath = path.join(projectFolder, 'exp.py');
fs.copyFileSync(expSourcePath, expDestPath);
return { type: 'default', expPath: expDestPath };
}
// If transfer_learning is 'sketch', generate a custom exp.py as before
if (training.transfer_learning === 'sketch') {
// ...existing custom exp.py generation logic here (copy from previous implementation)...
// For brevity, you can call generateYoloxInferenceExp or similar here, or inline the logic.
// Example:
const expContent = await generateYoloxInferenceExp(trainingId);
return { type: 'custom', expContent };
}
throw new Error('Unknown transfer_learning type: ' + training.transfer_learning);
}
async function saveYoloxExp(trainingId, outPath) {
const expResult = await generateYoloxExp(trainingId);
if (expResult.type === 'custom' && expResult.expContent) {
fs.writeFileSync(outPath, expResult.expContent);
return outPath;
} else if (expResult.type === 'default' && expResult.expPath) {
// Optionally copy the file if outPath is different
if (expResult.expPath !== outPath) {
fs.copyFileSync(expResult.expPath, outPath);
}
return outPath;
} else {
throw new Error('Unknown expResult type or missing content');
}
}
// Generate a second exp.py for inference, using the provided template and DB values
async function generateYoloxInferenceExp(trainingId, options = {}) {
let training = await Training.findByPk(trainingId);
if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } });
}
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// Always use the trainingId (project_details_id) for annotation file names
const projectDetailsId = training.project_details_id;
const dataDir = options.data_dir || '/home/kitraining/To_Annotate/';
const trainAnn = options.train_ann || `coco_project_${trainingId}_train.json`;
const valAnn = options.val_ann || `coco_project_${trainingId}_valid.json`;
const testAnn = options.test_ann || `coco_project_${trainingId}_test.json`;
// Get num_classes from TrainingProject.classes JSON
let numClasses = 80;
try {
const trainingProject = await TrainingProject.findByPk(projectDetailsId);
if (trainingProject && trainingProject.classes) {
let classesArr = trainingProject.classes;
if (typeof classesArr === 'string') {
classesArr = JSON.parse(classesArr);
}
if (Array.isArray(classesArr)) {
numClasses = classesArr.filter(c => c !== null && c !== undefined && c !== '').length;
} else if (typeof classesArr === 'object' && classesArr !== null) {
numClasses = Object.keys(classesArr).filter(k => classesArr[k] !== null && classesArr[k] !== undefined && classesArr[k] !== '').length;
}
}
} catch (e) {
console.warn('Could not determine num_classes from TrainingProject.classes:', e);
}
const depth = options.depth || training.depth || 1.00;
const width = options.width || training.width || 1.00;
const inputSize = options.input_size || training.input_size || [640, 640];
const mosaicScale = options.mosaic_scale || training.mosaic_scale || [0.1, 2];
const randomSize = options.random_size || training.random_size || [10, 20];
const testSize = options.test_size || training.test_size || [640, 640];
const expName = options.exp_name || 'inference_exp';
const enableMixup = options.enable_mixup !== undefined ? options.enable_mixup : false;
let expContent = '';
expContent += `#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.data_dir = "${dataDir}"\n self.train_ann = "${trainAnn}"\n self.val_ann = "${valAnn}"\n self.test_ann = "coco_project_${trainingId}_test.json"\n self.num_classes = ${numClasses}\n`;
// Set pretrained_ckpt if transfer_learning is 'coco'
if (training.transfer_learning && typeof training.transfer_learning === 'string' && training.transfer_learning.toLowerCase() === 'coco') {
const yoloxBaseDir = '/home/kitraining/Yolox/YOLOX-main';
const selectedModel = training.selected_model ? training.selected_model.replace(/\.pth$/i, '') : '';
if (selectedModel) {
expContent += ` self.pretrained_ckpt = r'${yoloxBaseDir}/pretrained/${selectedModel}.pth'\n`;
}
}
expContent += ` self.depth = ${depth}\n self.width = ${width}\n self.input_size = (${Array.isArray(inputSize) ? inputSize.join(', ') : inputSize})\n self.mosaic_scale = (${Array.isArray(mosaicScale) ? mosaicScale.join(', ') : mosaicScale})\n self.random_size = (${Array.isArray(randomSize) ? randomSize.join(', ') : randomSize})\n self.test_size = (${Array.isArray(testSize) ? testSize.join(', ') : testSize})\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]\n self.enable_mixup = ${enableMixup ? 'True' : 'False'}\n`;
return expContent;
}
// Save inference exp.py to a custom path
async function saveYoloxInferenceExp(trainingId, outPath, options = {}) {
const expContent = await generateYoloxInferenceExp(trainingId, options);
fs.writeFileSync(outPath, expContent);
return outPath;
}
module.exports = { generateYoloxExp, saveYoloxExp, generateYoloxInferenceExp, saveYoloxInferenceExp };

181
backend/services/generate_json_yolox.py Normal file → Executable file
View File

@@ -7,18 +7,52 @@ from models.Images import Image
from models.Annotation import Annotation
def generate_training_json(training_id):
"""Generate COCO JSON for training, validation, and test sets"""
# training_id is now project_details_id
training_project_details = TrainingProjectDetails.query.get(training_id)
"""Generate COCO JSON for training, validation, and test sets
Args:
training_id: Can be either a Training.id or TrainingProjectDetails.id
Function will automatically detect which one and find the correct details_id
"""
from models.training import Training
# First, try to get as a Training record
training_record = Training.query.get(training_id)
if training_record:
# It's a Training.id - use its project_details_id
details_id = training_record.project_details_id
print(f'[generate_training_json] Using training_id={training_id}, mapped to project_details_id={details_id}')
else:
# Try as TrainingProjectDetails.id directly
details_id = training_id
print(f'[generate_training_json] Using training_id={training_id} as project_details_id directly')
training_project_details = TrainingProjectDetails.query.get(details_id)
if not training_project_details:
raise Exception(f'No TrainingProjectDetails found for project_details_id {training_id}')
raise Exception(f'No TrainingProjectDetails found for id {training_id} (details_id: {details_id})')
details_obj = training_project_details.to_dict()
# Get parent project for name
training_project = TrainingProject.query.get(details_obj['project_id'])
# Get the data directory setting for image paths
from services.settings_service import get_setting
data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
# Fix UNC path if it's missing the \\ prefix
# Check if it looks like a UNC path without proper prefix (e.g., "192.168.1.19\...")
if data_dir and not data_dir.startswith('\\\\') and not data_dir.startswith('/'):
# Check if it starts with an IP address pattern
import re
if re.match(r'^\d+\.\d+\.\d+\.\d+[/\\]', data_dir):
data_dir = '\\\\' + data_dir
# Ensure data_dir ends with separator
if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
data_dir += os.sep
# Get split percentages (default values if not set)
train_percent = details_obj.get('train_percent', 85)
valid_percent = details_obj.get('valid_percent', 10)
@@ -32,51 +66,113 @@ def generate_training_json(training_id):
image_id = 0
annotation_id = 0
for cls in details_obj['class_map']:
asg_map = []
list_asg = cls[1]
# Build category list and mapping from class_map dictionary {source: target}
class_map = details_obj.get('class_map', {})
for asg in list_asg:
asg_map.append({'original': asg[0], 'mapped': asg[1]})
# Build category list and mapping
if asg[1] and asg[1] not in category_map:
category_map[asg[1]] = category_id
coco_categories.append({'id': category_id, 'name': asg[1], 'supercategory': ''})
for source_class, target_class in class_map.items():
if target_class and target_class not in category_map:
category_map[target_class] = category_id
coco_categories.append({'id': category_id, 'name': target_class, 'supercategory': ''})
category_id += 1
# Get images for this project
images = Image.query.filter_by(project_id=cls[0]).all()
# Get all annotation projects (Label Studio project IDs)
annotation_projects = details_obj.get('annotation_projects', [])
# Get class mappings from database grouped by Label Studio project
from models.ClassMapping import ClassMapping
all_mappings = ClassMapping.query.filter_by(project_details_id=training_id).all()
# Group mappings by Label Studio project ID
mappings_by_project = {}
for mapping in all_mappings:
ls_proj_id = mapping.label_studio_project_id
if ls_proj_id not in mappings_by_project:
mappings_by_project[ls_proj_id] = {}
mappings_by_project[ls_proj_id][mapping.source_class] = mapping.target_class
# Also add target class to category map if not present
if mapping.target_class and mapping.target_class not in category_map:
category_map[mapping.target_class] = category_id
coco_categories.append({'id': category_id, 'name': mapping.target_class, 'supercategory': ''})
category_id += 1
# Iterate through each annotation project to collect images and annotations
for ls_project_id in annotation_projects:
# Get images for this Label Studio project
images = Image.query.filter_by(project_id=ls_project_id).all()
for image in images:
image_id += 1
file_name = image.image_path
# Clean up file path
# Clean up file path from Label Studio format
if '%20' in file_name:
file_name = file_name.replace('%20', ' ')
if file_name and file_name.startswith('/data/local-files/?d='):
file_name = file_name.replace('/data/local-files/?d=', '')
file_name = file_name.replace('/home/kitraining/home/kitraining/', '')
if file_name and file_name.startswith('home/kitraining/To_Annotate/'):
file_name = file_name.replace('home/kitraining/To_Annotate/', '')
# Remove any Label Studio prefixes but keep full path
# Common Label Studio patterns
prefixes_to_remove = [
'//192.168.1.19/home/kitraining/To_Annotate/',
'192.168.1.19/home/kitraining/To_Annotate/',
'/home/kitraining/home/kitraining/',
'home/kitraining/To_Annotate/',
'/home/kitraining/To_Annotate/',
]
# Try each prefix
for prefix in prefixes_to_remove:
if file_name.startswith(prefix):
file_name = file_name[len(prefix):]
break
# Construct ABSOLUTE path using data_dir
# Detect platform for proper path handling
import platform
is_windows = platform.system() == 'Windows'
# Normalize data_dir and file_name based on platform
if is_windows:
# Windows: use backslashes
normalized_data_dir = data_dir.rstrip('/\\').replace('/', '\\')
file_name = file_name.replace('/', '\\')
else:
# Linux/Mac: use forward slashes
normalized_data_dir = data_dir.rstrip('/\\').replace('\\', '/')
file_name = file_name.replace('\\', '/')
# Check if already absolute path
is_absolute = False
if is_windows:
is_absolute = file_name.startswith('\\\\') or (len(file_name) > 1 and file_name[1] == ':')
else:
is_absolute = file_name.startswith('/')
if not is_absolute:
# It's a relative path, combine with data_dir
if is_windows and normalized_data_dir.startswith('\\\\'):
# Windows UNC path
file_name = normalized_data_dir + '\\' + file_name
else:
# Regular path (Windows or Linux)
file_name = os.path.join(normalized_data_dir, file_name)
# Get annotations for this image
annotations = Annotation.query.filter_by(image_id=image.image_id).all()
coco_images.append({
'id': image_id,
'file_name': file_name,
'file_name': file_name, # Use absolute path
'width': image.width or 0,
'height': image.height or 0
})
for annotation in annotations:
# Translate class name using asg_map
mapped_class = annotation.Label
for map_entry in asg_map:
if annotation.Label == map_entry['original']:
mapped_class = map_entry['mapped']
break
# Translate class name using class_map for this specific Label Studio project
original_class = annotation.Label
project_class_map = mappings_by_project.get(ls_project_id, {})
mapped_class = project_class_map.get(original_class, original_class)
# Only add annotation if mapped_class is valid
if mapped_class and mapped_class in category_map:
@@ -146,14 +242,35 @@ def generate_training_json(training_id):
test_json = build_coco_json(test_images, test_annotations, coco_categories)
# Create output directory
from services.settings_service import get_setting
from models.training import Training
output_base_path = get_setting('yolox_output_path', './backend')
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}'
annotations_dir = '/home/kitraining/To_Annotate/annotations'
# Get training record to use its name and ID for folder and file names
# Use the same training_id that was passed in (if it was a Training.id)
# or find the first training for this details_id
if not training_record:
training_record = Training.query.filter_by(project_details_id=details_id).first()
if training_record:
training_folder_name = f"{training_record.exp_name or training_record.training_name or 'training'}_{training_record.id}"
training_folder_name = training_folder_name.replace(' ', '_')
training_file_id = training_record.id
else:
training_folder_name = str(details_id)
training_file_id = details_id
# Save annotations to the configured output folder
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
os.makedirs(annotations_dir, exist_ok=True)
# Write to files
train_path = f'{annotations_dir}/coco_project_{training_id}_train.json'
valid_path = f'{annotations_dir}/coco_project_{training_id}_valid.json'
test_path = f'{annotations_dir}/coco_project_{training_id}_test.json'
train_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_train.json')
valid_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_valid.json')
test_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_test.json')
with open(train_path, 'w') as f:
json.dump(train_json, f, indent=2)
@@ -162,11 +279,11 @@ def generate_training_json(training_id):
with open(test_path, 'w') as f:
json.dump(test_json, f, indent=2)
print(f'COCO JSON splits written to {annotations_dir} for trainingId {training_id}')
print(f'COCO JSON splits written to {annotations_dir} for training_id={training_file_id} (details_id={details_id})')
# Also generate inference exp.py
from services.generate_yolox_exp import generate_yolox_inference_exp
project_folder = os.path.join(os.path.dirname(__file__), '..', project_name, str(training_id))
project_folder = os.path.join(output_base_path, project_name, str(training_id))
os.makedirs(project_folder, exist_ok=True)
inference_exp_path = os.path.join(project_folder, 'exp_infer.py')

191
backend/services/generate_yolox_exp.py Normal file → Executable file
View File

@@ -82,30 +82,42 @@ def generate_yolox_inference_exp(training_id, options=None, use_base_config=Fals
if not training:
raise Exception(f'Training not found for trainingId or project_details_id: {training_id}')
# Always use the training_id (project_details_id) for annotation file names
# Always use the project_details_id for annotation file names and paths
project_details_id = training.project_details_id
data_dir = options.get('data_dir', '/home/kitraining/To_Annotate/')
# Get annotation file names from options or use defaults
# Use training.id (not project_details_id) for consistency with generate_training_json
train_ann = options.get('train_ann', f'coco_project_{training_id}_train.json')
val_ann = options.get('val_ann', f'coco_project_{training_id}_valid.json')
test_ann = options.get('test_ann', f'coco_project_{training_id}_test.json')
# Get num_classes from TrainingProject.classes JSON
# Get data_dir - this should point to where IMAGES are located (not annotations)
# YOLOX will combine data_dir + file_name from COCO JSON to find images
# The annotations are in a separate location (output folder)
from services.settings_service import get_setting
from models.TrainingProjectDetails import TrainingProjectDetails
if 'data_dir' in options:
data_dir = options['data_dir']
else:
# Use the yolox_data_dir setting - this is where training images are stored
data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
# Ensure it ends with a separator
if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
data_dir += os.sep
# Get num_classes from ProjectClass table (3NF)
num_classes = 80
try:
from models.ProjectClass import ProjectClass
training_project = TrainingProject.query.get(project_details_id)
if training_project and training_project.classes:
classes_arr = training_project.classes
if isinstance(classes_arr, str):
import json
classes_arr = json.loads(classes_arr)
if isinstance(classes_arr, list):
num_classes = len([c for c in classes_arr if c not in [None, '']])
elif isinstance(classes_arr, dict):
num_classes = len([k for k, v in classes_arr.items() if v not in [None, '']])
if training_project:
# Count classes from ProjectClass table
class_count = ProjectClass.query.filter_by(project_id=training_project.project_id).count()
if class_count > 0:
num_classes = class_count
except Exception as e:
print(f'Could not determine num_classes from TrainingProject.classes: {e}')
print(f'Could not determine num_classes from ProjectClass: {e}')
# Initialize config dictionary
config = {}
@@ -120,13 +132,29 @@ def generate_yolox_inference_exp(training_id, options=None, use_base_config=Fals
print(f'Warning: Could not load base config for {training.selected_model}: {e}')
print('Falling back to custom settings only')
# Get size arrays from TrainingSize table (3NF)
from models.TrainingSize import TrainingSize
def get_size_array(training_id, size_type):
"""Helper to get size array from TrainingSize table"""
sizes = TrainingSize.query.filter_by(
training_id=training_id,
size_type=size_type
).order_by(TrainingSize.value_order).all()
return [s.value for s in sizes] if sizes else None
input_size = get_size_array(training.id, 'input_size')
test_size = get_size_array(training.id, 'test_size')
mosaic_scale = get_size_array(training.id, 'mosaic_scale')
mixup_scale = get_size_array(training.id, 'mixup_scale')
# Override with user-defined values from training table (only if they exist and are not None)
user_overrides = {
'depth': training.depth,
'width': training.width,
'input_size': training.input_size,
'mosaic_scale': training.mosaic_scale,
'test_size': training.test_size,
'input_size': input_size,
'mosaic_scale': mosaic_scale,
'test_size': test_size,
'enable_mixup': training.enable_mixup,
'max_epoch': training.max_epoch,
'warmup_epochs': training.warmup_epochs,
@@ -149,7 +177,7 @@ def generate_yolox_inference_exp(training_id, options=None, use_base_config=Fals
'degrees': training.degrees,
'translate': training.translate,
'shear': training.shear,
'mixup_scale': training.mixup_scale,
'mixup_scale': mixup_scale,
'activation': training.activation,
}
@@ -171,6 +199,31 @@ def generate_yolox_inference_exp(training_id, options=None, use_base_config=Fals
config.setdefault('enable_mixup', False)
config.setdefault('exp_name', 'inference_exp')
# Prepare data_dir for template - escape backslashes and remove trailing separator
data_dir_clean = data_dir.rstrip('/\\')
data_dir_escaped = data_dir_clean.replace('\\', '\\\\')
# Calculate annotations directory (where JSON files are stored)
# This is in the output folder, not with the images
from models.TrainingProjectDetails import TrainingProjectDetails
details = TrainingProjectDetails.query.get(project_details_id)
if details:
training_project = TrainingProject.query.get(details.project_id)
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details.project_id}'
else:
project_name = f'project_{project_details_id}'
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
training_folder_name = training_folder_name.replace(' ', '_')
output_base_path = get_setting('yolox_output_path', './backend')
annotations_parent_dir = os.path.join(output_base_path, project_name, training_folder_name)
annotations_parent_escaped = annotations_parent_dir.replace('\\', '\\\\')
# Set output directory for checkpoints - models subdirectory
models_dir = os.path.join(annotations_parent_dir, 'models')
models_dir_escaped = models_dir.replace('\\', '\\\\')
# Build exp content
exp_content = f'''#!/usr/bin/env python3
# -*- coding:utf-8 -*-
@@ -184,11 +237,17 @@ from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "{data_dir}"
self.data_dir = "{data_dir_escaped}" # Where images are located
self.annotations_dir = "{annotations_parent_escaped}" # Where annotation JSONs are located
self.output_dir = "{models_dir_escaped}" # Where checkpoints will be saved
self.train_ann = "{train_ann}"
self.val_ann = "{val_ann}"
self.test_ann = "{test_ann}"
self.num_classes = {num_classes}
# Disable train2017 subdirectory - our images are directly in data_dir
self.name = ""
# Set data workers for training
self.data_num_workers = 8
'''
# Set pretrained_ckpt if transfer_learning is 'coco'
@@ -198,21 +257,107 @@ class Exp(MyExp):
if selected_model:
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n"
# Format arrays
def format_value(val):
# Format arrays and values for Python code generation
# Integer-only parameters (sizes, epochs, intervals)
integer_params = {
'input_size', 'test_size', 'random_size', 'max_epoch', 'warmup_epochs',
'no_aug_epochs', 'print_interval', 'eval_interval', 'multiscale_range',
'data_num_workers', 'num_classes'
}
def format_value(val, param_name=''):
if isinstance(val, (list, tuple)):
return '(' + ', '.join(map(str, val)) + ')'
# Check if this parameter should have integer values
if param_name in integer_params:
# Convert all values to integers
formatted_items = [str(int(float(item))) if isinstance(item, (int, float)) else str(item) for item in val]
else:
# Keep as floats or original type
formatted_items = []
for item in val:
if isinstance(item, float):
formatted_items.append(str(item))
elif isinstance(item, int):
formatted_items.append(str(item))
else:
formatted_items.append(str(item))
return '(' + ', '.join(formatted_items) + ')'
elif isinstance(val, bool):
return str(val)
elif isinstance(val, str):
return f'"{val}"'
elif isinstance(val, int):
return str(val)
elif isinstance(val, float):
return str(val)
else:
return str(val)
# Add all config parameters to exp
for key, value in config.items():
if key not in ['exp_name']: # exp_name is handled separately
exp_content += f" self.{key} = {format_value(value)}\n"
exp_content += f" self.{key} = {format_value(value, key)}\n"
# Add get_dataset override using name parameter for image directory
exp_content += '''
def get_dataset(self, cache=False, cache_type="ram"):
"""Override to use name parameter for images directory"""
from yolox.data import COCODataset
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
# Use empty string for name since we have absolute paths in JSON
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.train_ann,
name="",
img_size=self.input_size,
preproc=self.preproc if hasattr(self, 'preproc') else None,
cache=cache,
cache_type=cache_type,
)
def get_eval_dataset(self, **kwargs):
"""Override eval dataset using name parameter"""
from yolox.data import COCODataset, ValTransform
testdev = kwargs.get("testdev", False)
legacy = kwargs.get("legacy", False)
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.val_ann if not testdev else self.test_ann,
name="",
img_size=self.test_size,
preproc=ValTransform(legacy=legacy), # Use proper validation transform
)
def get_eval_loader(self, batch_size, is_distributed, **kwargs):
"""Standard YOLOX eval loader - matches official implementation"""
import torch
import torch.distributed as dist
from torch.utils.data import DataLoader
valdataset = self.get_eval_dataset(**kwargs)
if is_distributed:
batch_size = batch_size // dist.get_world_size()
sampler = torch.utils.data.distributed.DistributedSampler(
valdataset, shuffle=False
)
else:
sampler = torch.utils.data.SequentialSampler(valdataset)
dataloader_kwargs = {
"num_workers": self.data_num_workers,
"pin_memory": True,
"sampler": sampler,
}
dataloader_kwargs["batch_size"] = batch_size
val_loader = DataLoader(valdataset, **dataloader_kwargs)
return val_loader
'''
# Add exp_name at the end (uses dynamic path)
exp_content += f''' self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

View File

@@ -1,48 +0,0 @@
const Training = require('../models/training.js');
const fs = require('fs');
const path = require('path');
async function pushYoloxExpToDb(settings) {
// Normalize boolean and array fields for DB
const normalized = { ...settings };
// Map 'act' from frontend to 'activation' for DB
if (normalized.act !== undefined) {
normalized.activation = normalized.act;
delete normalized.act;
}
// Convert 'on'/'off' to boolean for save_history_ckpt
if (typeof normalized.save_history_ckpt === 'string') {
normalized.save_history_ckpt = normalized.save_history_ckpt === 'on' ? true : false;
}
// Convert comma-separated strings to arrays for input_size, test_size, mosaic_scale, mixup_scale
['input_size', 'test_size', 'mosaic_scale', 'mixup_scale'].forEach(key => {
if (typeof normalized[key] === 'string') {
const arr = normalized[key].split(',').map(v => parseFloat(v.trim()));
normalized[key] = arr.length === 1 ? arr[0] : arr;
}
});
// Find TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id: normalized.project_id } });
if (!details) throw new Error('TrainingProjectDetails not found for project_id ' + normalized.project_id);
normalized.project_details_id = details.id;
// Create DB row
const training = await Training.create(normalized);
return training;
}
async function generateYoloxExpFromDb(trainingId) {
// Fetch training row from DB
const training = await Training.findByPk(trainingId);
if (!training) throw new Error('Training not found');
// Template for exp.py
const expTemplate = `#!/usr/bin/env python3\n# Copyright (c) Megvii Inc. All rights reserved.\n\nimport os\nimport random\n\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nfrom .base_exp import BaseExp\n\n__all__ = [\"Exp\", \"check_exp_value\"]\n\nclass Exp(BaseExp):\n def __init__(self):\n super().__init__()\n\n # ---------------- model config ---------------- #\n self.num_classes = ${training.num_classes || 80}\n self.depth = ${training.depth || 1.00}\n self.width = ${training.width || 1.00}\n self.act = \"${training.activation || training.act || 'silu'}\"\n\n # ---------------- dataloader config ---------------- #\n self.data_num_workers = ${training.data_num_workers || 4}\n self.input_size = (${Array.isArray(training.input_size) ? training.input_size.join(', ') : '640, 640'})\n self.multiscale_range = ${training.multiscale_range || 5}\n self.data_dir = ${training.data_dir ? `\"${training.data_dir}\"` : 'None'}\n self.train_ann = \"${training.train_ann || 'instances_train2017.json'}\"\n self.val_ann = \"${training.val_ann || 'instances_val2017.json'}\"\n self.test_ann = \"${training.test_ann || 'instances_test2017.json'}\"\n\n # --------------- transform config ----------------- #\n self.mosaic_prob = ${training.mosaic_prob !== undefined ? training.mosaic_prob : 1.0}\n self.mixup_prob = ${training.mixup_prob !== undefined ? training.mixup_prob : 1.0}\n self.hsv_prob = ${training.hsv_prob !== undefined ? training.hsv_prob : 1.0}\n self.flip_prob = ${training.flip_prob !== undefined ? training.flip_prob : 0.5}\n self.degrees = ${training.degrees !== undefined ? training.degrees : 10.0}\n self.translate = ${training.translate !== undefined ? training.translate : 0.1}\n self.mosaic_scale = (${Array.isArray(training.mosaic_scale) ? training.mosaic_scale.join(', ') : '0.1, 2'})\n self.enable_mixup = ${training.enable_mixup !== undefined ? training.enable_mixup : true}\n self.mixup_scale = (${Array.isArray(training.mixup_scale) ? training.mixup_scale.join(', ') : '0.5, 1.5'})\n self.shear = ${training.shear !== undefined ? training.shear : 2.0}\n\n # -------------- training config --------------------- #\n self.warmup_epochs = ${training.warmup_epochs !== undefined ? training.warmup_epochs : 5}\n self.max_epoch = ${training.max_epoch !== undefined ? training.max_epoch : 300}\n self.warmup_lr = ${training.warmup_lr !== undefined ? training.warmup_lr : 0}\n self.min_lr_ratio = ${training.min_lr_ratio !== undefined ? training.min_lr_ratio : 0.05}\n self.basic_lr_per_img = ${training.basic_lr_per_img !== undefined ? training.basic_lr_per_img : 0.01 / 64.0}\n self.scheduler = \"${training.scheduler || 'yoloxwarmcos'}\"\n self.no_aug_epochs = ${training.no_aug_epochs !== undefined ? training.no_aug_epochs : 15}\n self.ema = ${training.ema !== undefined ? training.ema : true}\n self.weight_decay = ${training.weight_decay !== undefined ? training.weight_decay : 5e-4}\n self.momentum = ${training.momentum !== undefined ? training.momentum : 0.9}\n self.print_interval = ${training.print_interval !== undefined ? training.print_interval : 10}\n self.eval_interval = ${training.eval_interval !== undefined ? training.eval_interval : 10}\n self.save_history_ckpt = ${training.save_history_ckpt !== undefined ? training.save_history_ckpt : true}\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n\n # ----------------- testing config ------------------ #\n self.test_size = (${Array.isArray(training.test_size) ? training.test_size.join(', ') : '640, 640'})\n self.test_conf = ${training.test_conf !== undefined ? training.test_conf : 0.01}\n self.nmsthre = ${training.nmsthre !== undefined ? training.nmsthre : 0.65}\n\n # ... rest of the template ...\n\ndef check_exp_value(exp: Exp):\n h, w = exp.input_size\n assert h % 32 == 0 and w % 32 == 0, \"input size must be multiples of 32\"\n`;
// Save to file in output directory
const outDir = path.join(__dirname, '../../', training.project_id ? `project_${training.project_id}/${trainingId}` : 'exp_files');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
const filePath = path.join(outDir, 'exp.py');
fs.writeFileSync(filePath, expTemplate);
return filePath;
}
module.exports = { pushYoloxExpToDb, generateYoloxExpFromDb };

31
backend/services/push_yolox_exp.py Normal file → Executable file
View File

@@ -1,5 +1,6 @@
from models.training import Training
from models.TrainingProjectDetails import TrainingProjectDetails
from models.TrainingSize import TrainingSize
from database.database import db
def push_yolox_exp_to_db(settings):
@@ -25,15 +26,23 @@ def push_yolox_exp_to_db(settings):
else:
normalized[bool_field] = bool(val)
# Convert comma-separated strings to arrays for JSON fields
# Extract size arrays for separate TrainingSize table (3NF)
size_arrays = {}
for key in ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale']:
if key in normalized and isinstance(normalized[key], str):
if key in normalized:
if isinstance(normalized[key], str):
parts = [p.strip() for p in normalized[key].split(',') if p.strip()]
try:
arr = [float(p) for p in parts]
except Exception:
arr = parts
normalized[key] = arr[0] if len(arr) == 1 else arr
size_arrays[key] = arr[0] if len(arr) == 1 else (arr if isinstance(arr, list) else [arr])
elif isinstance(normalized[key], list):
size_arrays[key] = normalized[key]
elif normalized[key] is not None:
size_arrays[key] = [float(normalized[key])]
# Remove from normalized dict since it won't be stored in training table
del normalized[key]
# Ensure we have a TrainingProjectDetails row for project_id
project_id = normalized.get('project_id')
@@ -67,8 +76,6 @@ def push_yolox_exp_to_db(settings):
filtered[k] = v.lower() in ('1', 'true', 'on')
else:
filtered[k] = bool(v)
elif 'JSON' in col_type:
filtered[k] = v
elif 'LargeBinary' in col_type:
# If a file path was passed, store its bytes; otherwise store raw bytes
if isinstance(v, str):
@@ -87,6 +94,20 @@ def push_yolox_exp_to_db(settings):
# Create DB row
training = Training(**filtered)
db.session.add(training)
db.session.flush() # Get training.id
# Save size arrays to TrainingSize table (3NF)
for size_type, values in size_arrays.items():
if values and isinstance(values, list):
for order, value in enumerate(values):
size_record = TrainingSize(
training_id=training.id,
size_type=size_type,
value_order=order,
value=float(value)
)
db.session.add(size_record)
db.session.commit()
return training

View File

@@ -1,120 +0,0 @@
const sequelize = require('../database/database.js');
const { Project, Img, Ann } = require('../models');
const { fetchLableStudioProject, fetchProjectIdsAndTitles } = require('./fetch-labelstudio.js');
const updateStatus = { running: false };
async function seedLabelStudio() {
updateStatus.running = true;
console.log('Seeding started');
try {
await sequelize.sync();
const projects = await fetchProjectIdsAndTitles();
for (const project of projects) {
console.log(`Processing project ${project.id} (${project.title})`);
// Upsert project in DB
await Project.upsert({ project_id: project.id, title: project.title });
// Fetch project data (annotations array)
const data = await fetchLableStudioProject(project.id);
if (!Array.isArray(data) || data.length === 0) {
console.log(`No annotation data for project ${project.id}`);
continue;
}
// Remove old images and annotations for this project
const oldImages = await Img.findAll({ where: { project_id: project.id } });
const oldImageIds = oldImages.map(img => img.image_id);
if (oldImageIds.length > 0) {
await Ann.destroy({ where: { image_id: oldImageIds } });
await Img.destroy({ where: { project_id: project.id } });
console.log(`Deleted ${oldImageIds.length} old images and their annotations for project ${project.id}`);
}
// Prepare arrays
const imagesBulk = [];
const annsBulk = [];
for (const ann of data) {
// Extract width/height
let width = null;
let height = null;
if (Array.isArray(ann.label_rectangles) && ann.label_rectangles.length > 0) {
width = ann.label_rectangles[0].original_width;
height = ann.label_rectangles[0].original_height;
} else if (Array.isArray(ann.label) && ann.label.length > 0 && ann.label[0].original_width && ann.label[0].original_height) {
width = ann.label[0].original_width;
height = ann.label[0].original_height;
}
// Only push image and annotations if width and height are valid
if (width && height) {
imagesBulk.push({
project_id: project.id,
image_path: ann.image,
width,
height
});
// Handle multiple annotations per image
if (Array.isArray(ann.label_rectangles)) {
for (const ann_detail of ann.label_rectangles) {
annsBulk.push({
image_path: ann.image,
x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
});
}
} else if (Array.isArray(ann.label)) {
for (const ann_detail of ann.label) {
annsBulk.push({
image_path: ann.image,
x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
});
}
}
}
}
// 1) Insert images and get generated IDs
const insertedImages = await Img.bulkCreate(imagesBulk, { returning: true });
// 2) Map image_path -> image_id
const imageMap = {};
for (const img of insertedImages) {
imageMap[img.image_path] = img.image_id;
}
// 3) Assign correct image_id to each annotation
for (const ann of annsBulk) {
ann.image_id = imageMap[ann.image_path];
delete ann.image_path; // cleanup
}
// 4) Insert annotations
await Ann.bulkCreate(annsBulk);
console.log(`Inserted ${imagesBulk.length} images and ${annsBulk.length} annotations for project ${project.id}`);
}
console.log('Seeding done');
return { success: true, message: 'Data inserted successfully!' };
} catch (error) {
console.error('Error inserting data:', error);
return { success: false, message: error.message };
} finally {
updateStatus.running = false;
console.log('updateStatus.running set to false');
}
}
module.exports = { seedLabelStudio, updateStatus };

4
backend/services/seed_label_studio.py Normal file → Executable file
View File

@@ -64,8 +64,8 @@ def seed_label_studio():
image_data = {
'project_id': project['id'],
'image_path': ann.get('image'),
'width': width,
'height': height
'width': int(width), # Ensure integer
'height': int(height) # Ensure integer
}
images_bulk.append(image_data)

View File

@@ -0,0 +1,71 @@
"""
Settings Service - Manages global application settings
"""
from models.Settings import Settings
from database.database import db
def get_setting(key, default=None):
"""Get a setting value by key"""
setting = Settings.query.filter_by(key=key).first()
return setting.value if setting else default
def set_setting(key, value, description=None):
"""Set a setting value"""
setting = Settings.query.filter_by(key=key).first()
if setting:
setting.value = value
if description:
setting.description = description
else:
setting = Settings(key=key, value=value, description=description)
db.session.add(setting)
db.session.commit()
return setting
def get_all_settings():
"""Get all settings as a dictionary"""
settings = Settings.query.all()
return {s.key: s.value for s in settings}
def get_all_settings_detailed():
"""Get all settings with full details"""
settings = Settings.query.all()
return [s.to_dict() for s in settings]
def initialize_default_settings():
"""Initialize default settings if they don't exist"""
defaults = {
'labelstudio_api_url': {
'value': 'http://192.168.1.19:8080/api',
'description': 'Label Studio API URL'
},
'labelstudio_api_token': {
'value': 'c1cef980b7c73004f4ee880a42839313b863869f',
'description': 'Label Studio API Token'
},
'yolox_path': {
'value': 'C:/YOLOX',
'description': 'Path to YOLOX installation directory'
},
'yolox_venv_path': {
'value': '/home/kitraining/Yolox/yolox_venv/bin/activate',
'description': 'Path to YOLOX virtual environment activation script'
},
'yolox_output_path': {
'value': './backend',
'description': 'Output folder for YOLOX experiment files and JSONs'
},
'yolox_data_dir': {
'value': '/home/kitraining/To_Annotate/',
'description': 'Data directory path for YOLOX training (where images are located)'
}
}
for key, data in defaults.items():
existing = Settings.query.filter_by(key=key).first()
if not existing:
setting = Settings(key=key, value=data['value'], description=data['description'])
db.session.add(setting)
db.session.commit()

View File

@@ -0,0 +1,206 @@
"""
Training Queue Manager
Manages a queue of training jobs and tracks their progress
"""
import threading
import queue
import subprocess
import re
import os
from services.settings_service import get_setting
from models.training import Training
class TrainingQueueManager:
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
if self._initialized:
return
self.queue = queue.Queue()
self.current_training = None
self.current_process = None
self.worker_thread = None
self.running = False
self._initialized = True
# Start the worker thread
self.start_worker()
def start_worker(self):
"""Start the background worker thread"""
if self.worker_thread is None or not self.worker_thread.is_alive():
self.running = True
self.worker_thread = threading.Thread(target=self._process_queue, daemon=True)
self.worker_thread.start()
def add_to_queue(self, training_id, command, cwd):
"""Add a training job to the queue"""
job = {
'training_id': training_id,
'command': command,
'cwd': cwd,
'iteration': 0,
'max_epoch': 300 # Will be updated from training record
}
# Get max_epoch from training record
try:
training = Training.query.get(training_id)
if training:
job['max_epoch'] = training.max_epoch or 300
job['name'] = training.exp_name or f'Training {training_id}'
except:
pass
self.queue.put(job)
print(f'Added training {training_id} to queue. Queue size: {self.queue.qsize()}')
def _process_queue(self):
"""Worker thread that processes the queue"""
while self.running:
try:
# Wait for a job (blocking with timeout)
job = self.queue.get(timeout=1)
print(f'Starting training {job["training_id"]} from queue')
self.current_training = job
# Execute the training command
self._run_training(job)
# Mark as done
self.queue.task_done()
self.current_training = None
self.current_process = None
except queue.Empty:
continue
except Exception as e:
print(f'Error processing training job: {e}')
self.current_training = None
self.current_process = None
def _run_training(self, job):
"""Run a training command and monitor its output"""
try:
import platform
is_windows = platform.system() == 'Windows'
# Start process
self.current_process = subprocess.Popen(
job['command'],
shell=True,
cwd=job['cwd'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
universal_newlines=True,
bufsize=1
)
# Monitor output for progress
for line in iter(self.current_process.stdout.readline, ''):
if line:
print(line.strip())
# Parse epoch and iteration from YOLOX output
# Example: "epoch: 3/300, iter: 90/101"
epoch_match = re.search(r'epoch:\s*(\d+)/(\d+)', line, re.IGNORECASE)
iter_match = re.search(r'iter:\s*(\d+)/(\d+)', line, re.IGNORECASE)
if epoch_match:
current_epoch = int(epoch_match.group(1))
total_epochs = int(epoch_match.group(2))
if self.current_training:
self.current_training['current_epoch'] = current_epoch
self.current_training['max_epoch'] = total_epochs
# Debug log
print(f'[PROGRESS] Parsed epoch: {current_epoch}/{total_epochs}')
if iter_match:
current_iter = int(iter_match.group(1))
total_iters = int(iter_match.group(2))
if self.current_training:
self.current_training['current_iter'] = current_iter
self.current_training['total_iters'] = total_iters
# Calculate overall progress percentage
if 'current_epoch' in self.current_training and 'max_epoch' in self.current_training:
epoch_progress = (self.current_training['current_epoch'] - 1) / self.current_training['max_epoch']
iter_progress = current_iter / total_iters / self.current_training['max_epoch']
total_progress = (epoch_progress + iter_progress) * 100
self.current_training['progress'] = round(total_progress, 2)
# Debug log
print(f'[PROGRESS] Epoch {self.current_training["current_epoch"]}/{self.current_training["max_epoch"]}, Iter {current_iter}/{total_iters}, Progress: {self.current_training["progress"]}%')
# Wait for completion
self.current_process.wait()
print(f'Training {job["training_id"]} completed with exit code {self.current_process.returncode}')
except Exception as e:
print(f'Error running training: {e}')
def get_status(self):
"""Get current status of training queue"""
queue_items = []
# Get items from queue without removing them
temp_items = []
while not self.queue.empty():
try:
item = self.queue.get_nowait()
temp_items.append(item)
queue_items.append({
'training_id': item['training_id'],
'name': item.get('name', f'Training {item["training_id"]}'),
'max_epoch': item.get('max_epoch', 300)
})
except queue.Empty:
break
# Put items back
for item in temp_items:
self.queue.put(item)
result = {
'current': None,
'queue': queue_items
}
if self.current_training:
current_epoch = self.current_training.get('current_epoch', 0)
max_epoch = self.current_training.get('max_epoch', 300)
result['current'] = {
'training_id': self.current_training['training_id'],
'name': self.current_training.get('name', f'Training {self.current_training["training_id"]}'),
'epoch': current_epoch, # For backward compatibility
'current_epoch': current_epoch,
'max_epoch': max_epoch,
'current_iter': self.current_training.get('current_iter', 0),
'total_iters': self.current_training.get('total_iters', 0),
'progress': self.current_training.get('progress', 0.0),
'iteration': current_epoch # For backward compatibility
}
return result
def stop(self):
"""Stop the worker thread"""
self.running = False
if self.current_process:
try:
self.current_process.terminate()
except:
pass
# Global instance
training_queue = TrainingQueueManager()

View File

@@ -0,0 +1,244 @@
"""
Validate dataset for training - check for problematic images and annotations
"""
import os
import json
from PIL import Image
import cv2
def validate_coco_json(json_path, data_dir):
"""
Validate a COCO JSON file and check all images
Args:
json_path: Path to COCO JSON file
data_dir: Directory where images are located
Returns:
dict with validation results
"""
print(f"\n{'='*60}")
print(f"Validating: {json_path}")
print(f"{'='*60}\n")
issues = {
'missing_images': [],
'corrupted_images': [],
'zero_dimension_images': [],
'invalid_annotations': [],
'zero_area_boxes': []
}
try:
with open(json_path, 'r') as f:
coco_data = json.load(f)
except Exception as e:
print(f"❌ Failed to load JSON: {e}")
return issues
images = coco_data.get('images', [])
annotations = coco_data.get('annotations', [])
print(f"📊 Dataset Stats:")
print(f" Images: {len(images)}")
print(f" Annotations: {len(annotations)}")
print(f" Categories: {len(coco_data.get('categories', []))}")
print()
# Validate images
print("🔍 Validating images...")
for idx, img_info in enumerate(images):
img_id = img_info.get('id')
file_name = img_info.get('file_name', '')
width = img_info.get('width', 0)
height = img_info.get('height', 0)
# Check if image file exists
# Try to construct the full path
if os.path.isabs(file_name):
img_path = file_name
else:
img_path = os.path.join(data_dir, file_name)
if not os.path.exists(img_path):
issues['missing_images'].append({
'id': img_id,
'file_name': file_name,
'expected_path': img_path
})
continue
# Check if image can be loaded
try:
# Try with PIL
with Image.open(img_path) as pil_img:
pil_width, pil_height = pil_img.size
# Check if dimensions match JSON
if pil_width != width or pil_height != height:
print(f"⚠️ Image {img_id}: Dimension mismatch - JSON: {width}x{height}, Actual: {pil_width}x{pil_height}")
# Check for zero dimensions
if pil_width == 0 or pil_height == 0:
issues['zero_dimension_images'].append({
'id': img_id,
'file_name': file_name,
'dimensions': f"{pil_width}x{pil_height}"
})
except Exception as e:
issues['corrupted_images'].append({
'id': img_id,
'file_name': file_name,
'error': str(e)
})
# Progress indicator
if (idx + 1) % 100 == 0:
print(f" Checked {idx + 1}/{len(images)} images...")
print(f"✅ Image validation complete\n")
# Validate annotations
print("🔍 Validating annotations...")
for idx, ann in enumerate(annotations):
ann_id = ann.get('id')
img_id = ann.get('image_id')
bbox = ann.get('bbox', [])
if len(bbox) != 4:
issues['invalid_annotations'].append({
'id': ann_id,
'image_id': img_id,
'reason': f'Invalid bbox length: {len(bbox)}'
})
continue
x, y, w, h = bbox
# Check for zero or negative dimensions
if w <= 0 or h <= 0:
issues['zero_area_boxes'].append({
'id': ann_id,
'image_id': img_id,
'bbox': bbox,
'reason': f'Zero or negative dimensions: w={w}, h={h}'
})
# Check for extremely small boxes (potential issue with mixup)
if w < 1 or h < 1:
issues['zero_area_boxes'].append({
'id': ann_id,
'image_id': img_id,
'bbox': bbox,
'reason': f'Extremely small box: w={w}, h={h}'
})
# Progress indicator
if (idx + 1) % 1000 == 0:
print(f" Checked {idx + 1}/{len(annotations)} annotations...")
print(f"✅ Annotation validation complete\n")
# Print summary
print(f"\n{'='*60}")
print("VALIDATION SUMMARY")
print(f"{'='*60}\n")
total_issues = sum(len(v) for v in issues.values())
if total_issues == 0:
print("✅ No issues found! Dataset is ready for training.")
else:
print(f"⚠️ Found {total_issues} total issues:\n")
if issues['missing_images']:
print(f" ❌ Missing images: {len(issues['missing_images'])}")
for item in issues['missing_images'][:5]: # Show first 5
print(f" - {item['file_name']}")
if len(issues['missing_images']) > 5:
print(f" ... and {len(issues['missing_images']) - 5} more")
if issues['corrupted_images']:
print(f" ❌ Corrupted images: {len(issues['corrupted_images'])}")
for item in issues['corrupted_images'][:5]:
print(f" - {item['file_name']}: {item['error']}")
if len(issues['corrupted_images']) > 5:
print(f" ... and {len(issues['corrupted_images']) - 5} more")
if issues['zero_dimension_images']:
print(f" ❌ Zero dimension images: {len(issues['zero_dimension_images'])}")
for item in issues['zero_dimension_images'][:5]:
print(f" - {item['file_name']}: {item['dimensions']}")
if len(issues['zero_dimension_images']) > 5:
print(f" ... and {len(issues['zero_dimension_images']) - 5} more")
if issues['invalid_annotations']:
print(f" ❌ Invalid annotations: {len(issues['invalid_annotations'])}")
for item in issues['invalid_annotations'][:5]:
print(f" - Ann ID {item['id']}: {item['reason']}")
if len(issues['invalid_annotations']) > 5:
print(f" ... and {len(issues['invalid_annotations']) - 5} more")
if issues['zero_area_boxes']:
print(f" ⚠️ Zero/tiny area boxes: {len(issues['zero_area_boxes'])}")
print(f" These may cause issues with mixup augmentation!")
for item in issues['zero_area_boxes'][:5]:
print(f" - Ann ID {item['id']}, bbox: {item['bbox']}")
if len(issues['zero_area_boxes']) > 5:
print(f" ... and {len(issues['zero_area_boxes']) - 5} more")
print()
return issues
def validate_training_dataset(training_id):
"""
Validate all COCO JSON files for a training
Args:
training_id: The training ID to validate
"""
from models.training import Training
from models.TrainingProject import TrainingProject
from services.settings_service import get_setting
training = Training.query.get(training_id)
if not training:
print(f"❌ Training {training_id} not found")
return
# Get paths
from models.TrainingProjectDetails import TrainingProjectDetails
details = TrainingProjectDetails.query.get(training.project_details_id)
training_project = TrainingProject.query.get(details.project_id)
project_name = training_project.title.replace(' ', '_') if training_project else f'project_{details.project_id}'
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
training_folder_name = training_folder_name.replace(' ', '_')
output_base_path = get_setting('yolox_output_path', './backend')
data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
# Validate each split
splits = ['train', 'valid', 'test']
all_issues = {}
for split in splits:
json_file = os.path.join(annotations_dir, f'coco_project_{training_id}_{split}.json')
if os.path.exists(json_file):
all_issues[split] = validate_coco_json(json_file, data_dir)
else:
print(f"⚠️ JSON file not found: {json_file}")
return all_issues
if __name__ == '__main__':
import sys
if len(sys.argv) > 1:
training_id = int(sys.argv[1])
validate_training_dataset(training_id)
else:
print("Usage: python validate_dataset.py <training_id>")

0
backend/start.py Normal file → Executable file
View File

47
backend/test/7/exp.py Executable file
View File

@@ -0,0 +1,47 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_3_train.json"
self.val_ann = "coco_project_3_valid.json"
self.test_ann = "coco_project_3_test.json"
self.num_classes = 2
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

BIN
documentation/Projektdoku.pdf Executable file

Binary file not shown.

View File

@@ -0,0 +1,764 @@
# Projektdokumentation: mb ai Trainer
**Autor:** [Ihr Name]
**Datum:** 1. Dezember 2025
**Projekt:** mb ai Trainer - Webbasiertes YOLOX Trainings-Management-System
---
## Inhaltsverzeichnis
1. [Einleitung](#1-einleitung)
- 1.1 [Das Unternehmen](#11-das-unternehmen)
- 1.2 [Projektumfeld](#12-projektumfeld)
- 1.3 [Projektziel](#13-projektziel)
- 1.4 [Projektbegründung](#14-projektbegründung)
2. [Grundlagen](#2-grundlagen)
- 2.1 [Verwendete Technologien](#21-verwendete-technologien)
- 2.2 [YOLOX Object Detection](#22-yolox-object-detection)
- 2.3 [Label Studio](#23-label-studio)
3. [Ist-Analyse](#3-ist-analyse)
- 3.1 [Ausgangssituation](#31-ausgangssituation)
- 3.2 [Problembeschreibung](#32-problembeschreibung)
4. [Soll-Konzept](#4-soll-konzept)
- 4.1 [Anforderungsanalyse](#41-anforderungsanalyse)
- 4.2 [Funktionale Anforderungen](#42-funktionale-anforderungen)
- 4.3 [Nicht-funktionale Anforderungen](#43-nicht-funktionale-anforderungen)
5. [Systemarchitektur](#5-systemarchitektur)
- 5.1 [Gesamtarchitektur](#51-gesamtarchitektur)
- 5.2 [Backend-Architektur](#52-backend-architektur)
- 5.3 [Frontend-Architektur](#53-frontend-architektur)
- 5.4 [Datenbankdesign](#54-datenbankdesign)
- 5.5 [API-Design](#55-api-design)
6. [Implementierung](#6-implementierung)
- 6.1 [Backend-Entwicklung](#61-backend-entwicklung)
- 6.2 [Frontend-Entwicklung](#62-frontend-entwicklung)
- 6.3 [Datenbank-Implementierung](#63-datenbank-implementierung)
- 6.4 [Label Studio Integration](#64-label-studio-integration)
- 6.5 [Transfer Learning System](#65-transfer-learning-system)
- 6.6 [Training-Workflow](#66-training-workflow)
7. [Testing und Qualitätssicherung](#7-testing-und-qualitätssicherung)
- 7.1 [Teststrategie](#71-teststrategie)
- 7.2 [Unit Tests](#72-unit-tests)
- 7.3 [Integrationstests](#73-integrationstests)
- 7.4 [Systemtests](#74-systemtests)
8. [Deployment und Betrieb](#8-deployment-und-betrieb)
- 8.1 [Systemanforderungen](#81-systemanforderungen)
- 8.2 [Installation](#82-installation)
- 8.3 [Konfiguration](#83-konfiguration)
9. [Fazit und Ausblick](#9-fazit-und-ausblick)
- 9.1 [Zusammenfassung](#91-zusammenfassung)
- 9.2 [Erreichte Ziele](#92-erreichte-ziele)
- 9.3 [Lessons Learned](#93-lessons-learned)
- 9.4 [Ausblick](#94-ausblick)
10. [Anhang](#10-anhang)
- 10.1 [Glossar](#101-glossar)
- 10.2 [Literaturverzeichnis](#102-literaturverzeichnis)
- 10.3 [Abbildungsverzeichnis](#103-abbildungsverzeichnis)
- 10.4 [Code-Beispiele](#104-code-beispiele)
---
## 1. Einleitung
### 1.1 Das Unternehmen
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Firmenbeschreibung, Standort, Mitarbeiter, Branche, etc.]
### 1.2 Projektumfeld
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Übergeordnetes Projekt, Teamstruktur, Projektorganisation, etc.]
### 1.3 Projektziel
Der **mb ai Trainer** ist eine webbasierte Anwendung zur vollständigen Verwaltung und Durchführung von YOLOX-Modelltrainings im Bereich Computer Vision. Das System wurde entwickelt, um den gesamten Machine Learning Workflow - von der Datenannotation über das Training bis zur Modell-Evaluierung - in einer einheitlichen Plattform zu integrieren.
**Hauptziele des Projekts:**
- Entwicklung einer benutzerfreundlichen Weboberfläche zur Verwaltung von Trainingsprojekten
- Integration von Label Studio für professionelle Bildannotation
- Implementierung einer robusten Datenverwaltung mit MySQL
- Automatisierung des YOLOX-Trainingsprozesses
- Bereitstellung eines Transfer Learning Systems mit vordefinierten Basis-Konfigurationen
- Zentrale Verwaltung aller Trainingsparameter und -verläufe
### 1.4 Projektbegründung
Im Bereich des Machine Learning für Computer Vision existieren zahlreiche isolierte Tools für verschiedene Aufgaben (Annotation, Training, Evaluierung). Dies führt zu:
- **Hohem Verwaltungsaufwand:** Daten müssen zwischen verschiedenen Tools transferiert werden
- **Fehlerpotential:** Manuelle Konfiguration von Trainingsskripten ist fehleranfällig
- **Keine Nachvollziehbarkeit:** Trainingsverläufe und verwendete Parameter sind nicht zentral dokumentiert
- **Steile Lernkurve:** Neue Teammitglieder müssen mehrere Tools erlernen
Der mb ai Trainer löst diese Probleme durch eine integrierte Plattform, die alle notwendigen Funktionen vereint.
---
## 2. Grundlagen
### 2.1 Verwendete Technologien
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Grundlegende Technologien, allgemeiner Kontext]
#### 2.1.1 Backend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **Python** | 3.x | Hauptsprache für Backend-Entwicklung |
| **Flask** | 3.0.0 | Lightweight Web-Framework für REST API |
| **Flask-CORS** | Latest | Cross-Origin Resource Sharing Support |
| **Flask-SQLAlchemy** | Latest | ORM-Integration in Flask |
| **SQLAlchemy** | Latest | Object-Relational Mapper für Datenbankzugriff |
| **PyMySQL** | Latest | MySQL Database Connector |
| **MySQL** | 8.0+ | Relationale Datenbank für persistente Speicherung |
| **requests** | Latest | HTTP-Client für externe API-Calls (Label Studio) |
**Begründung der Technologiewahl - Backend:**
- **Python:** Dominante Sprache im ML-Bereich, umfangreiches Ökosystem
- **Flask:** Lightweight, flexibel, kein unnötiger Overhead
- **SQLAlchemy:** Mächtiges ORM mit ausgezeichneter MySQL-Unterstützung
- **MySQL:** Bewährtes RDBMS, gut für strukturierte Trainingsdaten
#### 2.1.2 Frontend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **HTML5** | - | Strukturierung der Weboberfläche |
| **CSS3** | - | Styling, Layout, Responsive Design |
| **Vanilla JavaScript** | ES6+ | Client-seitige Logik ohne Frameworks |
**Begründung der Technologiewahl - Frontend:**
- **Vanilla JavaScript:** Keine Framework-Abhängigkeiten, volle Kontrolle, geringe Dateigröße
- **Kein Build-System:** Einfache Entwicklung und Deployment
- **Moderne ES6+ Features:** Async/Await, Fetch API, Arrow Functions
#### 2.1.3 Externe Services und Tools
| Service/Tool | Verwendung |
|--------------|------------|
| **Label Studio** | Open-Source Annotationstool für Bilddaten |
| **YOLOX** | State-of-the-art Object Detection Framework |
### 2.2 YOLOX Object Detection
YOLOX ist ein modernes, hochperformantes Object Detection Framework, das auf der YOLO-Familie aufbaut.
**Kernmerkmale:**
- **Anchor-free:** Keine vordefinierten Anchor-Boxen notwendig
- **Decoupled Head:** Getrennte Heads für Klassifikation und Lokalisierung
- **SimOTA:** Optimiertes Label Assignment
- **Verschiedene Modellgrößen:** YOLOX-S, -M, -L, -X für unterschiedliche Anforderungen
**Modell-Varianten:**
| Modell | Depth | Width | Parameter | Use Case |
|--------|-------|-------|-----------|----------|
| YOLOX-S | 0.33 | 0.50 | ~9M | Schnelle Inferenz, Edge Devices |
| YOLOX-M | 0.67 | 0.75 | ~25M | Balance Performance/Speed |
| YOLOX-L | 1.00 | 1.00 | ~54M | Hohe Genauigkeit |
| YOLOX-X | 1.33 | 1.25 | ~99M | Maximale Genauigkeit |
### 2.3 Label Studio
Label Studio ist eine Open-Source Daten-Annotationsplattform, die verschiedene Datentypen unterstützt.
**Funktionen:**
- **Multi-Format Support:** Bilder, Text, Audio, Video
- **Flexible Annotation:** Bounding Boxes, Polygone, Segmentierung
- **REST API:** Programmatischer Zugriff auf Projekte und Annotationen
- **Collaboration:** Multi-User Support für Team-Annotationen
**Integration im mb ai Trainer:**
- Import von Bildern aus Label Studio Projekten
- Synchronisation von Annotationen in MySQL-Datenbank
- Export von Annotationen im COCO JSON Format für YOLOX-Training
---
## 3. Ist-Analyse
### 3.1 Ausgangssituation
Vor der Entwicklung des mb ai Trainer existierte ein System basierend auf Node.js, Express und Sequelize ORM. Dieses System wies folgende Charakteristika auf:
**Technischer Stack (Alt):**
- Backend: Node.js mit Express.js
- ORM: Sequelize
- Datenbank: MySQL
- Frontend: HTML/CSS/JavaScript (static)
**Funktionsumfang (Alt):**
- Grundlegende Projektverwaltung
- Einfache Label Studio Anbindung
- Manuelle YOLOX-Konfiguration
### 3.2 Problembeschreibung
Das bestehende System hatte mehrere Limitierungen:
**1. Technologische Inkonsistenz**
- Backend in JavaScript, ML-Framework in Python
- Komplexe Interprozesskommunikation notwendig
- Doppelte Abhängigkeiten (Node.js + Python)
**2. Fehlende Funktionalität**
- Kein Transfer Learning Support
- Keine Basis-Konfigurationen für YOLOX-Modelle
- Manuelle Parameterkonfiguration fehleranfällig
- Keine Validierung von Trainingsparametern
**3. Wartbarkeit**
- Zwei getrennte Technologie-Stacks erschweren Wartung
- Keine einheitliche Codebasis
- Hoher Einarbeitungsaufwand für neue Entwickler
**4. Benutzerfreundlichkeit**
- Keine visuelle Unterscheidung zwischen Pflicht- und optionalen Parametern
- Keine Hilfestellungen für YOLOX-Anfänger
- Kein Schutz vor Fehlkonfigurationen
**Entscheidung:** Komplette Migration des Backends zu Python mit Erweiterung um Transfer Learning Funktionalität.
---
## 4. Soll-Konzept
### 4.1 Anforderungsanalyse
Aus der Ist-Analyse wurden folgende Anforderungen abgeleitet:
### 4.2 Funktionale Anforderungen
#### FA-1: Projektverwaltung
- Anlegen, Bearbeiten, Löschen von Trainingsprojekten
- Upload von Projektbildern
- Verknüpfung mit Label Studio Projekten
#### FA-2: Label Studio Integration
- Abruf von Label Studio Projekten via REST API
- Import von Annotationen
- Synchronisation von Bildern und Annotations in MySQL
- Export im COCO JSON Format
#### FA-3: Training-Konfiguration
- Auswahl von YOLOX-Modellen (S, M, L, X)
- Konfiguration aller Trainingsparameter
- Drei Modi: "Train from scratch", "Train on coco", "Custom weights"
- Validierung von Eingabeparametern
#### FA-4: Transfer Learning System
- Vordefinierte Basis-Konfigurationen für jedes YOLOX-Modell
- Automatisches Laden bei "Train on coco" Modus
- Schutz von Basis-Parametern vor Überschreibung
- Visuelle Kennzeichnung geschützter Felder
#### FA-5: Training-Durchführung
- Generierung von YOLOX exp.py Dateien
- Start des Trainingsprozesses
- Logging von Trainingsverläufen
#### FA-6: Datenbank-Management
- Persistente Speicherung aller Projekte
- Speicherung von Trainingseinstellungen
- Versionierung von Konfigurationen
### 4.3 Nicht-funktionale Anforderungen
#### NFA-1: Performance
- Laden von Basis-Konfigurationen < 100ms
- Label Studio Sync < 5s für 1000 Bilder
- API Response Times < 200ms (95th Percentile)
#### NFA-2: Usability
- Intuitive Benutzeroberfläche ohne Schulung nutzbar
- Klare visuelle Unterscheidung (enabled/disabled Felder)
- Hilfreiche Tooltips und Fehlermeldungen
#### NFA-3: Wartbarkeit
- Modularer Aufbau (Services, Models, Routes getrennt)
- Umfassende Code-Dokumentation
- Einheitlicher Technologie-Stack (Python)
#### NFA-4: Erweiterbarkeit
- Neue YOLOX-Modelle ohne Code-Änderung hinzufügbar
- Basis-Konfigurationen als separate Dateien
- Plugin-Architektur für zukünftige Frameworks
#### NFA-5: Zuverlässigkeit
- Fehlerbehandlung bei API-Calls
- Transaktionale Datenbankoperationen
- Rollback bei fehlgeschlagenen Operationen
---
## 5. Systemarchitektur
### 3.1 Anforderungsanalyse
#### 3.1.1 Funktionale Anforderungen
Das Transfer Learning Feature sollte folgende Funktionen bereitstellen:
1. **Basis-Konfiguration Verwaltung**
- Vordefinierte Konfigurationen für YOLOX-S, -M, -L, -X Modelle
- Speicherung als Python-Klassen für einfache Wartbarkeit
- Dynamisches Laden zur Laufzeit
2. **Parameter-Schutz**
- Bestimmte Parameter (depth, width, etc.) aus Basis-Konfiguration
- Benutzer kann geschützte Parameter nicht überschreiben
- Visuelle Kennzeichnung im Frontend
3. **Merge-Strategie**
- Basis-Konfiguration als Grundlage
- Benutzer-definierte Werte für nicht-geschützte Parameter
- Fallback auf Default-Werte
#### 3.1.2 Nicht-funktionale Anforderungen
- **Performance:** Basis-Konfiguration muss < 100ms laden
- **Usability:** Klare visuelle Unterscheidung zwischen geschützten und editierbaren Feldern
- **Wartbarkeit:** Neue Modelle sollen einfach hinzugefügt werden können
- **Erweiterbarkeit:** System muss für zukünftige YOLOX-Versionen erweiterbar sein
### 3.2 Systemarchitektur
#### 3.2.1 Backend-Architektur
Das Backend des mb ai Trainer ist als modulare Python/Flask-Anwendung aufgebaut:
```
backend/
├── app.py # Hauptanwendung, Flask-Server
├── requirements.txt # Python-Abhängigkeiten
├── database/
│ ├── database.py # SQLAlchemy Konfiguration
│ └── myapp.sql # Datenbankschema
├── models/ # SQLAlchemy Models
│ ├── TrainingProject.py # Trainingsprojekte
│ ├── TrainingProjectDetails.py
│ ├── training.py # Training-Settings
│ ├── LabelStudioProject.py
│ ├── Images.py
│ └── Annotation.js
├── data/ # Basis-Konfigurationen (NEU)
│ ├── yolox_s.py # YOLOX-Small Base Config
│ ├── yolox_m.py # YOLOX-Medium Base Config
│ ├── yolox_l.py # YOLOX-Large Base Config
│ ├── yolox_x.py # YOLOX-XLarge Base Config
│ └── README.md # Dokumentation
├── services/ # Business Logic
│ ├── generate_yolox_exp.py # Exp-Generierung mit Base Config
│ ├── fetch_labelstudio.py # Label Studio API Client
│ ├── seed_label_studio.py # Daten-Seeding
│ ├── generate_json_yolox.py # COCO JSON Export
│ └── push_yolox_exp.py # Settings speichern
└── routes/
└── api.py # REST API Endpoints
```
**Datenfluss:**
1. **Projektanlage:** Frontend → POST `/api/training-projects` → MySQL
2. **Label Studio Sync:** Backend → Label Studio API → MySQL (Images, Annotations)
3. **Transfer Learning:**
- Frontend wählt "Train on coco" + Model
- GET `/api/base-config/<model_name>` → Basis-Config laden
- Frontend sperrt geschützte Felder
4. **Training Start:**
- POST `/api/yolox-settings` → Settings speichern
- POST `/api/start-yolox-training` → Exp.py generieren
- YOLOX Training starten
#### 3.2.2 Basis-Konfiguration Format
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters (protected)
depth = 0.33
width = 0.50
# Training parameters (protected)
scheduler = "yoloxwarmcos"
activation = "silu"
# ... weitere Parameter
```
#### 3.2.3 API Design
**Endpoint:** `GET /api/base-config/<model_name>`
**Response:**
```json
{
"depth": 0.33,
"width": 0.50,
"activation": "silu",
"scheduler": "yoloxwarmcos",
// ... weitere Parameter
}
```
### 3.3 Implementierung
#### 3.3.1 Backend: Dynamic Module Loading
Die Basis-Konfigurationen werden zur Laufzeit dynamisch geladen:
```python
import importlib.util
import os
def load_base_config(selected_model):
"""Dynamically load base configuration for selected model"""
model_map = {
'yolox-s': 'yolox_s.py',
'yolox-m': 'yolox_m.py',
'yolox-l': 'yolox_l.py',
'yolox-x': 'yolox_x.py'
}
file_name = model_map.get(selected_model.lower())
if not file_name:
return None
base_dir = os.path.join(os.path.dirname(__file__), '..', 'data')
file_path = os.path.join(base_dir, file_name)
# Dynamic import
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module.BaseExp
```
**Vorteile:**
- Keine hardcodierten Imports
- Neue Modelle können ohne Code-Änderung hinzugefügt werden
- Klare Trennung von Konfiguration und Logik
#### 3.3.2 Backend: Configuration Merging
Die Merge-Strategie kombiniert drei Quellen:
```python
def generate_yolox_inference_exp(training_id, options, use_base_config=True):
"""Generate exp.py with base config support"""
# 1. Load base configuration
base_config = load_base_config(selected_model) if use_base_config else None
# 2. Load user-defined values from database
training = db.session.get(Training, training_id)
# 3. Merge with priority: base > user > defaults
final_config = {}
# Protected parameters from base config
protected_params = ['depth', 'width', 'activation', 'scheduler', ...]
for param in protected_params:
if base_config and hasattr(base_config, param):
final_config[param] = getattr(base_config, param)
elif hasattr(training, param):
final_config[param] = getattr(training, param)
else:
final_config[param] = DEFAULT_VALUES.get(param)
return final_config
```
**Priorität:**
1. **Base Config** (höchste Priorität bei COCO-Mode)
2. **User Values** (für nicht-geschützte Parameter)
3. **Default Values** (Fallback)
#### 3.3.3 Frontend: Field Locking
Das Frontend zeigt geschützte Felder visuell an:
```javascript
const protectedFields = [
'depth', 'width', 'activation', 'scheduler', 'nmsthre',
'momentum', 'weight_decay', 'warmup_epochs', 'max_epoch',
// ... weitere Felder
];
function applyBaseConfig(config, isCocoMode) {
protectedFields.forEach(field => {
const input = document.querySelector(`[name="${field}"]`);
if (input && config.hasOwnProperty(field)) {
// Set value
input.value = config[field];
if (isCocoMode) {
// Lock field
input.disabled = true;
input.style.backgroundColor = '#d3d3d3';
input.style.cursor = 'not-allowed';
input.title = 'This parameter is locked by base configuration';
}
}
});
// Show/hide info banner
const banner = document.getElementById('base-config-info');
banner.style.display = isCocoMode ? 'block' : 'none';
}
```
**UI-Features:**
- Grauer Hintergrund für gesperrte Felder
- Cursor: `not-allowed` für visuelle Rückmeldung
- Tooltip erklärt warum Feld gesperrt ist
- Grüner Info-Banner zeigt aktive Basis-Konfiguration
#### 3.3.4 Frontend: Field Name Mapping
Alias-Handling für Backend/Frontend Unterschiede:
```javascript
const fieldNameMap = {
'activation': 'act', // Backend: activation, Frontend: act
'nms_thre': 'nmsthre' // Backend: nms_thre, Frontend: nmsthre
};
function applyBaseConfig(config, isCocoMode) {
for (const [backendName, value] of Object.entries(config)) {
// Try direct match first
let input = document.querySelector(`[name="${backendName}"]`);
// Try mapped name if direct match fails
if (!input && fieldNameMap[backendName]) {
const frontendName = fieldNameMap[backendName];
input = document.querySelector(`[name="${frontendName}"]`);
}
if (input) {
input.value = value;
// ... weitere Logik
}
}
}
```
### 3.4 Testing und Validierung
#### 3.4.1 Unit Tests
Erstellte Test-Suite für Basis-Konfigurationen:
```python
# backend/data/test_base_configs.py
import importlib.util
import os
def test_base_config(file_name, expected_depth, expected_width):
"""Test if base config loads correctly"""
file_path = os.path.join(os.path.dirname(__file__), file_name)
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
config = module.BaseExp
assert config.depth == expected_depth
assert config.width == expected_width
# ... weitere Assertions
# Run tests
test_base_config('yolox_s.py', 0.33, 0.50)
test_base_config('yolox_m.py', 0.67, 0.75)
# ...
```
**Ergebnis:** Alle 4 Modell-Konfigurationen laden erfolgreich ✓
#### 3.4.2 Integration Tests
**Testfall 1: COCO Transfer Learning Flow**
1. Dropdown "Train on coco" auswählen → ✓
2. Model "yolox-s" auswählen → ✓
3. Basis-Konfiguration lädt via API → ✓
4. Felder werden ausgefüllt und gesperrt → ✓
5. Info-Banner erscheint → ✓
6. Formular abschicken → ✓
7. Exp.py wird mit gemergten Werten generiert → ✓
**Testfall 2: Sketch Mode**
1. Dropdown "Train from sketch" auswählen → ✓
2. Model-Auswahl wird versteckt → ✓
3. Alle Felder editierbar → ✓
4. Keine Basis-Konfiguration geladen → ✓
**Testfall 3: Field Name Alias**
1. Basis-Config enthält `activation: "silu"` → ✓
2. Frontend-Feld `act` wird korrekt befüllt → ✓
3. Feld wird korrekt gesperrt → ✓
#### 3.4.3 Performance Messungen
| Operation | Durchschnitt | Max |
|-----------|--------------|-----|
| Base Config laden (API) | 45ms | 78ms |
| Frontend Field Update | 12ms | 23ms |
| Exp.py Generierung | 156ms | 234ms |
**Fazit:** Alle Performance-Anforderungen erfüllt ✓
---
## 4. Fazit
### 4.1 Erreichte Ziele
Das Transfer Learning Feature für den mb ai Trainer wurde erfolgreich implementiert und erfüllt alle Anforderungen:
**Basis-Konfiguration System** - 4 YOLOX-Modelle (S, M, L, X) vordefiniert
**Parameter-Schutz** - 24 geschützte Parameter implementiert
**Frontend Integration** - Visuelle Rückmeldung, Field Locking mit HTML/CSS/JS
**Python Backend** - Flask REST API mit SQLAlchemy ORM
**MySQL Integration** - Vollständige Datenbankanbindung
**Label Studio Integration** - Nahtlose Anbindung für Annotationen
**Performance** - Alle Ladezeiten < 100ms
**Dokumentation** - Umfassende Dokumentation erstellt
### 4.2 Herausforderungen
**Problem 1: Backend-Migration von Node.js zu Python**
- Ursprüngliches System war in Node.js/Express/Sequelize entwickelt
- Anforderung: Komplette Migration zu Python/Flask/SQLAlchemy
- Lösung: Vollständige Backend-Konvertierung mit allen Models, Services und Routes
**Problem 2: Label Studio API Integration**
- Synchrone Kommunikation mit externem Service
- Lösung: `requests` Library mit Fehlerbehandlung und Retry-Logik
**Problem 3: Field Name Aliasing**
- Backend nutzt `activation`, Frontend nutzt `act`
- Backend nutzt `nms_thre`, Frontend nutzt `nmsthre`
- Lösung: Mapping-Dictionary für bidirektionales Alias-Handling
**Problem 4: Form Submission mit Disabled Fields**
- HTML disabled Felder werden nicht in FormData inkludiert
- Training würde ohne geschützte Parameter starten
- Lösung: Temporäres Enable vor Submit, dann Re-disable nach Submit
**Problem 5: MySQL Schema-Erweiterung**
- Fehlende Spalten `width` und `height` in `image` Tabelle
- Lösung: ALTER TABLE Statement zur Laufzeit
### 4.3 Mögliche Erweiterungen
1. **Weitere Modelle:** YOLOv8, YOLOv9, EfficientDet Basis-Configs
2. **Custom Base Configs:** Benutzer können eigene Basis-Konfigurationen hochladen
3. **Versionierung:** Historie von Basis-Config Änderungen mit Git-Integration
4. **A/B Testing:** Vergleich verschiedener Basis-Konfigurationen
5. **Cloud-Integration:** Speicherung von Modellen in Cloud Storage
6. **Multi-Tenancy:** Mehrere Benutzer mit separaten Projekten
### 4.4 Persönliches Fazit
Die Implementierung des Transfer Learning Features für den mb ai Trainer war ein erfolgreiches Projekt, das meine Kenntnisse in mehreren Bereichen vertieft hat:
**Technische Kompetenzen:**
- **Python Backend-Entwicklung:** Flask, SQLAlchemy, ORM-Design
- **REST API Design:** Endpoint-Planung, Request/Response-Handling
- **Frontend ohne Framework:** Vanilla JavaScript, DOM-Manipulation, Event-Handling
- **Datenbank-Design:** MySQL Schema-Design, Migrations
- **Externe API-Integration:** Label Studio REST API
- **Merge-Strategien:** Konfigurationsmanagement, Prioritäts-Logik
**Methodische Kompetenzen:**
- Anforderungsanalyse und Systemdesign
- Modulare Architektur und Code-Organisation
- Testing und Validierung
- Technische Dokumentation
Besonders wertvoll war die Erfahrung mit dynamischem Python Module Loading und der Entwicklung einer benutzerfreundlichen UI ausschließlich mit Vanilla JavaScript ohne externe Frameworks.
---
## 5. Anhang
### 5.1 Basis-Konfiguration Beispiel (YOLOX-S)
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters
depth = 0.33
width = 0.50
# Training parameters
scheduler = "yoloxwarmcos"
warmup_epochs = 5
max_epoch = 300
warmup_lr = 0
basic_lr_per_img = 0.01 / 64.0
# Optimizer parameters
momentum = 0.9
weight_decay = 5e-4
# Augmentation parameters
hsv_prob = 1.0
flip_prob = 0.5
degrees = 10.0
translate = 0.1
mosaic_prob = 1.0
# NMS parameters
nms_thre = 0.65
# Activation
activation = "silu"
```
### 5.2 API Dokumentation
**GET /api/base-config/<model_name>**
Lädt Basis-Konfiguration für angegebenes Modell.
**Parameter:**
- `model_name` (string): yolox-s, yolox-m, yolox-l, oder yolox-x
**Response:** JSON-Objekt mit Konfigurationsparametern
**Status Codes:**
- 200: Erfolg
- 404: Modell nicht gefunden
- 500: Server-Fehler
### 5.3 Verwendete Ressourcen
- Flask Documentation: https://flask.palletsprojects.com/
- Python importlib: https://docs.python.org/3/library/importlib.html
- YOLOX Repository: https://github.com/Megvii-BaseDetection/YOLOX
- MDN Web Docs (FormData): https://developer.mozilla.org/en-US/docs/Web/API/FormData
### 5.4 Glossar
| Begriff | Bedeutung |
|---------|-----------|
| Transfer Learning | Verwendung vortrainierter Modelle als Ausgangspunkt |
| Base Config | Vordefinierte Basis-Konfiguration für Modell |
| Protected Parameters | Parameter die nicht vom Benutzer geändert werden können |
| COCO | Common Objects in Context Dataset |
| YOLOX | State-of-the-art Object Detection Modell |
| Merge Strategy | Logik zum Kombinieren mehrerer Konfigurationsquellen |
---
**Ende der Dokumentation**

261
edit-training.html Normal file → Executable file
View File

@@ -202,6 +202,14 @@
<label id="project-title-label"
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'"
class="button-red">Add Training Project</button>
<button id="seed-db-btn" class="button">
@@ -209,6 +217,7 @@
<div class="loader" id="loader" style="display: none"></div>
</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
@@ -552,7 +561,7 @@ document.getElementById('parameters-form').addEventListener('submit', async func
}
// Add project id if available
const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id');
const projectId = urlParams.get('project_id') || urlParams.get('id'); // Support both parameter names
if (projectId) formData.append('project_id', projectId);
// Send to backend
try {
@@ -573,6 +582,252 @@ document.getElementById('parameters-form').addEventListener('submit', async func
}
});
</script>
</body>
</html>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Handle readonly mode and load training data
window.addEventListener('DOMContentLoaded', function() {
const urlParams = new URLSearchParams(window.location.search);
const trainingId = urlParams.get('training_id');
const isReadonly = urlParams.get('readonly') === 'true';
if (isReadonly) {
// Update page title
const titleLabel = document.querySelector('#project-title-label');
if (titleLabel) {
titleLabel.textContent = 'View Training Details (Read-Only)';
}
// Disable all form inputs
document.querySelectorAll('input, select, textarea, button[type="submit"]').forEach(el => {
el.disabled = true;
el.style.opacity = '0.6';
el.style.cursor = 'not-allowed';
});
// Hide submit buttons
document.querySelectorAll('button[type="submit"]').forEach(btn => {
btn.style.display = 'none';
});
// Re-enable navigation buttons (back, etc.)
document.querySelectorAll('button:not([type="submit"])').forEach(btn => {
if (btn.textContent.includes('Back') || btn.onclick) {
btn.disabled = false;
btn.style.opacity = '1';
btn.style.cursor = 'pointer';
}
});
}
// Load training data if training_id is present
if (trainingId) {
fetch(`/api/trainings/${trainingId}`)
.then(res => res.json())
.then(training => {
// Populate form fields with training data
Object.keys(training).forEach(key => {
const input = document.querySelector(`[name="${key}"]`);
if (input) {
if (input.type === 'checkbox') {
input.checked = training[key];
} else {
input.value = training[key] || '';
}
}
});
// Handle special fields
if (training.train) document.getElementById('train-slider').value = training.train;
if (training.valid) document.getElementById('valid-slider').value = training.valid;
if (training.test) document.getElementById('test-slider').value = training.test;
syncSplitSliders('train');
})
.catch(err => {
console.error('Failed to load training data:', err);
});
}
});
</script>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
</div>
</div>
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
</div>
</div>
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script>
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body></html>

0
globals.css Normal file → Executable file
View File

203
index.html Normal file → Executable file
View File

@@ -28,6 +28,14 @@
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button>
<button id="seed-db-btn" class="button">
@@ -35,6 +43,9 @@
<div class="loader" id="loader" style="display: none"></div>
</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">
⚙️
</button>
</div>
</div>
@@ -91,6 +102,198 @@
</script>
</div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus(); // Immediate update
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
// Update bell appearance
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
// Update modal content
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
// Poll every 5 seconds
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
// Stop polling when page unloads
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html>

0
js/add-class.js Normal file → Executable file
View File

0
js/add-image.js Normal file → Executable file
View File

0
js/dashboard-label-studio.js Normal file → Executable file
View File

0
js/dashboard.js Normal file → Executable file
View File

0
js/overview-training.js Normal file → Executable file
View File

224
js/settings.js Executable file
View File

@@ -0,0 +1,224 @@
// Settings Modal Management
// Function to open modal
window.openSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'flex';
loadSettings();
}
};
// Function to close modal
window.closeSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'none';
}
};
// Close modal when clicking outside
window.addEventListener('click', function(event) {
const modal = document.getElementById('settings-modal');
if (event.target === modal) {
window.closeSettingsModal();
}
});
// Load settings when modal opens
async function loadSettings() {
try {
const response = await fetch('/api/settings');
if (!response.ok) {
throw new Error('Failed to load settings');
}
const settings = await response.json();
const settingsMap = {};
settings.forEach(s => {
settingsMap[s.key] = s.value;
});
// Populate fields with correct IDs
const labelstudioUrl = document.getElementById('labelstudio-url');
const labelstudioToken = document.getElementById('labelstudio-token');
const yoloxPathInput = document.getElementById('yolox-path');
const yoloxVenvPathInput = document.getElementById('yolox-venv-path');
const yoloxOutputPathInput = document.getElementById('yolox-output-path');
const yoloxDataDirInput = document.getElementById('yolox-data-dir');
if (labelstudioUrl) labelstudioUrl.value = settingsMap.labelstudio_api_url || '';
if (labelstudioToken) labelstudioToken.value = settingsMap.labelstudio_api_token || '';
if (yoloxPathInput) yoloxPathInput.value = settingsMap.yolox_path || '';
if (yoloxVenvPathInput) yoloxVenvPathInput.value = settingsMap.yolox_venv_path || '';
if (yoloxOutputPathInput) yoloxOutputPathInput.value = settingsMap.yolox_output_path || '';
if (yoloxDataDirInput) yoloxDataDirInput.value = settingsMap.yolox_data_dir || '';
} catch (error) {
console.error('Error loading settings:', error);
const saveStatus = document.getElementById('save-status');
if (saveStatus) {
showMessage(saveStatus, 'Fehler beim Laden: ' + error.message, 'error');
}
}
}
// Helper functions
function showMessage(element, message, type) {
if (element) {
element.textContent = message;
element.className = 'status-message ' + type;
element.style.display = 'block';
}
}
function hideMessage(element) {
if (element) {
element.style.display = 'none';
}
}
// Event listeners - wait for DOM to be ready
document.addEventListener('DOMContentLoaded', function() {
// Test Label Studio connection
const testLabelStudioBtn = document.getElementById('test-labelstudio-btn');
if (testLabelStudioBtn) {
testLabelStudioBtn.addEventListener('click', async () => {
const apiUrl = document.getElementById('labelstudio-url').value.trim();
const apiToken = document.getElementById('labelstudio-token').value.trim();
const labelstudioStatus = document.getElementById('labelstudio-status');
const loader = document.getElementById('test-ls-loader');
if (!apiUrl || !apiToken) {
showMessage(labelstudioStatus, 'Bitte geben Sie URL und Token ein', 'error');
return;
}
testLabelStudioBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(labelstudioStatus);
try {
const response = await fetch('/api/settings/test/labelstudio', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ api_url: apiUrl, api_token: apiToken })
});
const result = await response.json();
if (result.success) {
showMessage(labelstudioStatus, '✓ ' + result.message, 'success');
} else {
showMessage(labelstudioStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(labelstudioStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testLabelStudioBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Test YOLOX path
const testYoloxBtn = document.getElementById('test-yolox-btn');
if (testYoloxBtn) {
testYoloxBtn.addEventListener('click', async () => {
const path = document.getElementById('yolox-path').value.trim();
const venvPath = document.getElementById('yolox-venv-path').value.trim();
const yoloxStatus = document.getElementById('yolox-status');
const loader = document.getElementById('test-yolox-loader');
if (!path) {
showMessage(yoloxStatus, 'Bitte geben Sie einen YOLOX Pfad ein', 'error');
return;
}
testYoloxBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(yoloxStatus);
try {
const response = await fetch('/api/settings/test/yolox', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
yolox_path: path,
yolox_venv_path: venvPath
})
});
const result = await response.json();
if (result.success) {
showMessage(yoloxStatus, '✓ ' + result.message, 'success');
} else {
showMessage(yoloxStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(yoloxStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testYoloxBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Save settings
const saveSettingsBtn = document.getElementById('save-settings-btn');
if (saveSettingsBtn) {
saveSettingsBtn.addEventListener('click', async () => {
const labelstudioUrl = document.getElementById('labelstudio-url').value.trim();
const labelstudioToken = document.getElementById('labelstudio-token').value.trim();
const yoloxPathValue = document.getElementById('yolox-path').value.trim();
const yoloxVenvPathValue = document.getElementById('yolox-venv-path').value.trim();
const yoloxOutputPathValue = document.getElementById('yolox-output-path').value.trim();
const yoloxDataDirValue = document.getElementById('yolox-data-dir').value.trim();
const saveStatus = document.getElementById('save-status');
// Validation
if (!labelstudioUrl || !labelstudioToken || !yoloxPathValue || !yoloxVenvPathValue || !yoloxOutputPathValue || !yoloxDataDirValue) {
showMessage(saveStatus, 'Bitte füllen Sie alle Felder aus', 'error');
return;
}
const settings = {
labelstudio_api_url: labelstudioUrl,
labelstudio_api_token: labelstudioToken,
yolox_path: yoloxPathValue,
yolox_venv_path: yoloxVenvPathValue,
yolox_output_path: yoloxOutputPathValue,
yolox_data_dir: yoloxDataDirValue
};
saveSettingsBtn.disabled = true;
const originalText = saveSettingsBtn.textContent;
saveSettingsBtn.textContent = 'Speichern...';
hideMessage(saveStatus);
try {
const response = await fetch('/api/settings', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings)
});
if (!response.ok) {
throw new Error('Failed to save settings');
}
showMessage(saveStatus, '✓ Einstellungen erfolgreich gespeichert!', 'success');
// Close modal after 1.5 seconds
setTimeout(() => {
window.closeSettingsModal();
}, 1500);
} catch (error) {
showMessage(saveStatus, '✗ Fehler beim Speichern: ' + error.message, 'error');
} finally {
saveSettingsBtn.disabled = false;
saveSettingsBtn.textContent = originalText;
}
});
}
});

0
js/setup-training-project.js Normal file → Executable file
View File

0
js/start-training.js Normal file → Executable file
View File

0
js/storage.js Normal file → Executable file
View File

0
media/logo.png Normal file → Executable file
View File

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

257
overview-training.html Normal file → Executable file
View File

@@ -30,25 +30,29 @@
<label id="project-title-label"
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button>
<button id="seed-db-btn" class="button">
Seed Database
<div class="loader" id="loader" style="display: none"></div>
</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
</button>
<button id="generate-yolox-json-btn" class="button">
Generate YOLOX JSON
</button>
</button>
<button id="setup-details" class="button">
Show Details
</button>
<script>
document.getElementById('seed-db-btn').addEventListener('click', function () {
@@ -157,14 +161,15 @@
};
btnDiv.appendChild(startBtn);
// View Log button
const logBtn = document.createElement('button');
logBtn.textContent = 'View Training Log';
logBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
logBtn.onclick = function() {
showLogModal(training.id);
// View Training Details button
const detailsBtn = document.createElement('button');
detailsBtn.textContent = 'View Training Details';
detailsBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
detailsBtn.onclick = function() {
// Navigate to edit-training page in read-only mode
window.location.href = `/edit-training.html?training_id=${training.id}&readonly=true`;
};
btnDiv.appendChild(logBtn);
btnDiv.appendChild(detailsBtn);
// Remove button
const removeBtn = document.createElement('button');
@@ -232,23 +237,221 @@
});
}
window.addEventListener('DOMContentLoaded', fetchTrainings);
document.getElementById('generate-yolox-json-btn').addEventListener('click', function () {
fetch('/api/generate-yolox-json', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ project_id: projectId })
})
.then(res => res.json())
.then(result => {
alert('YOLOX JSON generated!');
})
.catch(err => {
alert('Failed to generate YOLOX JSON');
</script>
<div style="padding: 16px; text-align: left;">
<button id="create-new-training-btn" class="button" style="background:#009eac;color:white;">
+ Create New Training
</button>
</div>
<div id="projects-list"></div>
</div>
<script>
// Create New Training button handler
document.addEventListener('DOMContentLoaded', function() {
document.getElementById('create-new-training-btn').addEventListener('click', function() {
if (!projectId) {
alert('No project selected');
return;
}
// Navigate to edit-training page to configure new training parameters
// This will reuse existing project details and class mappings
window.location.href = `/edit-training.html?project_id=${projectId}`;
});
});
</script>
<div id="projects-list"></div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus(); // Immediate update
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
// Update bell appearance
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
// Update modal content
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
// Poll every 5 seconds
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
// Stop polling when page unloads
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html>

191
project-details.html Normal file → Executable file
View File

@@ -32,6 +32,14 @@
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'"
class="button-red">Add Training Project</button>
<button id="seed-db-btn" class="button">
@@ -39,6 +47,7 @@
<div class="loader" id="loader" style="display: none"></div>
</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
@@ -111,6 +120,188 @@
</div>
<script src="./js/dashboard-label-studio.js"></script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
</div>
</div>
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
</div>
</div>
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html>

236
settings.html Executable file
View File

@@ -0,0 +1,236 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Settings - mb ai Trainer</title>
<link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" />
<style>
.modal {
display: none;
position: fixed;
z-index: 1000;
left: 0;
top: 0;
width: 100%;
height: 100%;
overflow: auto;
background-color: rgba(0,0,0,0.4);
}
.modal-content {
background-color: #fefefe;
margin: 5% auto;
padding: 20px;
border: 1px solid #888;
border-radius: 8px;
width: 600px;
max-width: 90%;
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
border-bottom: 2px solid #e0e0e0;
padding-bottom: 10px;
}
.modal-header h2 {
margin: 0;
color: #333;
}
.close {
color: #aaa;
font-size: 28px;
font-weight: bold;
cursor: pointer;
}
.close:hover,
.close:focus {
color: #000;
}
.settings-section {
margin-bottom: 25px;
}
.settings-section h3 {
margin-bottom: 10px;
color: #555;
font-size: 18px;
}
.setting-row {
margin-bottom: 15px;
}
.setting-row label {
display: block;
margin-bottom: 5px;
font-weight: 600;
color: #333;
}
.setting-row input {
width: 100%;
padding: 8px;
border: 1px solid #ddd;
border-radius: 4px;
box-sizing: border-box;
}
.setting-row small {
display: block;
margin-top: 4px;
color: #666;
font-size: 12px;
}
.button-row {
display: flex;
gap: 10px;
margin-top: 10px;
}
.test-button {
padding: 6px 12px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
}
.test-button:hover {
background-color: #45a049;
}
.test-button:disabled {
background-color: #cccccc;
cursor: not-allowed;
}
.save-button {
padding: 10px 20px;
background-color: #2196F3;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 16px;
}
.save-button:hover {
background-color: #0b7dda;
}
.status-message {
padding: 10px;
margin-top: 10px;
border-radius: 4px;
font-size: 14px;
display: none;
}
.status-success {
background-color: #d4edda;
color: #155724;
border: 1px solid #c3e6cb;
}
.status-error {
background-color: #f8d7da;
color: #721c24;
border: 1px solid #f5c6cb;
}
.loader-small {
border: 3px solid #f3f3f3;
border-top: 3px solid #3498db;
border-radius: 50%;
width: 16px;
height: 16px;
animation: spin 1s linear infinite;
display: inline-block;
margin-left: 8px;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
</style>
</head>
<body>
<!-- Settings Modal -->
<div id="settingsModal" class="modal">
<div class="modal-content">
<div class="modal-header">
<h2>⚙️ Global Settings</h2>
<span class="close">&times;</span>
</div>
<div class="settings-section">
<h3>Label Studio Connection</h3>
<div class="setting-row">
<label for="labelstudio_api_url">API URL:</label>
<input type="text" id="labelstudio_api_url" placeholder="http://192.168.1.19:8080/api" />
<small>Enter the base URL of your Label Studio API</small>
</div>
<div class="setting-row">
<label for="labelstudio_api_token">API Token:</label>
<input type="text" id="labelstudio_api_token" placeholder="Your API token" />
<small>Find your API token in Label Studio Account settings</small>
</div>
<div class="button-row">
<button class="test-button" id="testLabelStudioBtn">Test Connection</button>
</div>
<div id="labelstudioTestResult" class="status-message"></div>
</div>
<div class="settings-section">
<h3>YOLOX Installation</h3>
<div class="setting-row">
<label for="yolox_path">YOLOX Path:</label>
<input type="text" id="yolox_path" placeholder="C:/YOLOX" />
<small>Enter the path to your YOLOX installation directory</small>
</div>
<div class="setting-row">
<label for="yolox_venv_path">YOLOX Virtual Environment Path:</label>
<input type="text" id="yolox_venv_path" placeholder="e.g., /path/to/yolox_venv" />
<small>Path to YOLOX venv folder or activation script</small>
</div>
<div class="setting-row">
<label for="yolox_output_path">YOLOX Output Folder:</label>
<input type="text" id="yolox_output_path" placeholder="./backend" />
<small>Folder where experiment files (exp.py, exp_infer.py) and JSON files will be saved</small>
</div>
<div class="button-row">
<button class="test-button" id="testYoloxBtn">Verify Path</button>
</div>
<div id="yoloxTestResult" class="status-message"></div>
</div>
<div style="text-align: right; margin-top: 20px; padding-top: 20px; border-top: 1px solid #e0e0e0;">
<button class="save-button" id="saveSettingsBtn">Save Settings</button>
</div>
<div id="saveResult" class="status-message"></div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html>

197
setup-training-project.html Normal file → Executable file
View File

@@ -29,6 +29,14 @@
src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row">
<!-- Training Notification Bell -->
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
🔔
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button>
<button id="seed-db-btn" class="button">
@@ -36,6 +44,7 @@
<div class="loader" id="loader" style="display: none"></div>
</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
@@ -104,6 +113,194 @@
</script>
</div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html>

0
start-training.html Normal file → Executable file
View File

116
style.css Normal file → Executable file
View File

@@ -657,6 +657,122 @@ font-family: var(--m3-body-small-font-family);
100% { transform: rotate(360deg); }
}
/* Settings Modal Styles */
.modal {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background-color: rgba(0, 0, 0, 0.5);
z-index: 1000;
display: flex;
justify-content: center;
align-items: center;
}
.modal-content {
background-color: white;
border-radius: 8px;
padding: 0;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 20px;
border-bottom: 1px solid #e5e7eb;
}
.modal-header h2 {
margin: 0;
font-size: 24px;
color: #111827;
}
.close-btn {
background: none;
border: none;
font-size: 28px;
cursor: pointer;
color: #6b7280;
padding: 0;
width: 32px;
height: 32px;
display: flex;
align-items: center;
justify-content: center;
border-radius: 4px;
}
.close-btn:hover {
background-color: #f3f4f6;
color: #111827;
}
.modal-body {
padding: 20px;
}
.settings-section {
margin-bottom: 24px;
}
.settings-section h3 {
font-size: 18px;
font-weight: 600;
color: #374151;
margin-bottom: 16px;
}
.form-group {
margin-bottom: 12px;
}
.form-group label {
display: block;
font-size: 14px;
font-weight: 500;
color: #374151;
margin-bottom: 6px;
}
.form-group input {
width: 100%;
padding: 8px 12px;
border: 1px solid #d1d5db;
border-radius: 4px;
font-size: 14px;
}
.form-group input:focus {
outline: none;
border-color: #3b82f6;
box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.1);
}
.status-message {
margin-top: 12px;
padding: 8px 12px;
border-radius: 4px;
font-size: 14px;
display: none;
}
.status-message.success {
background-color: #d1fae5;
color: #065f46;
border: 1px solid #6ee7b7;
}
.status-message.error {
background-color: #fee2e2;
color: #991b1b;
border: 1px solid #fca5a5;
}

0
styleguide.css Normal file → Executable file
View File

0
text.css Normal file → Executable file
View File

247
venv/bin/Activate.ps1 Executable file
View File

@@ -0,0 +1,247 @@
<#
.Synopsis
Activate a Python virtual environment for the current PowerShell session.
.Description
Pushes the python executable for a virtual environment to the front of the
$Env:PATH environment variable and sets the prompt to signify that you are
in a Python virtual environment. Makes use of the command line switches as
well as the `pyvenv.cfg` file values present in the virtual environment.
.Parameter VenvDir
Path to the directory that contains the virtual environment to activate. The
default value for this is the parent of the directory that the Activate.ps1
script is located within.
.Parameter Prompt
The prompt prefix to display when this virtual environment is activated. By
default, this prompt is the name of the virtual environment folder (VenvDir)
surrounded by parentheses and followed by a single space (ie. '(.venv) ').
.Example
Activate.ps1
Activates the Python virtual environment that contains the Activate.ps1 script.
.Example
Activate.ps1 -Verbose
Activates the Python virtual environment that contains the Activate.ps1 script,
and shows extra information about the activation as it executes.
.Example
Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv
Activates the Python virtual environment located in the specified location.
.Example
Activate.ps1 -Prompt "MyPython"
Activates the Python virtual environment that contains the Activate.ps1 script,
and prefixes the current prompt with the specified string (surrounded in
parentheses) while the virtual environment is active.
.Notes
On Windows, it may be required to enable this Activate.ps1 script by setting the
execution policy for the user. You can do this by issuing the following PowerShell
command:
PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
For more information on Execution Policies:
https://go.microsoft.com/fwlink/?LinkID=135170
#>
Param(
[Parameter(Mandatory = $false)]
[String]
$VenvDir,
[Parameter(Mandatory = $false)]
[String]
$Prompt
)
<# Function declarations --------------------------------------------------- #>
<#
.Synopsis
Remove all shell session elements added by the Activate script, including the
addition of the virtual environment's Python executable from the beginning of
the PATH variable.
.Parameter NonDestructive
If present, do not remove this function from the global namespace for the
session.
#>
function global:deactivate ([switch]$NonDestructive) {
# Revert to original values
# The prior prompt:
if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) {
Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt
Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT
}
# The prior PYTHONHOME:
if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) {
Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME
Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME
}
# The prior PATH:
if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) {
Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH
Remove-Item -Path Env:_OLD_VIRTUAL_PATH
}
# Just remove the VIRTUAL_ENV altogether:
if (Test-Path -Path Env:VIRTUAL_ENV) {
Remove-Item -Path env:VIRTUAL_ENV
}
# Just remove VIRTUAL_ENV_PROMPT altogether.
if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) {
Remove-Item -Path env:VIRTUAL_ENV_PROMPT
}
# Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether:
if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) {
Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force
}
# Leave deactivate function in the global namespace if requested:
if (-not $NonDestructive) {
Remove-Item -Path function:deactivate
}
}
<#
.Description
Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the
given folder, and returns them in a map.
For each line in the pyvenv.cfg file, if that line can be parsed into exactly
two strings separated by `=` (with any amount of whitespace surrounding the =)
then it is considered a `key = value` line. The left hand string is the key,
the right hand is the value.
If the value starts with a `'` or a `"` then the first and last character is
stripped from the value before being captured.
.Parameter ConfigDir
Path to the directory that contains the `pyvenv.cfg` file.
#>
function Get-PyVenvConfig(
[String]
$ConfigDir
) {
Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg"
# Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue).
$pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue
# An empty map will be returned if no config file is found.
$pyvenvConfig = @{ }
if ($pyvenvConfigPath) {
Write-Verbose "File exists, parse `key = value` lines"
$pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath
$pyvenvConfigContent | ForEach-Object {
$keyval = $PSItem -split "\s*=\s*", 2
if ($keyval[0] -and $keyval[1]) {
$val = $keyval[1]
# Remove extraneous quotations around a string value.
if ("'""".Contains($val.Substring(0, 1))) {
$val = $val.Substring(1, $val.Length - 2)
}
$pyvenvConfig[$keyval[0]] = $val
Write-Verbose "Adding Key: '$($keyval[0])'='$val'"
}
}
}
return $pyvenvConfig
}
<# Begin Activate script --------------------------------------------------- #>
# Determine the containing directory of this script
$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition
$VenvExecDir = Get-Item -Path $VenvExecPath
Write-Verbose "Activation script is located in path: '$VenvExecPath'"
Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)"
Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)"
# Set values required in priority: CmdLine, ConfigFile, Default
# First, get the location of the virtual environment, it might not be
# VenvExecDir if specified on the command line.
if ($VenvDir) {
Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values"
}
else {
Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir."
$VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/")
Write-Verbose "VenvDir=$VenvDir"
}
# Next, read the `pyvenv.cfg` file to determine any required value such
# as `prompt`.
$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir
# Next, set the prompt from the command line, or the config file, or
# just use the name of the virtual environment folder.
if ($Prompt) {
Write-Verbose "Prompt specified as argument, using '$Prompt'"
}
else {
Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value"
if ($pyvenvCfg -and $pyvenvCfg['prompt']) {
Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'"
$Prompt = $pyvenvCfg['prompt'];
}
else {
Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virtual environment)"
Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'"
$Prompt = Split-Path -Path $venvDir -Leaf
}
}
Write-Verbose "Prompt = '$Prompt'"
Write-Verbose "VenvDir='$VenvDir'"
# Deactivate any currently active virtual environment, but leave the
# deactivate function in place.
deactivate -nondestructive
# Now set the environment variable VIRTUAL_ENV, used by many tools to determine
# that there is an activated venv.
$env:VIRTUAL_ENV = $VenvDir
if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) {
Write-Verbose "Setting prompt to '$Prompt'"
# Set the prompt to include the env name
# Make sure _OLD_VIRTUAL_PROMPT is global
function global:_OLD_VIRTUAL_PROMPT { "" }
Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT
New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt
function global:prompt {
Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) "
_OLD_VIRTUAL_PROMPT
}
$env:VIRTUAL_ENV_PROMPT = $Prompt
}
# Clear PYTHONHOME
if (Test-Path -Path Env:PYTHONHOME) {
Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME
Remove-Item -Path Env:PYTHONHOME
}
# Add the venv to the PATH
Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH
$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH"

69
venv/bin/activate Executable file
View File

@@ -0,0 +1,69 @@
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
# reset old environment variables
if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
PATH="${_OLD_VIRTUAL_PATH:-}"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi
if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
PS1="${_OLD_VIRTUAL_PS1:-}"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
unset VIRTUAL_ENV_PROMPT
if [ ! "${1:-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV=/home/kitraining/coco-tool/Abschluss-Projekt/venv
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/"bin":$PATH"
export PATH
# unset PYTHONHOME if set
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
# could use `if (set -u; : $PYTHONHOME) ;` in bash
if [ -n "${PYTHONHOME:-}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
_OLD_VIRTUAL_PS1="${PS1:-}"
PS1='(venv) '"${PS1:-}"
export PS1
VIRTUAL_ENV_PROMPT='(venv) '
export VIRTUAL_ENV_PROMPT
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi

26
venv/bin/activate.csh Executable file
View File

@@ -0,0 +1,26 @@
# This file must be used with "source bin/activate.csh" *from csh*.
# You cannot run it directly.
# Created by Davide Di Blasi <davidedb@gmail.com>.
# Ported to Python 3.3 venv by Andrew Svetlov <andrew.svetlov@gmail.com>
alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; unsetenv VIRTUAL_ENV_PROMPT; test "\!:*" != "nondestructive" && unalias deactivate'
# Unset irrelevant variables.
deactivate nondestructive
setenv VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
set _OLD_VIRTUAL_PATH="$PATH"
setenv PATH "$VIRTUAL_ENV/"bin":$PATH"
set _OLD_VIRTUAL_PROMPT="$prompt"
if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then
set prompt = '(venv) '"$prompt"
setenv VIRTUAL_ENV_PROMPT '(venv) '
endif
alias pydoc python -m pydoc
rehash

69
venv/bin/activate.fish Executable file
View File

@@ -0,0 +1,69 @@
# This file must be used with "source <venv>/bin/activate.fish" *from fish*
# (https://fishshell.com/); you cannot run it directly.
function deactivate -d "Exit virtual environment and return to normal shell environment"
# reset old environment variables
if test -n "$_OLD_VIRTUAL_PATH"
set -gx PATH $_OLD_VIRTUAL_PATH
set -e _OLD_VIRTUAL_PATH
end
if test -n "$_OLD_VIRTUAL_PYTHONHOME"
set -gx PYTHONHOME $_OLD_VIRTUAL_PYTHONHOME
set -e _OLD_VIRTUAL_PYTHONHOME
end
if test -n "$_OLD_FISH_PROMPT_OVERRIDE"
set -e _OLD_FISH_PROMPT_OVERRIDE
# prevents error when using nested fish instances (Issue #93858)
if functions -q _old_fish_prompt
functions -e fish_prompt
functions -c _old_fish_prompt fish_prompt
functions -e _old_fish_prompt
end
end
set -e VIRTUAL_ENV
set -e VIRTUAL_ENV_PROMPT
if test "$argv[1]" != "nondestructive"
# Self-destruct!
functions -e deactivate
end
end
# Unset irrelevant variables.
deactivate nondestructive
set -gx VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
set -gx _OLD_VIRTUAL_PATH $PATH
set -gx PATH "$VIRTUAL_ENV/"bin $PATH
# Unset PYTHONHOME if set.
if set -q PYTHONHOME
set -gx _OLD_VIRTUAL_PYTHONHOME $PYTHONHOME
set -e PYTHONHOME
end
if test -z "$VIRTUAL_ENV_DISABLE_PROMPT"
# fish uses a function instead of an env var to generate the prompt.
# Save the current fish_prompt function as the function _old_fish_prompt.
functions -c fish_prompt _old_fish_prompt
# With the original prompt function renamed, we can override with our own.
function fish_prompt
# Save the return status of the last command.
set -l old_status $status
# Output the venv prompt; color taken from the blue of the Python logo.
printf "%s%s%s" (set_color 4B8BBE) '(venv) ' (set_color normal)
# Restore the return status of the previous command.
echo "exit $old_status" | .
# Output the original/"old" prompt.
_old_fish_prompt
end
set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV"
set -gx VIRTUAL_ENV_PROMPT '(venv) '
end

8
venv/bin/dotenv Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from dotenv.__main__ import cli
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(cli())

8
venv/bin/flask Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from flask.cli import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/normalizer Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from charset_normalizer.cli import cli_detect
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(cli_detect())

8
venv/bin/pip Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/pip3 Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/pip3.11 Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

1
venv/bin/python Symbolic link
View File

@@ -0,0 +1 @@
python3

Some files were not shown because too many files have changed in this diff Show More