philipp/Abschluss-Projekt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

training fix. add global settings

Browse Source
This commit is contained in:
Crussade
2025-12-02 09:31:52 +01:00
parent 55b1b2b5fe
commit c3c7e042bb
86 changed files with 77512 additions and 7054 deletions
Download Patch File Download Diff File Expand all files Collapse all files

434
TRANSFER_LEARNING_FEATURE.md
View File

@@ -1,217 +1,217 @@
# Transfer Learning Base Configuration Feature # Transfer Learning Base Configuration Feature
## Overview ## Overview
This feature implements automatic loading of base configurations when "Train on COCO" transfer learning is selected. Base parameters are loaded from `backend/data/` based on the selected YOLOX model, and these protected fields are displayed as greyed out and non-editable in the frontend. This feature implements automatic loading of base configurations when "Train on COCO" transfer learning is selected. Base parameters are loaded from `backend/data/` based on the selected YOLOX model, and these protected fields are displayed as greyed out and non-editable in the frontend.
## Components Modified/Created ## Components Modified/Created
### Backend ### Backend
#### 1. Base Configuration Files (`backend/data/`) #### 1. Base Configuration Files (`backend/data/`)
- **`yolox_s.py`** - Base config for YOLOX-Small (depth=0.33, width=0.50) - **`yolox_s.py`** - Base config for YOLOX-Small (depth=0.33, width=0.50)
- **`yolox_m.py`** - Base config for YOLOX-Medium (depth=0.67, width=0.75) - **`yolox_m.py`** - Base config for YOLOX-Medium (depth=0.67, width=0.75)
- **`yolox_l.py`** - Base config for YOLOX-Large (depth=1.0, width=1.0) - **`yolox_l.py`** - Base config for YOLOX-Large (depth=1.0, width=1.0)
- **`yolox_x.py`** - Base config for YOLOX-XLarge (depth=1.33, width=1.25) - **`yolox_x.py`** - Base config for YOLOX-XLarge (depth=1.33, width=1.25)
Each file contains a `BaseExp` class with protected parameters: Each file contains a `BaseExp` class with protected parameters:
- Model architecture (depth, width, activation) - Model architecture (depth, width, activation)
- Training hyperparameters (max_epoch, warmup_epochs, scheduler, etc.) - Training hyperparameters (max_epoch, warmup_epochs, scheduler, etc.)
- Optimizer settings (momentum, weight_decay) - Optimizer settings (momentum, weight_decay)
- Augmentation probabilities (mosaic_prob, mixup_prob, etc.) - Augmentation probabilities (mosaic_prob, mixup_prob, etc.)
- Input/output sizes - Input/output sizes
#### 2. Services (`backend/services/generate_yolox_exp.py`) #### 2. Services (`backend/services/generate_yolox_exp.py`)
**New functions:** **New functions:**
- `load_base_config(selected_model)` - Dynamically loads base config using importlib - `load_base_config(selected_model)` - Dynamically loads base config using importlib
- Modified `generate_yolox_inference_exp()` to support `use_base_config` parameter - Modified `generate_yolox_inference_exp()` to support `use_base_config` parameter
- Base config merging logic: base → user overrides → defaults - Base config merging logic: base → user overrides → defaults
**Behavior:** **Behavior:**
- `transfer_learning='coco'` → loads base config + applies user overrides - `transfer_learning='coco'` → loads base config + applies user overrides
- `transfer_learning='sketch'` → uses only user-defined values - `transfer_learning='sketch'` → uses only user-defined values
- Protected parameters from base config are preserved unless explicitly overridden - Protected parameters from base config are preserved unless explicitly overridden
#### 3. API Routes (`backend/routes/api.py`) #### 3. API Routes (`backend/routes/api.py`)
**New endpoint:** **New endpoint:**
```python ```python
@api_bp.route('/base-config/<model_name>', methods=['GET']) @api_bp.route('/base-config/<model_name>', methods=['GET'])
def get_base_config(model_name): def get_base_config(model_name):
``` ```
Returns the base configuration JSON for a specific YOLOX model. Returns the base configuration JSON for a specific YOLOX model.
### Frontend ### Frontend
#### 1. HTML (`edit-training.html`) #### 1. HTML (`edit-training.html`)
**Added:** **Added:**
- Info banner to indicate when base config is active - Info banner to indicate when base config is active
- CSS styles for disabled input fields (grey background, not-allowed cursor) - CSS styles for disabled input fields (grey background, not-allowed cursor)
- Visual feedback showing which model's base config is loaded - Visual feedback showing which model's base config is loaded
**Banner HTML:** **Banner HTML:**
```html ```html
<div id="base-config-info" style="display:none; ..."> <div id="base-config-info" style="display:none; ...">
🔒 Base Configuration Active 🔒 Base Configuration Active
Protected parameters are loaded from [model] base config Protected parameters are loaded from [model] base config
</div> </div>
``` ```
**CSS for disabled fields:** **CSS for disabled fields:**
```css ```css
.setting-row input[type="number"]:disabled, .setting-row input[type="number"]:disabled,
.setting-row input[type="text"]:disabled, .setting-row input[type="text"]:disabled,
.setting-row input[type="checkbox"]:disabled { .setting-row input[type="checkbox"]:disabled {
background: #d3d3d3 !important; background: #d3d3d3 !important;
color: #666 !important; color: #666 !important;
cursor: not-allowed !important; cursor: not-allowed !important;
border: 1px solid #999 !important; border: 1px solid #999 !important;
} }
``` ```
#### 2. JavaScript (`js/start-training.js`) #### 2. JavaScript (`js/start-training.js`)
**New functionality:** **New functionality:**
1. **Base Config Loading:** 1. **Base Config Loading:**
```javascript ```javascript
function loadBaseConfig(modelName) function loadBaseConfig(modelName)
``` ```
Fetches base config from `/api/base-config/<model>` Fetches base config from `/api/base-config/<model>`
2. **Apply Base Config:** 2. **Apply Base Config:**
```javascript ```javascript
function applyBaseConfig(config, isCocoMode) function applyBaseConfig(config, isCocoMode)
``` ```
- Applies config values to form fields - Applies config values to form fields
- Disables and greys out protected fields - Disables and greys out protected fields
- Shows/hides info banner - Shows/hides info banner
- Adds tooltips to disabled fields - Adds tooltips to disabled fields
3. **Update Transfer Learning Mode:** 3. **Update Transfer Learning Mode:**
```javascript ```javascript
function updateTransferLearningMode() function updateTransferLearningMode()
``` ```
- Monitors changes to "Transfer Learning" dropdown - Monitors changes to "Transfer Learning" dropdown
- Monitors changes to "Select Model" dropdown - Monitors changes to "Select Model" dropdown
- Loads appropriate base config when COCO mode is selected - Loads appropriate base config when COCO mode is selected
- Clears base config when sketch mode is selected - Clears base config when sketch mode is selected
4. **Form Submission Enhancement:** 4. **Form Submission Enhancement:**
- Temporarily enables disabled fields before submission - Temporarily enables disabled fields before submission
- Ensures protected parameters are included in form data - Ensures protected parameters are included in form data
- Re-disables fields after collection - Re-disables fields after collection
**Protected Fields List:** **Protected Fields List:**
```javascript ```javascript
const protectedFields = [ const protectedFields = [
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr', 'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay', 'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup', 'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees', 'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval' 'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
]; ];
``` ```
## User Flow ## User Flow
### 1. Normal Custom Training (Train from sketch) ### 1. Normal Custom Training (Train from sketch)
- User selects model: e.g., "YOLOX-s" - User selects model: e.g., "YOLOX-s"
- User selects "Train from sketch" - User selects "Train from sketch"
- All fields are editable (white background) - All fields are editable (white background)
- User can customize all parameters - User can customize all parameters
- Submission uses user-defined values only - Submission uses user-defined values only
### 2. COCO Transfer Learning (Train on COCO) ### 2. COCO Transfer Learning (Train on COCO)
- User selects model: e.g., "YOLOX-s" - User selects model: e.g., "YOLOX-s"
- User selects "Train on coco" - User selects "Train on coco"
- **Automatic actions:** - **Automatic actions:**
1. Frontend calls `/api/base-config/YOLOX-s` 1. Frontend calls `/api/base-config/YOLOX-s`
2. Base config is loaded and applied 2. Base config is loaded and applied
3. Protected fields become greyed out and disabled 3. Protected fields become greyed out and disabled
4. Green info banner appears: "🔒 Base Configuration Active" 4. Green info banner appears: "🔒 Base Configuration Active"
5. Tooltip on hover: "Protected by base config for YOLOX-s. Switch to 'Train from sketch' to customize." 5. Tooltip on hover: "Protected by base config for YOLOX-s. Switch to 'Train from sketch' to customize."
- User can still edit non-protected fields - User can still edit non-protected fields
- On submit: both base config values AND user overrides are sent to backend - On submit: both base config values AND user overrides are sent to backend
- Backend generates exp.py with merged settings - Backend generates exp.py with merged settings
### 3. Switching Models ### 3. Switching Models
- User changes from "YOLOX-s" to "YOLOX-l" (while in COCO mode) - User changes from "YOLOX-s" to "YOLOX-l" (while in COCO mode)
- Frontend automatically: - Frontend automatically:
1. Fetches new base config for YOLOX-l 1. Fetches new base config for YOLOX-l
2. Updates field values (depth=1.0, width=1.0, etc.) 2. Updates field values (depth=1.0, width=1.0, etc.)
3. Updates banner text to show "YOLOX-l" 3. Updates banner text to show "YOLOX-l"
- Protected parameters update to match new model's architecture - Protected parameters update to match new model's architecture
## Testing ## Testing
### Manual Test Steps: ### Manual Test Steps:
1. **Test Base Config Loading:** 1. **Test Base Config Loading:**
```bash ```bash
cd backend/data cd backend/data
python test_base_configs.py python test_base_configs.py
``` ```
Should display all parameters for yolox-s, yolox-m, yolox-l, yolox-x Should display all parameters for yolox-s, yolox-m, yolox-l, yolox-x
2. **Test API Endpoint:** 2. **Test API Endpoint:**
```bash ```bash
# Start Flask server # Start Flask server
cd backend cd backend
python app.py python app.py
# In another terminal: # In another terminal:
curl http://localhost:3000/api/base-config/YOLOX-s curl http://localhost:3000/api/base-config/YOLOX-s
``` ```
Should return JSON with depth, width, activation, etc. Should return JSON with depth, width, activation, etc.
3. **Test Frontend:** 3. **Test Frontend:**
- Open `edit-training.html?id=<project_id>` in browser - Open `edit-training.html?id=<project_id>` in browser
- Select "YOLOX-s" model - Select "YOLOX-s" model
- Select "Train on coco" → fields should grey out - Select "Train on coco" → fields should grey out
- Select "Train from sketch" → fields should become editable - Select "Train from sketch" → fields should become editable
- Switch to "YOLOX-l" (in COCO mode) → values should update - Switch to "YOLOX-l" (in COCO mode) → values should update
- Open browser console and check for: `Applied base config. Protected fields: depth, width, ...` - Open browser console and check for: `Applied base config. Protected fields: depth, width, ...`
4. **Test Form Submission:** 4. **Test Form Submission:**
- With COCO mode active (fields greyed out) - With COCO mode active (fields greyed out)
- Click "Save Parameters" - Click "Save Parameters"
- Check browser Network tab → POST to `/api/yolox-settings` - Check browser Network tab → POST to `/api/yolox-settings`
- Verify payload includes protected parameters (depth, width, etc.) - Verify payload includes protected parameters (depth, width, etc.)
- Check Flask logs for successful save - Check Flask logs for successful save
### Expected Behaviors: ### Expected Behaviors:
✅ **COCO mode + YOLOX-s:** ✅ **COCO mode + YOLOX-s:**
- depth: 0.33 (greyed out) - depth: 0.33 (greyed out)
- width: 0.50 (greyed out) - width: 0.50 (greyed out)
- activation: silu (greyed out) - activation: silu (greyed out)
- Info banner visible - Info banner visible
✅ **COCO mode + YOLOX-l:** ✅ **COCO mode + YOLOX-l:**
- depth: 1.0 (greyed out) - depth: 1.0 (greyed out)
- width: 1.0 (greyed out) - width: 1.0 (greyed out)
- activation: silu (greyed out) - activation: silu (greyed out)
✅ **Sketch mode:** ✅ **Sketch mode:**
- All fields white/editable - All fields white/editable
- No info banner - No info banner
- User can set any values - User can set any values
## Documentation ## Documentation
- **`backend/data/README.md`** - Complete guide on base config system - **`backend/data/README.md`** - Complete guide on base config system
- **`backend/data/test_base_configs.py`** - Test script for base configs - **`backend/data/test_base_configs.py`** - Test script for base configs
## Benefits ## Benefits
1. **Proven defaults:** Users start with battle-tested COCO pretraining settings 1. **Proven defaults:** Users start with battle-tested COCO pretraining settings
2. **Prevents mistakes:** Can't accidentally break model architecture by changing depth/width 2. **Prevents mistakes:** Can't accidentally break model architecture by changing depth/width
3. **Easy customization:** Can still override specific parameters if needed 3. **Easy customization:** Can still override specific parameters if needed
4. **Visual feedback:** Clear indication of which fields are protected 4. **Visual feedback:** Clear indication of which fields are protected
5. **Model-specific:** Each model (s/m/l/x) has appropriate architecture defaults 5. **Model-specific:** Each model (s/m/l/x) has appropriate architecture defaults
6. **Flexible:** Can easily add new models by creating new base config files 6. **Flexible:** Can easily add new models by creating new base config files
## Future Enhancements ## Future Enhancements
- Add "Override" button next to protected fields to unlock individual parameters - Add "Override" button next to protected fields to unlock individual parameters
- Show diff comparison between base config and user overrides - Show diff comparison between base config and user overrides
- Add validation warnings if user tries values far from base config ranges - Add validation warnings if user tries values far from base config ranges
- Export final merged config as preview before training - Export final merged config as preview before training

414
add-project.html
View File

@@ -1,175 +1,241 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body> <body>
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon> style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" class="button-red">Add Training Project</button> <button id="Add Training Project" class="button-red">Add Training Project</button>
<button id="Add Dataset" class="button">Add Dataset</button> <button id="Add Dataset" class="button">Add Dataset</button>
<button id="Import Dataset" class="button">Refresh Label-Studio</button> <button id="Import Dataset" class="button">Refresh Label-Studio</button>
<button id="seed-db-btn" class="button">Seed Database</button> <button id="seed-db-btn" class="button">Seed Database</button>
</div> <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div> </div>
</div>
<div class="popup">
<div class="upload-button" onclick="uploadButtonHandler()"> <div class="popup">
<span class="upload-button-text">Upload</span> <div class="upload-button" onclick="uploadButtonHandler()">
</div> <span class="upload-button-text">Upload</span>
<div class="image"></div> </div>
<div class="image"></div>
<div class="div">
<div class="div">
<div class="add-category">
<div class="add-category">
<input class="div-wrapper" placeholder="Class name"></input>
<button class="upload-button-text-wrapper"> <input class="div-wrapper" placeholder="Class name"></input>
<span class="button-text-upload">Add Class</span> <button class="upload-button-text-wrapper">
</button> <span class="button-text-upload">Add Class</span>
</div> </button>
<input class="project-name" placeholder="Project Name" id="project_name_input"></input> </div>
<textarea class="add-description" placeholder="Description" id="project_description_input"></textarea> <input class="project-name" placeholder="Project Name" id="project_name_input"></input>
<div class="add-class-wrapper"> <textarea class="add-description" placeholder="Description" id="project_description_input"></textarea>
<script type="module"> <div class="add-class-wrapper">
import { addClass } from './js/add-class.js'; <script type="module">
addClass(); import { addClass } from './js/add-class.js';
</script> addClass();
</div> </script>
</div> </div>
<button class="confirm-button-datasetcreation"> </div>
<span class="button-text-upload">Confirm</span> <button class="confirm-button-datasetcreation">
</button> <span class="button-text-upload">Confirm</span>
</div> </button>
</div> </div>
</div> </div>
</div>
</div>
<script type="text/javascript" src="./js/add-image.js"></script> </div>
<script type="module"> <script type="text/javascript" src="./js/add-image.js"></script>
import { addClass } from './js/add-class.js'; <script type="module">
// Grab the inputs and the button import { addClass } from './js/add-class.js';
const projectNameInput = document.querySelector('.project-name'); // Grab the inputs and the button
const descriptionInput = document.querySelector('.add-description'); const projectNameInput = document.querySelector('.project-name');
const confirmButton = document.querySelector('.confirm-button-datasetcreation'); const descriptionInput = document.querySelector('.add-description');
const classWrapper = document.querySelector('.add-class-wrapper'); const confirmButton = document.querySelector('.confirm-button-datasetcreation');
const classWrapper = document.querySelector('.add-class-wrapper');
const addClassButton = document.querySelector('.upload-button-text-wrapper');
const addProjectButton = document.querySelector('.confirm-button-datasetcreation') const addClassButton = document.querySelector('.upload-button-text-wrapper');
const addProjectButton = document.querySelector('.confirm-button-datasetcreation')
addClassButton.addEventListener('click', () => {
const event = new Event('classListUpdated'); addClassButton.addEventListener('click', () => {
document.dispatchEvent(event); const event = new Event('classListUpdated');
}); document.dispatchEvent(event);
});
// Function to update button state
function updateButtonState() { // Function to update button state
const projectName = projectNameInput.value.trim(); function updateButtonState() {
const description = descriptionInput.value.trim(); const projectName = projectNameInput.value.trim();
const existingClasses = classWrapper.querySelectorAll('.overlap-group'); const description = descriptionInput.value.trim();
const existingClasses = classWrapper.querySelectorAll('.overlap-group');
// Disable button if either field is empty
if (projectName === '' || description === '' || existingClasses.length === 0) { // Disable button if either field is empty
confirmButton.disabled = true; if (projectName === '' || description === '' || existingClasses.length === 0) {
confirmButton.style.cursor = 'not-allowed'; confirmButton.disabled = true;
} else { confirmButton.style.cursor = 'not-allowed';
confirmButton.disabled = false; } else {
confirmButton.style.cursor = 'pointer'; confirmButton.disabled = false;
} confirmButton.style.cursor = 'pointer';
} }
}
// Initial check on page load
updateButtonState(); // Initial check on page load
updateButtonState();
projectNameInput.addEventListener('input', updateButtonState);
descriptionInput.addEventListener('input', updateButtonState); projectNameInput.addEventListener('input', updateButtonState);
document.addEventListener('classListUpdated', updateButtonState); descriptionInput.addEventListener('input', updateButtonState);
</script> document.addEventListener('classListUpdated', updateButtonState);
</script>
<script>
document.getElementById('seed-db-btn').addEventListener('click', function () { <script>
fetch('/api/seed') document.getElementById('seed-db-btn').addEventListener('click', function () {
}); fetch('/api/seed')
</script> });
<script> </script>
document.getElementById('seed-db-btn').addEventListener('click', function () { <script>
const elLoader = document.getElementById("loader") document.getElementById('seed-db-btn').addEventListener('click', function () {
elLoader.style.display = "inherit" const elLoader = document.getElementById("loader")
elLoader.style.display = "inherit"
fetch('/api/seed')
.finally(() => { fetch('/api/seed')
// Instead of hiding loader immediately, poll /api/update-status until done .finally(() => {
function pollStatus() { // Instead of hiding loader immediately, poll /api/update-status until done
fetch('/api/update-status') function pollStatus() {
.then(res => res.json()) fetch('/api/update-status')
.then(status => { .then(res => res.json())
if (status && status.running) { .then(status => {
// Still running, poll again after short delay if (status && status.running) {
setTimeout(pollStatus, 5000); // Still running, poll again after short delay
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
pollStatus(); }
}) pollStatus();
}); })
});
// Show loader if backend is still processing on page load
// Show loader if backend is still processing on page load
function pollStatus() {
const elLoader = document.getElementById("loader"); function pollStatus() {
fetch('/api/update-status') const elLoader = document.getElementById("loader");
.then(res => res.json()) fetch('/api/update-status')
.then(res => res.json())
.then(status => {
if (status && status.running) { .then(status => {
elLoader.style.display = "inherit"; if (status && status.running) {
setTimeout(pollStatus, 5000); elLoader.style.display = "inherit";
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
}
</script>
</script>
</body> <!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html> </html>

80
backend/.gitignore vendored
View File

@@ -1,40 +1,40 @@
# Python # Python
__pycache__/ __pycache__/
*.py[cod] *.py[cod]
*$py.class *$py.class
*.so *.so
.Python .Python
venv/ venv/
env/ env/
ENV/ ENV/
*.egg-info/ *.egg-info/
dist/ dist/
build/ build/
# Flask # Flask
instance/ instance/
.webassets-cache .webassets-cache
# Database # Database
*.db *.db
*.sqlite *.sqlite
# Environment # Environment
.env .env
.flaskenv .flaskenv
# IDE # IDE
.vscode/ .vscode/
.idea/ .idea/
*.swp *.swp
*.swo *.swo
# OS # OS
.DS_Store .DS_Store
Thumbs.db Thumbs.db
# Logs # Logs
*.log *.log
# Uploads # Uploads
uploads/*.pth uploads/*.pth

28
backend/0815/27/exp_infer.py
View File

@@ -1,28 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_27_train.json"
self.val_ann = "coco_project_27_valid.json"
self.test_ann = "coco_project_27_test.json"
self.num_classes = 80
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.random_size = (10, 20)
self.test_size = (640.0, 640.0)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

48
backend/1/6/exp.py Normal file
View File

@@ -0,0 +1,48 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_2_train.json"
self.val_ann = "coco_project_2_valid.json"
self.test_ann = "coco_project_2_test.json"
self.num_classes = 2
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
self.activation = "silu"
self.depth = 0.33
self.scheduler = "yoloxwarmcos"
self.width = 0.5
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.random_size = (10, 20)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

48
backend/1/6/exp_infer.py Normal file
View File

@@ -0,0 +1,48 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_6_train.json"
self.val_ann = "coco_project_6_valid.json"
self.test_ann = "coco_project_6_test.json"
self.num_classes = 2
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
self.depth = 0.33
self.width = 0.5
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

148
backend/CONVERSION_SUMMARY.md
View File

@@ -1,148 +0,0 @@
# Backend Conversion Summary
## ✅ Conversion Complete
Your Node.js backend has been successfully converted to Python using Flask and SQLAlchemy.
## 📁 New Python Files Created
### Core Application
- **app.py** - Main Flask application (replaces server.js)
- **start.py** - Startup script for easy launching
- **requirements.txt** - Python dependencies (replaces package.json)
### Database Layer
- **database/database.py** - SQLAlchemy database configuration (replaces database.js)
### Models (Sequelize → SQLAlchemy)
- **models/TrainingProject.py**
- **models/TrainingProjectDetails.py**
- **models/training.py**
- **models/LabelStudioProject.py**
- **models/Images.py**
- **models/Annotation.py**
- **models/__init__.py**
### API Routes
- **routes/api.py** - All API endpoints converted to Flask blueprints (replaces api.js)
- **routes/__init__.py**
### Services
- **services/fetch_labelstudio.py** - Label Studio API integration
- **services/seed_label_studio.py** - Database seeding logic
- **services/generate_json_yolox.py** - COCO JSON generation
- **services/generate_yolox_exp.py** - YOLOX experiment file generation
- **services/push_yolox_exp.py** - Save training settings to DB
- **services/__init__.py**
### Documentation
- **README.md** - Comprehensive documentation
- **QUICKSTART.md** - Quick setup guide
- **.gitignore** - Python-specific ignore patterns
## 🔄 Key Changes
### Technology Stack
| Component | Node.js | Python |
|-----------|---------|--------|
| Framework | Express.js | Flask |
| ORM | Sequelize | SQLAlchemy |
| HTTP Client | node-fetch | requests |
| Package Manager | npm | pip |
| Runtime | Node.js | Python 3.8+ |
### API Compatibility
✅ All endpoints preserved with same URLs
✅ Request/response formats maintained
✅ Same database schema
✅ Same business logic
### Converted Features
- ✅ Training project management
- ✅ Label Studio integration
- ✅ YOLOX configuration and training
- ✅ File upload handling
- ✅ Image and annotation management
- ✅ COCO JSON generation
- ✅ Training logs
## 🚀 Getting Started
1. **Install dependencies:**
```bash
cd backend
python -m venv venv
.\venv\Scripts\Activate.ps1 # Windows
pip install -r requirements.txt
```
2. **Run the server:**
```bash
python start.py
```
3. **Server runs at:** `http://0.0.0.0:3000`
## 📦 Dependencies Installed
- Flask 3.0.0 - Web framework
- Flask-CORS 4.0.0 - Cross-origin resource sharing
- Flask-SQLAlchemy 3.1.1 - ORM integration
- SQLAlchemy 2.0.23 - Database ORM
- PyMySQL 1.1.0 - MySQL driver
- requests 2.31.0 - HTTP client
- Pillow 10.1.0 - Image processing
## ⚠️ Important Notes
1. **Virtual Environment**: Always activate the virtual environment before running
2. **Database**: MySQL must be running with the `myapp` database created
3. **Credentials**: Update database credentials in `app.py` if needed
4. **Python Version**: Requires Python 3.8 or higher
## 🧪 Testing
Test the conversion:
```bash
# Get all training projects
curl http://localhost:3000/api/training-projects
# Get Label Studio projects
curl http://localhost:3000/api/label-studio-projects
```
## 📝 Original Files
Your original Node.js files remain untouched:
- server.js
- package.json
- routes/api.js
- models/*.js (JavaScript)
- services/*.js (JavaScript)
You can keep them as backup or remove them once you verify the Python version works correctly.
## 🔍 What to Verify
1. ✅ Database connection works
2. ✅ All API endpoints respond correctly
3. ✅ File uploads work
4. ✅ Label Studio integration works
5. ✅ YOLOX training can be triggered
6. ✅ COCO JSON generation works
## 🐛 Troubleshooting
See **QUICKSTART.md** for common issues and solutions.
## 📚 Further Documentation
- **README.md** - Complete project documentation
- **QUICKSTART.md** - Setup guide
- **API Documentation** - All endpoints documented in README.md
---
**Conversion completed successfully!** 🎉
Your backend is now running on Python with Flask and SQLAlchemy.

226
backend/QUICKSTART.md
View File

@@ -1,113 +1,113 @@
# Quick Start Guide - Python Backend # Quick Start Guide - Python Backend
## Step-by-Step Setup ## Step-by-Step Setup
### 1. Install Python ### 1. Install Python
Make sure you have Python 3.8 or higher installed: Make sure you have Python 3.8 or higher installed:
```bash ```bash
python --version python --version
``` ```
### 2. Create Virtual Environment ### 2. Create Virtual Environment
```bash ```bash
cd backend cd backend
python -m venv venv python -m venv venv
``` ```
### 3. Activate Virtual Environment ### 3. Activate Virtual Environment
**Windows:** **Windows:**
```powershell ```powershell
.\venv\Scripts\Activate.ps1 .\venv\Scripts\Activate.ps1
``` ```
**Linux/Mac:** **Linux/Mac:**
```bash ```bash
source venv/bin/activate source venv/bin/activate
``` ```
### 4. Install Dependencies ### 4. Install Dependencies
```bash ```bash
pip install -r requirements.txt pip install -r requirements.txt
``` ```
### 5. Verify Database Connection ### 5. Verify Database Connection
Make sure MySQL is running and the database `myapp` exists: Make sure MySQL is running and the database `myapp` exists:
```sql ```sql
CREATE DATABASE IF NOT EXISTS myapp; CREATE DATABASE IF NOT EXISTS myapp;
``` ```
### 6. Run the Server ### 6. Run the Server
```bash ```bash
python start.py python start.py
``` ```
Or: Or:
```bash ```bash
python app.py python app.py
``` ```
The server should now be running at `http://0.0.0.0:3000` The server should now be running at `http://0.0.0.0:3000`
## Testing the API ## Testing the API
Test if the server is working: Test if the server is working:
```bash ```bash
curl http://localhost:3000/api/training-projects curl http://localhost:3000/api/training-projects
``` ```
## Common Issues ## Common Issues
### ModuleNotFoundError ### ModuleNotFoundError
If you get import errors, make sure you've activated the virtual environment and installed all dependencies. If you get import errors, make sure you've activated the virtual environment and installed all dependencies.
### Database Connection Error ### Database Connection Error
Check that: Check that:
- MySQL is running - MySQL is running
- Database credentials in `app.py` are correct - Database credentials in `app.py` are correct
- Database `myapp` exists - Database `myapp` exists
### Port Already in Use ### Port Already in Use
If port 3000 is already in use, modify the port in `app.py`: If port 3000 is already in use, modify the port in `app.py`:
```python ```python
app.run(host='0.0.0.0', port=3001, debug=True) app.run(host='0.0.0.0', port=3001, debug=True)
``` ```
## What Changed from Node.js ## What Changed from Node.js
1. **Server Framework**: Express.js → Flask 1. **Server Framework**: Express.js → Flask
2. **ORM**: Sequelize → SQLAlchemy 2. **ORM**: Sequelize → SQLAlchemy
3. **HTTP Client**: node-fetch → requests 3. **HTTP Client**: node-fetch → requests
4. **Package Manager**: npm → pip 4. **Package Manager**: npm → pip
5. **Dependencies**: package.json → requirements.txt 5. **Dependencies**: package.json → requirements.txt
6. **Startup**: `node server.js` → `python app.py` 6. **Startup**: `node server.js` → `python app.py`
## Next Steps ## Next Steps
1. Test all API endpoints 1. Test all API endpoints
2. Update frontend to point to the new Python backend (if needed) 2. Update frontend to point to the new Python backend (if needed)
3. Migrate any remaining Node.js-specific logic 3. Migrate any remaining Node.js-specific logic
4. Test file uploads and downloads 4. Test file uploads and downloads
5. Test YOLOX training functionality 5. Test YOLOX training functionality
## File Structure Comparison ## File Structure Comparison
**Before (Node.js):** **Before (Node.js):**
``` ```
backend/ backend/
├── server.js ├── server.js
├── package.json ├── package.json
├── routes/api.js ├── routes/api.js
├── models/*.js ├── models/*.js
└── services/*.js └── services/*.js
``` ```
**After (Python):** **After (Python):**
``` ```
backend/ backend/
├── app.py ├── app.py
├── requirements.txt ├── requirements.txt
├── routes/api.py ├── routes/api.py
├── models/*.py ├── models/*.py
└── services/*.py └── services/*.py
``` ```

214
backend/README.md
View File

@@ -1,107 +1,107 @@
# Python Backend for COCO Tool # Python Backend for COCO Tool
This is the converted Python backend using Flask and SQLAlchemy. This is the converted Python backend using Flask and SQLAlchemy.
## Setup ## Setup
1. Create a virtual environment (recommended): 1. Create a virtual environment (recommended):
```bash ```bash
python -m venv venv python -m venv venv
``` ```
2. Activate the virtual environment: 2. Activate the virtual environment:
- Windows: `venv\Scripts\activate` - Windows: `venv\Scripts\activate`
- Linux/Mac: `source venv/bin/activate` - Linux/Mac: `source venv/bin/activate`
3. Install dependencies: 3. Install dependencies:
```bash ```bash
pip install -r requirements.txt pip install -r requirements.txt
``` ```
## Running the Server ## Running the Server
### Option 1: Using start.py ### Option 1: Using start.py
```bash ```bash
python start.py python start.py
``` ```
### Option 2: Using Flask directly ### Option 2: Using Flask directly
```bash ```bash
python app.py python app.py
``` ```
### Option 3: Using Flask CLI ### Option 3: Using Flask CLI
```bash ```bash
flask --app app run --host=0.0.0.0 --port=3000 flask --app app run --host=0.0.0.0 --port=3000
``` ```
The server will start on `http://0.0.0.0:3000` The server will start on `http://0.0.0.0:3000`
## Database Configuration ## Database Configuration
The database configuration is in `database/database.py`. Default settings: The database configuration is in `database/database.py`. Default settings:
- Host: localhost - Host: localhost
- Database: myapp - Database: myapp
- User: root - User: root
- Password: root - Password: root
Modify `app.py` to change these settings. Modify `app.py` to change these settings.
## Project Structure ## Project Structure
``` ```
backend/ backend/
├── app.py # Main Flask application ├── app.py # Main Flask application
├── start.py # Startup script ├── start.py # Startup script
├── requirements.txt # Python dependencies ├── requirements.txt # Python dependencies
├── database/ ├── database/
│ └── database.py # Database configuration │ └── database.py # Database configuration
├── models/ # SQLAlchemy models ├── models/ # SQLAlchemy models
│ ├── __init__.py │ ├── __init__.py
│ ├── Annotation.py │ ├── Annotation.py
│ ├── Images.py │ ├── Images.py
│ ├── LabelStudioProject.py │ ├── LabelStudioProject.py
│ ├── training.py │ ├── training.py
│ ├── TrainingProject.py │ ├── TrainingProject.py
│ └── TrainingProjectDetails.py │ └── TrainingProjectDetails.py
├── routes/ ├── routes/
│ └── api.py # API endpoints │ └── api.py # API endpoints
└── services/ # Business logic └── services/ # Business logic
├── fetch_labelstudio.py ├── fetch_labelstudio.py
├── generate_json_yolox.py ├── generate_json_yolox.py
├── generate_yolox_exp.py ├── generate_yolox_exp.py
├── push_yolox_exp.py ├── push_yolox_exp.py
└── seed_label_studio.py └── seed_label_studio.py
``` ```
## API Endpoints ## API Endpoints
All endpoints are prefixed with `/api`: All endpoints are prefixed with `/api`:
- `GET /api/seed` - Seed database from Label Studio - `GET /api/seed` - Seed database from Label Studio
- `POST /api/generate-yolox-json` - Generate YOLOX training files - `POST /api/generate-yolox-json` - Generate YOLOX training files
- `POST /api/start-yolox-training` - Start YOLOX training - `POST /api/start-yolox-training` - Start YOLOX training
- `GET /api/training-log` - Get training logs - `GET /api/training-log` - Get training logs
- `GET/POST /api/training-projects` - Manage training projects - `GET/POST /api/training-projects` - Manage training projects
- `GET /api/label-studio-projects` - Get Label Studio projects - `GET /api/label-studio-projects` - Get Label Studio projects
- `GET/POST/PUT /api/training-project-details` - Manage project details - `GET/POST/PUT /api/training-project-details` - Manage project details
- `POST /api/yolox-settings` - Save YOLOX settings - `POST /api/yolox-settings` - Save YOLOX settings
- `GET/DELETE /api/trainings` - Manage trainings - `GET/DELETE /api/trainings` - Manage trainings
- `DELETE /api/training-projects/:id` - Delete training project - `DELETE /api/training-projects/:id` - Delete training project
## Migration Notes ## Migration Notes
This is a direct conversion from Node.js/Express to Python/Flask: This is a direct conversion from Node.js/Express to Python/Flask:
- Express → Flask - Express → Flask
- Sequelize ORM → SQLAlchemy ORM - Sequelize ORM → SQLAlchemy ORM
- node-fetch → requests library - node-fetch → requests library
- Async routes maintained where needed - Async routes maintained where needed
- All file paths and logic preserved from original - All file paths and logic preserved from original
## Differences from Node.js Version ## Differences from Node.js Version
1. Python uses async/await differently - some routes may need adjustments 1. Python uses async/await differently - some routes may need adjustments
2. File handling uses Python's built-in open() instead of fs module 2. File handling uses Python's built-in open() instead of fs module
3. Subprocess calls use Python's subprocess module 3. Subprocess calls use Python's subprocess module
4. JSON handling uses Python's json module 4. JSON handling uses Python's json module
5. Path operations use os.path instead of Node's path module 5. Path operations use os.path instead of Node's path module

91
backend/app.py
View File

@@ -1,43 +1,48 @@
from flask import Flask, send_from_directory from flask import Flask, send_from_directory
from flask_cors import CORS from flask_cors import CORS
import os import os
from database.database import db, init_db from database.database import db, init_db
app = Flask(__name__, static_folder='..', static_url_path='') app = Flask(__name__, static_folder='..', static_url_path='')
CORS(app) CORS(app)
# Configure database # Configure database
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp' app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp2'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
# Initialize database # Initialize database
db.init_app(app) db.init_app(app)
# Import and register blueprints # Import and register blueprints
from routes.api import api_bp from routes.api import api_bp
app.register_blueprint(api_bp, url_prefix='/api') app.register_blueprint(api_bp, url_prefix='/api')
# Serve static files (HTML, CSS, JS) # Serve static files (HTML, CSS, JS)
@app.route('/') @app.route('/')
def index(): def index():
return send_from_directory('..', 'index.html') return send_from_directory('..', 'index.html')
@app.route('/<path:path>') @app.route('/<path:path>')
def serve_static(path): def serve_static(path):
return send_from_directory('..', path) return send_from_directory('..', path)
# Initialize DB and start server # Initialize DB and start server
if __name__ == '__main__': if __name__ == '__main__':
with app.app_context(): with app.app_context():
try: try:
# Test database connection # Test database connection
db.engine.connect() db.engine.connect()
print('DB connection established.') print('DB connection established.')
# Create tables if they don't exist # Create tables if they don't exist
db.create_all() db.create_all()
# Start server # Initialize default settings
app.run(host='0.0.0.0', port=3000, debug=True) from services.settings_service import initialize_default_settings
except Exception as err: initialize_default_settings()
print(f'Failed to start: {err}') print('Settings initialized.')
# Start server
app.run(host='0.0.0.0', port=4000, debug=True)
except Exception as err:
print(f'Failed to start: {err}')

25
backend/asdf/5/exp_infer.py
View File

@@ -1,25 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_5_train.json"
self.val_ann = "coco_project_5_valid.json"
self.test_ann = "coco_project_5_test.json"
self.num_classes = 4
self.depth = 1.0
self.width = 1.0
self.input_size = (640, 640)
self.mosaic_scale = (0.1, 2)
self.random_size = (10, 20)
self.test_size = (640, 640)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

84
backend/backend/1/1/exp_infer.py Normal file
View File

@@ -0,0 +1,84 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate" # Where images are located
self.annotations_dir = "./backend/1/custom_exp_1" # Where annotation JSONs are located
self.train_ann = "coco_project_1_train.json"
self.val_ann = "coco_project_1_valid.json"
self.test_ann = "coco_project_1_test.json"
self.num_classes = 2
# Disable train2017 subdirectory - our images are directly in data_dir
self.name = ""
# Set data workers for training
self.data_num_workers = 8
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
def get_dataset(self, cache=False, cache_type="ram"):
"""Override to use name parameter for images directory"""
from yolox.data import COCODataset
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
# Use empty string for name since we have absolute paths in JSON
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.train_ann,
name="",
img_size=self.input_size,
preproc=self.preproc if hasattr(self, 'preproc') else None,
cache=cache,
cache_type=cache_type,
)
def get_eval_dataset(self, **kwargs):
"""Override eval dataset using name parameter"""
from yolox.data import COCODataset
testdev = kwargs.get("testdev", False)
legacy = kwargs.get("legacy", False)
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.val_ann if not testdev else self.test_ann,
name="",
img_size=self.test_size,
preproc=None, # No preprocessing for evaluation
)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

3634
backend/backend/1/custom_exp_1/annotations/coco_project_1_test.json Normal file
View File

File diff suppressed because it is too large Load Diff

57310
backend/backend/1/custom_exp_1/annotations/coco_project_1_train.json Normal file
View File

File diff suppressed because it is too large Load Diff

6874
backend/backend/1/custom_exp_1/annotations/coco_project_1_valid.json Normal file
View File

File diff suppressed because it is too large Load Diff

84
backend/backend/1/custom_exp_1/exp.py Normal file
View File

@@ -0,0 +1,84 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate" # Where images are located
self.annotations_dir = "./backend/1/custom_exp_1" # Where annotation JSONs are located
self.train_ann = "coco_project_1_train.json"
self.val_ann = "coco_project_1_valid.json"
self.test_ann = "coco_project_1_test.json"
self.num_classes = 2
# Disable train2017 subdirectory - our images are directly in data_dir
self.name = ""
# Set data workers for training
self.data_num_workers = 8
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
def get_dataset(self, cache=False, cache_type="ram"):
"""Override to use name parameter for images directory"""
from yolox.data import COCODataset
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
# Use empty string for name since we have absolute paths in JSON
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.train_ann,
name="",
img_size=self.input_size,
preproc=self.preproc if hasattr(self, 'preproc') else None,
cache=cache,
cache_type=cache_type,
)
def get_eval_dataset(self, **kwargs):
"""Override eval dataset using name parameter"""
from yolox.data import COCODataset
testdev = kwargs.get("testdev", False)
legacy = kwargs.get("legacy", False)
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.val_ann if not testdev else self.test_ann,
name="",
img_size=self.test_size,
preproc=None, # No preprocessing for evaluation
)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

28
backend/check_db.py
View File

@@ -1,14 +1,14 @@
import pymysql import pymysql
conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp') conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp2')
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute('DESCRIBE image') cursor.execute('DESCRIBE image')
rows = cursor.fetchall() rows = cursor.fetchall()
print("Current 'image' table structure:") print("Current 'image' table structure:")
print("-" * 60) print("-" * 60)
for row in rows: for row in rows:
print(f"Field: {row[0]:<15} Type: {row[1]:<15} Null: {row[2]}") print(f"Field: {row[0]:<15} Type: {row[1]:<15} Null: {row[2]}")
print("-" * 60) print("-" * 60)
conn.close() conn.close()

280
backend/data/README.md
View File

@@ -1,140 +1,140 @@
# YOLOX Base Configuration System # YOLOX Base Configuration System
## Overview ## Overview
This directory contains base experiment configurations for YOLOX models. These configurations define "protected" parameters that are preserved during transfer learning from COCO-pretrained models. This directory contains base experiment configurations for YOLOX models. These configurations define "protected" parameters that are preserved during transfer learning from COCO-pretrained models.
## How It Works ## How It Works
### Transfer Learning Flow ### Transfer Learning Flow
1. **COCO Transfer Learning** (`transfer_learning = 'coco'`): 1. **COCO Transfer Learning** (`transfer_learning = 'coco'`):
- Loads base configuration from `data/yolox_*.py` based on `selected_model` - Loads base configuration from `data/yolox_*.py` based on `selected_model`
- Base parameters are **protected** and used as defaults - Base parameters are **protected** and used as defaults
- User settings from the form only override what's explicitly set - User settings from the form only override what's explicitly set
- Result: Best of both worlds - proven COCO settings + your customizations - Result: Best of both worlds - proven COCO settings + your customizations
2. **Sketch/Custom Training** (`transfer_learning = 'sketch'`): 2. **Sketch/Custom Training** (`transfer_learning = 'sketch'`):
- No base configuration loaded - No base configuration loaded
- Uses only user-defined parameters from the training form - Uses only user-defined parameters from the training form
- Full control over all settings - Full control over all settings
### Base Configuration Files ### Base Configuration Files
- `yolox_s.py` - YOLOX-Small (depth=0.33, width=0.50) - `yolox_s.py` - YOLOX-Small (depth=0.33, width=0.50)
- `yolox_m.py` - YOLOX-Medium (depth=0.67, width=0.75) - `yolox_m.py` - YOLOX-Medium (depth=0.67, width=0.75)
- `yolox_l.py` - YOLOX-Large (depth=1.0, width=1.0) - `yolox_l.py` - YOLOX-Large (depth=1.0, width=1.0)
- `yolox_x.py` - YOLOX-XLarge (depth=1.33, width=1.25) - `yolox_x.py` - YOLOX-XLarge (depth=1.33, width=1.25)
### Protected Parameters ### Protected Parameters
These parameters are defined in base configs and **preserved** unless explicitly overridden: These parameters are defined in base configs and **preserved** unless explicitly overridden:
**Model Architecture:** **Model Architecture:**
- `depth` - Model depth multiplier - `depth` - Model depth multiplier
- `width` - Model width multiplier - `width` - Model width multiplier
- `activation` - Activation function (silu) - `activation` - Activation function (silu)
**Training Hyperparameters:** **Training Hyperparameters:**
- `basic_lr_per_img` - Learning rate per image - `basic_lr_per_img` - Learning rate per image
- `scheduler` - LR scheduler (yoloxwarmcos) - `scheduler` - LR scheduler (yoloxwarmcos)
- `warmup_epochs` - Warmup epochs - `warmup_epochs` - Warmup epochs
- `max_epoch` - Maximum training epochs - `max_epoch` - Maximum training epochs
- `no_aug_epochs` - No augmentation epochs - `no_aug_epochs` - No augmentation epochs
- `min_lr_ratio` - Minimum LR ratio - `min_lr_ratio` - Minimum LR ratio
**Optimizer:** **Optimizer:**
- `momentum` - SGD momentum - `momentum` - SGD momentum
- `weight_decay` - Weight decay - `weight_decay` - Weight decay
**Augmentation:** **Augmentation:**
- `mosaic_prob` - Mosaic probability - `mosaic_prob` - Mosaic probability
- `mixup_prob` - Mixup probability - `mixup_prob` - Mixup probability
- `hsv_prob` - HSV augmentation probability - `hsv_prob` - HSV augmentation probability
- `flip_prob` - Flip probability - `flip_prob` - Flip probability
- `degrees` - Rotation degrees - `degrees` - Rotation degrees
- `translate` - Translation - `translate` - Translation
- `shear` - Shear - `shear` - Shear
- `mosaic_scale` - Mosaic scale range - `mosaic_scale` - Mosaic scale range
- `mixup_scale` - Mixup scale range - `mixup_scale` - Mixup scale range
- `enable_mixup` - Enable mixup - `enable_mixup` - Enable mixup
**Input/Output:** **Input/Output:**
- `input_size` - Training input size - `input_size` - Training input size
- `test_size` - Testing size - `test_size` - Testing size
- `random_size` - Random size range - `random_size` - Random size range
**Evaluation:** **Evaluation:**
- `eval_interval` - Evaluation interval - `eval_interval` - Evaluation interval
- `print_interval` - Print interval - `print_interval` - Print interval
## Customizing Base Configurations ## Customizing Base Configurations
### Adding a New Model ### Adding a New Model
Create a new file `data/yolox_MODELNAME.py`: Create a new file `data/yolox_MODELNAME.py`:
```python ```python
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-MODELNAME # Base configuration for YOLOX-MODELNAME
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-MODELNAME""" """Base experiment configuration for YOLOX-MODELNAME"""
# Define protected parameters # Define protected parameters
depth = 1.0 depth = 1.0
width = 1.0 width = 1.0
# ... other parameters # ... other parameters
``` ```
### Modifying Parameters ### Modifying Parameters
Edit the corresponding `yolox_*.py` file and update the `BaseExp` class attributes. Edit the corresponding `yolox_*.py` file and update the `BaseExp` class attributes.
**Example:** To change YOLOX-S max epochs: **Example:** To change YOLOX-S max epochs:
```python ```python
# In data/yolox_s.py # In data/yolox_s.py
class BaseExp: class BaseExp:
max_epoch = 500 # Changed from 300 max_epoch = 500 # Changed from 300
# ... other parameters # ... other parameters
``` ```
## Parameter Priority ## Parameter Priority
The merge logic follows this priority (highest to lowest): The merge logic follows this priority (highest to lowest):
1. **User form values** (if explicitly set, not None) 1. **User form values** (if explicitly set, not None)
2. **Base config values** (if transfer_learning='coco') 2. **Base config values** (if transfer_learning='coco')
3. **Default fallbacks** (hardcoded minimums) 3. **Default fallbacks** (hardcoded minimums)
## Example ## Example
### COCO Transfer Learning ### COCO Transfer Learning
``` ```
User sets in form: max_epoch=100, depth=0.5 User sets in form: max_epoch=100, depth=0.5
Base config (yolox_s.py) has: depth=0.33, width=0.50, max_epoch=300 Base config (yolox_s.py) has: depth=0.33, width=0.50, max_epoch=300
Result: depth=0.5 (user override), width=0.50 (base), max_epoch=100 (user override) Result: depth=0.5 (user override), width=0.50 (base), max_epoch=100 (user override)
``` ```
### Sketch Training ### Sketch Training
``` ```
User sets in form: max_epoch=100, depth=0.5 User sets in form: max_epoch=100, depth=0.5
No base config loaded No base config loaded
Result: depth=0.5 (user), max_epoch=100 (user), width=1.0 (default fallback) Result: depth=0.5 (user), max_epoch=100 (user), width=1.0 (default fallback)
``` ```
## Debugging ## Debugging
To see which base config was loaded, check Flask logs: To see which base config was loaded, check Flask logs:
``` ```
Loaded base config for yolox-s: ['depth', 'width', 'activation', ...] Loaded base config for yolox-s: ['depth', 'width', 'activation', ...]
``` ```
If base config fails to load: If base config fails to load:
``` ```
Warning: Could not load base config for yolox-s: [error message] Warning: Could not load base config for yolox-s: [error message]
Falling back to custom settings only Falling back to custom settings only
``` ```

2
backend/data/__init__.py
View File

@@ -1 +1 @@
# Base experiment configurations for YOLOX models # Base experiment configurations for YOLOX models

158
backend/data/test_base_configs.py
View File

@@ -1,79 +1,79 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Test script to demonstrate base configuration loading for YOLOX models Test script to demonstrate base configuration loading for YOLOX models
""" """
import sys import sys
import os import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..')) sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from services.generate_yolox_exp import load_base_config from services.generate_yolox_exp import load_base_config
def test_base_configs(): def test_base_configs():
"""Test loading all base configurations""" """Test loading all base configurations"""
models = ['yolox-s', 'yolox-m', 'yolox-l', 'yolox-x'] models = ['yolox-s', 'yolox-m', 'yolox-l', 'yolox-x']
print("=" * 80) print("=" * 80)
print("YOLOX Base Configuration Test") print("YOLOX Base Configuration Test")
print("=" * 80) print("=" * 80)
for model in models: for model in models:
print(f"\n{'='*80}") print(f"\n{'='*80}")
print(f"Model: {model.upper()}") print(f"Model: {model.upper()}")
print(f"{'='*80}") print(f"{'='*80}")
try: try:
config = load_base_config(model) config = load_base_config(model)
# Group parameters by category # Group parameters by category
arch_params = ['depth', 'width', 'activation'] arch_params = ['depth', 'width', 'activation']
training_params = ['max_epoch', 'warmup_epochs', 'basic_lr_per_img', 'scheduler', training_params = ['max_epoch', 'warmup_epochs', 'basic_lr_per_img', 'scheduler',
'no_aug_epochs', 'min_lr_ratio'] 'no_aug_epochs', 'min_lr_ratio']
optimizer_params = ['momentum', 'weight_decay'] optimizer_params = ['momentum', 'weight_decay']
augmentation_params = ['mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', augmentation_params = ['mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob',
'degrees', 'translate', 'shear', 'mosaic_scale', 'degrees', 'translate', 'shear', 'mosaic_scale',
'mixup_scale', 'enable_mixup'] 'mixup_scale', 'enable_mixup']
input_params = ['input_size', 'test_size', 'random_size'] input_params = ['input_size', 'test_size', 'random_size']
eval_params = ['eval_interval', 'print_interval'] eval_params = ['eval_interval', 'print_interval']
print("\n[Architecture]") print("\n[Architecture]")
for param in arch_params: for param in arch_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Training Hyperparameters]") print("\n[Training Hyperparameters]")
for param in training_params: for param in training_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Optimizer]") print("\n[Optimizer]")
for param in optimizer_params: for param in optimizer_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Data Augmentation]") print("\n[Data Augmentation]")
for param in augmentation_params: for param in augmentation_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Input/Output]") print("\n[Input/Output]")
for param in input_params: for param in input_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Evaluation]") print("\n[Evaluation]")
for param in eval_params: for param in eval_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print(f"\n✓ Successfully loaded {len(config)} parameters") print(f"\n✓ Successfully loaded {len(config)} parameters")
except Exception as e: except Exception as e:
print(f"✗ Error loading config: {e}") print(f"✗ Error loading config: {e}")
print("\n" + "="*80) print("\n" + "="*80)
print("Test Complete") print("Test Complete")
print("="*80) print("="*80)
if __name__ == '__main__': if __name__ == '__main__':
test_base_configs() test_base_configs()

30
backend/data/yolox_l.py
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-L model # Base configuration for YOLOX-L model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-L""" """Base experiment configuration for YOLOX-L"""
# Model architecture (protected - always use these for yolox-l) # Model architecture (protected - always use these for yolox-l)
depth = 1.0 depth = 1.0
width = 1.0 width = 1.0
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

28
backend/data/yolox_m.py
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-M model # Base configuration for YOLOX-M model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-M""" """Base experiment configuration for YOLOX-M"""
# Model architecture (protected - always use these for yolox-m) # Model architecture (protected - always use these for yolox-m)
depth = 0.67 depth = 0.67
width = 0.75 width = 0.75
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

34
backend/data/yolox_s.py
View File

@@ -1,17 +1,17 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-S model # Base configuration for YOLOX-S model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-S""" """Base experiment configuration for YOLOX-S"""
# Model architecture (protected - always use these for yolox-s) # Model architecture (protected - always use these for yolox-s)
depth = 0.33 depth = 0.33
width = 0.50 width = 0.50
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

28
backend/data/yolox_x.py
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-X model # Base configuration for YOLOX-X model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-X""" """Base experiment configuration for YOLOX-X"""
# Model architecture (protected - always use these for yolox-x) # Model architecture (protected - always use these for yolox-x)
depth = 1.33 depth = 1.33
width = 1.25 width = 1.25
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

8
backend/database/__init__.py
View File

@@ -1,4 +1,4 @@
# Database module # Database module
from database.database import db from database.database import db
__all__ = ['db'] __all__ = ['db']

20
backend/database/database.js
View File

@@ -1,10 +1,10 @@
// database.js // database.js
const { Sequelize } = require('sequelize'); const { Sequelize } = require('sequelize');
const sequelize = new Sequelize('myapp', 'root', 'root', { const sequelize = new Sequelize('myapp', 'root', 'root', {
host: 'localhost', host: 'localhost',
dialect: 'mysql', dialect: 'mysql',
logging: false, logging: false,
}); });
module.exports = sequelize; module.exports = sequelize;

18
backend/database/database.py
View File

@@ -1,9 +1,9 @@
from flask_sqlalchemy import SQLAlchemy from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy() db = SQLAlchemy()
def init_db(app): def init_db(app):
"""Initialize database with app context""" """Initialize database with app context"""
db.init_app(app) db.init_app(app)
with app.app_context(): with app.app_context():
db.create_all() db.create_all()

12
backend/database/migration_add_image_dimensions.sql
View File

@@ -1,12 +0,0 @@
-- Migration: Add width and height columns to image table
-- Date: 2025-11-27
USE myapp;
-- Add width and height columns to image table
ALTER TABLE `image`
ADD COLUMN `width` FLOAT NULL AFTER `image_path`,
ADD COLUMN `height` FLOAT NULL AFTER `width`;
-- Verify the changes
DESCRIBE `image`;

80
backend/models/Annotation.js
View File

@@ -1,40 +1,40 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const Annotation = sequelize.define('Annotation', { const Annotation = sequelize.define('Annotation', {
annotation_id: { annotation_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
primaryKey: true, primaryKey: true,
autoIncrement: true, autoIncrement: true,
}, },
image_id: { image_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
allowNull: false, allowNull: false,
}, },
x: { x: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: false, allowNull: false,
}, },
y: { y: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: false, allowNull: false,
}, },
height: { height: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: false, allowNull: false,
}, },
width: { width: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: false, allowNull: false,
}, },
Label: { Label: {
type: DataTypes.STRING, type: DataTypes.STRING,
allowNull: false, allowNull: false,
}, },
}, { }, {
tableName: 'annotation', tableName: 'annotation',
timestamps: false, timestamps: false,
}); });
module.exports = Annotation; module.exports = Annotation;

46
backend/models/Annotation.py
View File

@@ -1,23 +1,23 @@
from database.database import db from database.database import db
class Annotation(db.Model): class Annotation(db.Model):
__tablename__ = 'annotation' __tablename__ = 'annotation'
annotation_id = db.Column(db.Integer, primary_key=True, autoincrement=True) annotation_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_id = db.Column(db.Integer, nullable=False) image_id = db.Column(db.Integer, db.ForeignKey('image.image_id', ondelete='CASCADE'), nullable=False)
x = db.Column(db.Float, nullable=False) x = db.Column(db.Float, nullable=False)
y = db.Column(db.Float, nullable=False) y = db.Column(db.Float, nullable=False)
height = db.Column(db.Float, nullable=False) height = db.Column(db.Float, nullable=False)
width = db.Column(db.Float, nullable=False) width = db.Column(db.Float, nullable=False)
Label = db.Column(db.String(255), nullable=False) Label = db.Column(db.String(255), nullable=False)
def to_dict(self): def to_dict(self):
return { return {
'annotation_id': self.annotation_id, 'annotation_id': self.annotation_id,
'image_id': self.image_id, 'image_id': self.image_id,
'x': self.x, 'x': self.x,
'y': self.y, 'y': self.y,
'height': self.height, 'height': self.height,
'width': self.width, 'width': self.width,
'Label': self.Label 'Label': self.Label
} }

21
backend/models/AnnotationProjectMapping.py Normal file
View File

@@ -0,0 +1,21 @@
from database.database import db
class AnnotationProjectMapping(db.Model):
"""Mapping between training project details and label studio projects (3NF)"""
__tablename__ = 'annotation_project_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
# Unique constraint: each label studio project can only be mapped once per training project details
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', name='uq_annotation_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id
}

25
backend/models/ClassMapping.py Normal file
View File

@@ -0,0 +1,25 @@
from database.database import db
class ClassMapping(db.Model):
"""Class name mappings for training project details (3NF)"""
__tablename__ = 'class_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
source_class = db.Column(db.String(255), nullable=False)
target_class = db.Column(db.String(255), nullable=False)
# Unique constraint: each source class can only be mapped once per project details AND label studio project
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', 'source_class', name='uq_class_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id,
'source_class': self.source_class,
'target_class': self.target_class
}

70
backend/models/Images.js
View File

@@ -1,35 +1,35 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const Image = sequelize.define('Image', { const Image = sequelize.define('Image', {
image_id: { image_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
primaryKey: true, primaryKey: true,
autoIncrement: true, autoIncrement: true,
}, },
image_path: { image_path: {
type: DataTypes.STRING, type: DataTypes.STRING,
allowNull: false, allowNull: false,
}, },
project_id: { project_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
allowNull: false, allowNull: false,
}, },
width: { width: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: true, allowNull: true,
}, },
height: { height: {
type: DataTypes.FLOAT, type: DataTypes.FLOAT,
allowNull: true, allowNull: true,
}, },
}, { }, {
tableName: 'image', tableName: 'image',
timestamps: false, timestamps: false,
}); });
module.exports = Image; module.exports = Image;

38
backend/models/Images.py
View File

@@ -1,19 +1,19 @@
from database.database import db from database.database import db
class Image(db.Model): class Image(db.Model):
__tablename__ = 'image' __tablename__ = 'image'
image_id = db.Column(db.Integer, primary_key=True, autoincrement=True) image_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_path = db.Column(db.String(500), nullable=False) image_path = db.Column(db.String(500), nullable=False)
project_id = db.Column(db.Integer, nullable=False) project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
width = db.Column(db.Float) width = db.Column(db.Integer)
height = db.Column(db.Float) height = db.Column(db.Integer)
def to_dict(self): def to_dict(self):
return { return {
'image_id': self.image_id, 'image_id': self.image_id,
'image_path': self.image_path, 'image_path': self.image_path,
'project_id': self.project_id, 'project_id': self.project_id,
'width': self.width, 'width': self.width,
'height': self.height 'height': self.height
} }

48
backend/models/LabelStudioProject.js
View File

@@ -1,24 +1,24 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const Label_studio_project = sequelize.define('LabelStudioProject', { const Label_studio_project = sequelize.define('LabelStudioProject', {
project_id: { project_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
primaryKey: true, primaryKey: true,
unique: true, unique: true,
allowNull: false, allowNull: false,
}, },
title:{ title:{
type: DataTypes.STRING, type: DataTypes.STRING,
allowNull: false, allowNull: false,
} }
}, { }, {
tableName: 'label_studio_project', tableName: 'label_studio_project',
timestamps: false, timestamps: false,
}); });
module.exports = Label_studio_project; module.exports = Label_studio_project;

26
backend/models/LabelStudioProject.py
View File

@@ -1,13 +1,13 @@
from database.database import db from database.database import db
class LabelStudioProject(db.Model): class LabelStudioProject(db.Model):
__tablename__ = 'label_studio_project' __tablename__ = 'label_studio_project'
project_id = db.Column(db.Integer, primary_key=True, unique=True) project_id = db.Column(db.Integer, primary_key=True, unique=True)
title = db.Column(db.String(255), nullable=False) title = db.Column(db.String(255), nullable=False)
def to_dict(self): def to_dict(self):
return { return {
'project_id': self.project_id, 'project_id': self.project_id,
'title': self.title 'title': self.title
} }

23
backend/models/ProjectClass.py Normal file
View File

@@ -0,0 +1,23 @@
from database.database import db
class ProjectClass(db.Model):
"""Class definitions for training projects (3NF)"""
__tablename__ = 'project_class'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False)
class_name = db.Column(db.String(255), nullable=False)
display_order = db.Column(db.Integer, default=0)
# Unique constraint: one class name per project
__table_args__ = (
db.UniqueConstraint('project_id', 'class_name', name='uq_project_class'),
)
def to_dict(self):
return {
'id': self.id,
'project_id': self.project_id,
'class_name': self.class_name,
'display_order': self.display_order
}

21
backend/models/Settings.py Normal file
View File

@@ -0,0 +1,21 @@
from database.database import db
class Settings(db.Model):
__tablename__ = 'settings'
id = db.Column(db.Integer, primary_key=True)
key = db.Column(db.String(255), unique=True, nullable=False)
value = db.Column(db.Text, nullable=True)
description = db.Column(db.String(500), nullable=True)
created_at = db.Column(db.DateTime, server_default=db.func.now())
updated_at = db.Column(db.DateTime, server_default=db.func.now(), onupdate=db.func.now())
def to_dict(self):
return {
'id': self.id,
'key': self.key,
'value': self.value,
'description': self.description,
'created_at': self.created_at.isoformat() if self.created_at else None,
'updated_at': self.updated_at.isoformat() if self.updated_at else None
}

76
backend/models/TrainingProject.js
View File

@@ -1,38 +1,38 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const Training_Project = sequelize.define('LabelStudioProject', { const Training_Project = sequelize.define('LabelStudioProject', {
project_id: { project_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
primaryKey: true, primaryKey: true,
unique: true, unique: true,
allowNull: false, allowNull: false,
autoIncrement: true, autoIncrement: true,
}, },
title:{ title:{
type: DataTypes.STRING, type: DataTypes.STRING,
allowNull: false, allowNull: false,
}, },
description: { description: {
type: DataTypes.STRING, type: DataTypes.STRING,
}, },
classes: { classes: {
type: DataTypes.JSON, type: DataTypes.JSON,
allowNull: false, allowNull: false,
}, },
project_image: { project_image: {
type: DataTypes.BLOB, type: DataTypes.BLOB,
}, },
project_image_type: { project_image_type: {
type: DataTypes.STRING, type: DataTypes.STRING,
allowNull: true, allowNull: true,
} }
}, { }, {
tableName: 'training_project', tableName: 'training_project',
timestamps: false, timestamps: false,
}); });
module.exports = Training_Project; module.exports = Training_Project;

64
backend/models/TrainingProject.py
View File

@@ -1,28 +1,36 @@
from database.database import db from database.database import db
class TrainingProject(db.Model): class TrainingProject(db.Model):
__tablename__ = 'training_project' __tablename__ = 'training_project'
project_id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True) project_id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
title = db.Column(db.String(255), nullable=False) title = db.Column(db.String(255), nullable=False)
description = db.Column(db.String(500)) description = db.Column(db.String(500))
classes = db.Column(db.JSON, nullable=False) project_image = db.Column(db.LargeBinary)
project_image = db.Column(db.LargeBinary) project_image_type = db.Column(db.String(100))
project_image_type = db.Column(db.String(100))
# Relationship to classes (3NF)
def to_dict(self): classes_relation = db.relationship('ProjectClass', backref='project', lazy=True, cascade='all, delete-orphan')
result = {
'project_id': self.project_id, def to_dict(self, include_classes=True):
'title': self.title, result = {
'description': self.description, 'project_id': self.project_id,
'classes': self.classes, 'title': self.title,
'project_image_type': self.project_image_type 'description': self.description,
} 'project_image_type': self.project_image_type
if self.project_image: }
import base64
base64_data = base64.b64encode(self.project_image).decode('utf-8') # Include classes as array for backwards compatibility
mime_type = self.project_image_type or 'image/png' if include_classes:
result['project_image'] = f'data:{mime_type};base64,{base64_data}' from models.ProjectClass import ProjectClass
else: classes = ProjectClass.query.filter_by(project_id=self.project_id).order_by(ProjectClass.display_order).all()
result['project_image'] = None result['classes'] = [c.class_name for c in classes]
return result
if self.project_image:
import base64
base64_data = base64.b64encode(self.project_image).decode('utf-8')
mime_type = self.project_image_type or 'image/png'
result['project_image'] = f'data:{mime_type};base64,{base64_data}'
else:
result['project_image'] = None
return result

66
backend/models/TrainingProjectDetails.js
View File

@@ -1,33 +1,33 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const TrainingProjectDetails = sequelize.define('TrainingProjectDetails', { const TrainingProjectDetails = sequelize.define('TrainingProjectDetails', {
id: { id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
primaryKey: true, primaryKey: true,
autoIncrement: true, autoIncrement: true,
unique: true, unique: true,
}, },
project_id: { project_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
allowNull: false, allowNull: false,
unique: true, unique: true,
}, },
annotation_projects: { annotation_projects: {
type: DataTypes.JSON, type: DataTypes.JSON,
allowNull: false, allowNull: false,
}, },
class_map: { class_map: {
type: DataTypes.JSON, type: DataTypes.JSON,
allowNull: true, allowNull: true,
}, },
description: { description: {
type: DataTypes.JSON, type: DataTypes.JSON,
allowNull: true, allowNull: true,
} }
}, { }, {
tableName: 'training_project_details', tableName: 'training_project_details',
timestamps: false, timestamps: false,
}); });
module.exports = TrainingProjectDetails; module.exports = TrainingProjectDetails;

54
backend/models/TrainingProjectDetails.py
View File

@@ -1,19 +1,35 @@
from database.database import db from database.database import db
class TrainingProjectDetails(db.Model): class TrainingProjectDetails(db.Model):
__tablename__ = 'training_project_details' __tablename__ = 'training_project_details'
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True) id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
project_id = db.Column(db.Integer, nullable=False, unique=True) project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False, unique=True)
annotation_projects = db.Column(db.JSON, nullable=False) description_text = db.Column(db.Text) # Renamed from 'description' JSON to plain text
class_map = db.Column(db.JSON)
description = db.Column(db.JSON) # Relationships (3NF)
annotation_mappings = db.relationship('AnnotationProjectMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
def to_dict(self): class_mappings = db.relationship('ClassMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
return {
'id': self.id, def to_dict(self, include_mappings=True):
'project_id': self.project_id, result = {
'annotation_projects': self.annotation_projects, 'id': self.id,
'class_map': self.class_map, 'project_id': self.project_id,
'description': self.description 'description': self.description_text
} }
# Include mappings for backwards compatibility
if include_mappings:
from models.AnnotationProjectMapping import AnnotationProjectMapping
from models.ClassMapping import ClassMapping
# Get annotation projects as array
mappings = AnnotationProjectMapping.query.filter_by(project_details_id=self.id).all()
result['annotation_projects'] = [m.label_studio_project_id for m in mappings]
# Get class map as dictionary (grouped by label_studio_project_id for backwards compatibility)
# Return format: {source: target} (flattened across all projects)
class_maps = ClassMapping.query.filter_by(project_details_id=self.id).all()
result['class_map'] = {cm.source_class: cm.target_class for cm in class_maps}
return result

25
backend/models/TrainingSize.py Normal file
View File

@@ -0,0 +1,25 @@
from database.database import db
class TrainingSize(db.Model):
"""Size configurations for training (3NF - replaces JSON arrays)"""
__tablename__ = 'training_size'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
training_id = db.Column(db.Integer, db.ForeignKey('training.id', ondelete='CASCADE'), nullable=False)
size_type = db.Column(db.String(50), nullable=False) # 'input_size', 'test_size', 'mosaic_scale', 'mixup_scale'
value_order = db.Column(db.Integer, nullable=False, default=0) # Order in array (0=first, 1=second)
value = db.Column(db.Float, nullable=False)
# Composite key ensures proper ordering
__table_args__ = (
db.UniqueConstraint('training_id', 'size_type', 'value_order', name='uq_training_size'),
)
def to_dict(self):
return {
'id': self.id,
'training_id': self.training_id,
'size_type': self.size_type,
'value_order': self.value_order,
'value': self.value
}

42
backend/models/__init__.py
View File

@@ -1,16 +1,26 @@
# Import all models to ensure they are registered with SQLAlchemy # Import all models to ensure they are registered with SQLAlchemy
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from models.training import Training from models.training import Training
from models.LabelStudioProject import LabelStudioProject from models.LabelStudioProject import LabelStudioProject
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
from models.Settings import Settings
__all__ = [ from models.ProjectClass import ProjectClass
'TrainingProject', from models.AnnotationProjectMapping import AnnotationProjectMapping
'TrainingProjectDetails', from models.ClassMapping import ClassMapping
'Training', from models.TrainingSize import TrainingSize
'LabelStudioProject',
'Image', __all__ = [
'Annotation' 'TrainingProject',
] 'TrainingProjectDetails',
'Training',
'LabelStudioProject',
'Image',
'Annotation',
'Settings',
'ProjectClass',
'AnnotationProjectMapping',
'ClassMapping',
'TrainingSize'
]

60
backend/models/index.js
View File

@@ -1,30 +1,30 @@
const LabelStudioProject = require('./LabelStudioProject.js'); const LabelStudioProject = require('./LabelStudioProject.js');
const Annotation = require('./Annotation.js'); const Annotation = require('./Annotation.js');
const Image = require('./Images.js'); const Image = require('./Images.js');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const TrainingProjectDetails = require('./TrainingProjectDetails.js'); const TrainingProjectDetails = require('./TrainingProjectDetails.js');
const TrainingProject = require('./TrainingProject.js'); const TrainingProject = require('./TrainingProject.js');
const Training = require('./training.js'); const Training = require('./training.js');
const Project = LabelStudioProject; const Project = LabelStudioProject;
const Img = Image; const Img = Image;
const Ann = Annotation; const Ann = Annotation;
// Associations // Associations
Project.hasMany(Img, { foreignKey: 'project_id' }); Project.hasMany(Img, { foreignKey: 'project_id' });
Img.belongsTo(Project, { foreignKey: 'project_id' }); Img.belongsTo(Project, { foreignKey: 'project_id' });
Img.hasMany(Ann, { foreignKey: 'image_id' }); Img.hasMany(Ann, { foreignKey: 'image_id' });
Ann.belongsTo(Img, { foreignKey: 'image_id' }); Ann.belongsTo(Img, { foreignKey: 'image_id' });
// TrainingProjectDetails <-> TrainingProject // TrainingProjectDetails <-> TrainingProject
TrainingProjectDetails.belongsTo(TrainingProject, { foreignKey: 'project_id' }); TrainingProjectDetails.belongsTo(TrainingProject, { foreignKey: 'project_id' });
TrainingProject.hasOne(TrainingProjectDetails, { foreignKey: 'project_id' }); TrainingProject.hasOne(TrainingProjectDetails, { foreignKey: 'project_id' });
// Training <-> TrainingProjectDetails // Training <-> TrainingProjectDetails
Training.belongsTo(TrainingProjectDetails, { foreignKey: 'project_details_id' }); Training.belongsTo(TrainingProjectDetails, { foreignKey: 'project_details_id' });
TrainingProjectDetails.hasMany(Training, { foreignKey: 'project_details_id' }); TrainingProjectDetails.hasMany(Training, { foreignKey: 'project_details_id' });
module.exports = { Project, Img, Ann, TrainingProjectDetails, TrainingProject, Training }; module.exports = { Project, Img, Ann, TrainingProjectDetails, TrainingProject, Training };

280
backend/models/training.js
View File

@@ -1,140 +1,140 @@
const { DataTypes } = require('sequelize'); const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const Training = sequelize.define('training', { const Training = sequelize.define('training', {
id: { id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
autoIncrement: true, autoIncrement: true,
unique: true, unique: true,
primaryKey: true primaryKey: true
}, },
exp_name: { exp_name: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
max_epoch: { max_epoch: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
depth: { depth: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
width: { width: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
activation: { activation: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
warmup_epochs: { warmup_epochs: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
warmup_lr: { warmup_lr: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
basic_lr_per_img: { basic_lr_per_img: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
scheduler: { scheduler: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
no_aug_epochs: { no_aug_epochs: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
min_lr_ratio: { min_lr_ratio: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
ema: { ema: {
type: DataTypes.BOOLEAN type: DataTypes.BOOLEAN
}, },
weight_decay: { weight_decay: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
momentum: { momentum: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
input_size: { input_size: {
type: DataTypes.JSON type: DataTypes.JSON
}, },
print_interval: { print_interval: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
eval_interval: { eval_interval: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
save_history_ckpt: { save_history_ckpt: {
type: DataTypes.BOOLEAN type: DataTypes.BOOLEAN
}, },
test_size: { test_size: {
type: DataTypes.JSON type: DataTypes.JSON
}, },
test_conf: { test_conf: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
nms_thre: { nms_thre: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
multiscale_range: { multiscale_range: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
enable_mixup: { enable_mixup: {
type: DataTypes.BOOLEAN type: DataTypes.BOOLEAN
}, },
mosaic_prob: { mosaic_prob: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
mixup_prob: { mixup_prob: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
hsv_prob: { hsv_prob: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
flip_prob: { flip_prob: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
degrees: { degrees: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
mosaic_scale: { mosaic_scale: {
type: DataTypes.JSON type: DataTypes.JSON
}, },
mixup_scale: { mixup_scale: {
type: DataTypes.JSON type: DataTypes.JSON
}, },
translate: { translate: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
shear: { shear: {
type: DataTypes.FLOAT type: DataTypes.FLOAT
}, },
training_name: { training_name: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
project_details_id: { project_details_id: {
type: DataTypes.INTEGER, type: DataTypes.INTEGER,
allowNull: false allowNull: false
}, },
seed: { seed: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
train: { train: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
valid: { valid: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
test: { test: {
type: DataTypes.INTEGER type: DataTypes.INTEGER
}, },
selected_model: { selected_model: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
transfer_learning: { transfer_learning: {
type: DataTypes.STRING(255) type: DataTypes.STRING(255)
}, },
model_upload: { model_upload: {
type: DataTypes.BLOB type: DataTypes.BLOB
} }
}, { }, {
tableName: 'training', tableName: 'training',
timestamps: false timestamps: false
}); });
module.exports = Training; module.exports = Training;

201
backend/models/training.py
View File

@@ -1,92 +1,109 @@
from database.database import db from database.database import db
class Training(db.Model): class Training(db.Model):
__tablename__ = 'training' __tablename__ = 'training'
id = db.Column(db.Integer, primary_key=True, autoincrement=True, unique=True) id = db.Column(db.Integer, primary_key=True, autoincrement=True, unique=True)
exp_name = db.Column(db.String(255)) exp_name = db.Column(db.String(255))
max_epoch = db.Column(db.Integer) max_epoch = db.Column(db.Integer)
depth = db.Column(db.Float) depth = db.Column(db.Float)
width = db.Column(db.Float) width = db.Column(db.Float)
activation = db.Column(db.String(255)) activation = db.Column(db.String(255))
warmup_epochs = db.Column(db.Integer) warmup_epochs = db.Column(db.Integer)
warmup_lr = db.Column(db.Float) warmup_lr = db.Column(db.Float)
basic_lr_per_img = db.Column(db.Float) basic_lr_per_img = db.Column(db.Float)
scheduler = db.Column(db.String(255)) scheduler = db.Column(db.String(255))
no_aug_epochs = db.Column(db.Integer) no_aug_epochs = db.Column(db.Integer)
min_lr_ratio = db.Column(db.Float) min_lr_ratio = db.Column(db.Float)
ema = db.Column(db.Boolean) ema = db.Column(db.Boolean)
weight_decay = db.Column(db.Float) weight_decay = db.Column(db.Float)
momentum = db.Column(db.Float) momentum = db.Column(db.Float)
input_size = db.Column(db.JSON) # input_size moved to TrainingSize table
print_interval = db.Column(db.Integer) print_interval = db.Column(db.Integer)
eval_interval = db.Column(db.Integer) eval_interval = db.Column(db.Integer)
save_history_ckpt = db.Column(db.Boolean) save_history_ckpt = db.Column(db.Boolean)
test_size = db.Column(db.JSON) # test_size moved to TrainingSize table
test_conf = db.Column(db.Float) test_conf = db.Column(db.Float)
nms_thre = db.Column(db.Float) nms_thre = db.Column(db.Float)
multiscale_range = db.Column(db.Integer) multiscale_range = db.Column(db.Integer)
enable_mixup = db.Column(db.Boolean) enable_mixup = db.Column(db.Boolean)
mosaic_prob = db.Column(db.Float) mosaic_prob = db.Column(db.Float)
mixup_prob = db.Column(db.Float) mixup_prob = db.Column(db.Float)
hsv_prob = db.Column(db.Float) hsv_prob = db.Column(db.Float)
flip_prob = db.Column(db.Float) flip_prob = db.Column(db.Float)
degrees = db.Column(db.Float) degrees = db.Column(db.Float)
mosaic_scale = db.Column(db.JSON) # mosaic_scale moved to TrainingSize table
mixup_scale = db.Column(db.JSON) # mixup_scale moved to TrainingSize table
translate = db.Column(db.Float) translate = db.Column(db.Float)
shear = db.Column(db.Float) shear = db.Column(db.Float)
training_name = db.Column(db.String(255)) training_name = db.Column(db.String(255))
project_details_id = db.Column(db.Integer, nullable=False) project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
seed = db.Column(db.Integer) seed = db.Column(db.Integer)
train = db.Column(db.Integer) train = db.Column(db.Integer)
valid = db.Column(db.Integer) valid = db.Column(db.Integer)
test = db.Column(db.Integer) test = db.Column(db.Integer)
selected_model = db.Column(db.String(255)) selected_model = db.Column(db.String(255))
transfer_learning = db.Column(db.String(255)) transfer_learning = db.Column(db.String(255))
model_upload = db.Column(db.LargeBinary) model_upload = db.Column(db.LargeBinary)
def to_dict(self): # Relationship to size configurations (3NF)
return { size_configs = db.relationship('TrainingSize', backref='training', lazy=True, cascade='all, delete-orphan')
'id': self.id,
'exp_name': self.exp_name, def to_dict(self, include_sizes=True):
'max_epoch': self.max_epoch, result = {
'depth': self.depth, 'id': self.id,
'width': self.width, 'exp_name': self.exp_name,
'activation': self.activation, 'max_epoch': self.max_epoch,
'warmup_epochs': self.warmup_epochs, 'depth': self.depth,
'warmup_lr': self.warmup_lr, 'width': self.width,
'basic_lr_per_img': self.basic_lr_per_img, 'activation': self.activation,
'scheduler': self.scheduler, 'warmup_epochs': self.warmup_epochs,
'no_aug_epochs': self.no_aug_epochs, 'warmup_lr': self.warmup_lr,
'min_lr_ratio': self.min_lr_ratio, 'basic_lr_per_img': self.basic_lr_per_img,
'ema': self.ema, 'scheduler': self.scheduler,
'weight_decay': self.weight_decay, 'no_aug_epochs': self.no_aug_epochs,
'momentum': self.momentum, 'min_lr_ratio': self.min_lr_ratio,
'input_size': self.input_size, 'ema': self.ema,
'print_interval': self.print_interval, 'weight_decay': self.weight_decay,
'eval_interval': self.eval_interval, 'momentum': self.momentum,
'save_history_ckpt': self.save_history_ckpt, 'print_interval': self.print_interval,
'test_size': self.test_size, 'eval_interval': self.eval_interval,
'test_conf': self.test_conf, 'save_history_ckpt': self.save_history_ckpt,
'nms_thre': self.nms_thre, 'test_conf': self.test_conf,
'multiscale_range': self.multiscale_range, 'nms_thre': self.nms_thre,
'enable_mixup': self.enable_mixup, 'multiscale_range': self.multiscale_range,
'mosaic_prob': self.mosaic_prob, 'enable_mixup': self.enable_mixup,
'mixup_prob': self.mixup_prob, 'mosaic_prob': self.mosaic_prob,
'hsv_prob': self.hsv_prob, 'mixup_prob': self.mixup_prob,
'flip_prob': self.flip_prob, 'hsv_prob': self.hsv_prob,
'degrees': self.degrees, 'flip_prob': self.flip_prob,
'mosaic_scale': self.mosaic_scale, 'degrees': self.degrees,
'mixup_scale': self.mixup_scale, 'translate': self.translate,
'translate': self.translate, 'shear': self.shear,
'shear': self.shear, 'training_name': self.training_name,
'training_name': self.training_name, 'project_details_id': self.project_details_id,
'project_details_id': self.project_details_id, 'seed': self.seed,
'seed': self.seed, 'train': self.train,
'train': self.train, 'valid': self.valid,
'valid': self.valid, 'test': self.test,
'test': self.test, 'selected_model': self.selected_model,
'selected_model': self.selected_model, 'transfer_learning': self.transfer_learning
'transfer_learning': self.transfer_learning }
}
# Include size arrays for backwards compatibility
if include_sizes:
from models.TrainingSize import TrainingSize
def get_size_array(size_type):
sizes = TrainingSize.query.filter_by(
training_id=self.id,
size_type=size_type
).order_by(TrainingSize.value_order).all()
return [s.value for s in sizes] if sizes else None
result['input_size'] = get_size_array('input_size')
result['test_size'] = get_size_array('test_size')
result['mosaic_scale'] = get_size_array('mosaic_scale')
result['mixup_scale'] = get_size_array('mixup_scale')
return result

16
backend/requirements.txt
View File

@@ -1,8 +1,8 @@
Flask==3.0.0 Flask==3.0.0
Flask-CORS==4.0.0 Flask-CORS==4.0.0
Flask-SQLAlchemy==3.1.1 Flask-SQLAlchemy==3.1.1
SQLAlchemy==2.0.23 SQLAlchemy==2.0.23
PyMySQL==1.1.0 PyMySQL==1.1.0
python-dotenv==1.0.0 python-dotenv==1.0.0
requests==2.31.0 requests==2.31.0
Pillow==10.1.0 Pillow==10.1.0

2
backend/routes/__init__.py
View File

@@ -1 +1 @@
# Routes module # Routes module

992
backend/routes/api.js
View File

@@ -1,496 +1,496 @@
const express = require('express'); const express = require('express');
const multer = require('multer'); const multer = require('multer');
const upload = multer(); const upload = multer();
const TrainingProject = require('../models/TrainingProject.js'); const TrainingProject = require('../models/TrainingProject.js');
const LabelStudioProject = require('../models/LabelStudioProject.js') const LabelStudioProject = require('../models/LabelStudioProject.js')
const { seedLabelStudio, updateStatus } = require('../services/seed-label-studio.js'); const { seedLabelStudio, updateStatus } = require('../services/seed-label-studio.js');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const {generateTrainingJson} = require('../services/generate-json-yolox.js') const {generateTrainingJson} = require('../services/generate-json-yolox.js')
const router = express.Router(); const router = express.Router();
// Ensure JSON bodies are parsed for all routes // Ensure JSON bodies are parsed for all routes
router.use(express.json()); router.use(express.json());
router.get('/seed', async (req, res) => { router.get('/seed', async (req, res) => {
const result = await seedLabelStudio(); const result = await seedLabelStudio();
res.json(result); res.json(result);
}); });
// Trigger generate-json-yolox.js // Trigger generate-json-yolox.js
router.post('/generate-yolox-json', async (req, res) => { router.post('/generate-yolox-json', async (req, res) => {
const { project_id } = req.body; const { project_id } = req.body;
if (!project_id) { if (!project_id) {
return res.status(400).json({ message: 'Missing project_id in request body' }); return res.status(400).json({ message: 'Missing project_id in request body' });
} }
try { try {
// Generate COCO JSONs // Generate COCO JSONs
// Find all TrainingProjectDetails for this project // Find all TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } }); const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) { if (!detailsRows || detailsRows.length === 0) {
return res.status(404).json({ message: 'No TrainingProjectDetails found for project ' + project_id }); return res.status(404).json({ message: 'No TrainingProjectDetails found for project ' + project_id });
} }
// For each details row, generate coco.jsons and exp.py in projectfolder/project_details_id // For each details row, generate coco.jsons and exp.py in projectfolder/project_details_id
const Training = require('../models/training.js'); const Training = require('../models/training.js');
const { saveYoloxExp } = require('../services/generate-yolox-exp.js'); const { saveYoloxExp } = require('../services/generate-yolox-exp.js');
const TrainingProject = require('../models/TrainingProject.js'); const TrainingProject = require('../models/TrainingProject.js');
const trainingProject = await TrainingProject.findByPk(project_id); const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`; const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
for (const details of detailsRows) { for (const details of detailsRows) {
const detailsId = details.id; const detailsId = details.id;
await generateTrainingJson(detailsId); await generateTrainingJson(detailsId);
const trainings = await Training.findAll({ where: { project_details_id: detailsId } }); const trainings = await Training.findAll({ where: { project_details_id: detailsId } });
if (trainings.length === 0) continue; if (trainings.length === 0) continue;
// For each training, save exp.py in projectfolder/project_details_id // For each training, save exp.py in projectfolder/project_details_id
const outDir = path.join(__dirname, '..', projectName, String(detailsId)); const outDir = path.join(__dirname, '..', projectName, String(detailsId));
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true }); if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
for (const training of trainings) { for (const training of trainings) {
const expFilePath = path.join(outDir, 'exp.py'); const expFilePath = path.join(outDir, 'exp.py');
await saveYoloxExp(training.id, expFilePath); await saveYoloxExp(training.id, expFilePath);
} }
} }
// Find all trainings for this project // Find all trainings for this project
// ...existing code... // ...existing code...
res.json({ message: 'YOLOX JSON and exp.py generated for project ' + project_id }); res.json({ message: 'YOLOX JSON and exp.py generated for project ' + project_id });
} catch (err) { } catch (err) {
console.error('Error generating YOLOX JSON:', err); console.error('Error generating YOLOX JSON:', err);
res.status(500).json({ message: 'Failed to generate YOLOX JSON', error: err.message }); res.status(500).json({ message: 'Failed to generate YOLOX JSON', error: err.message });
} }
}); });
// Start YOLOX training // Start YOLOX training
const { spawn } = require('child_process'); const { spawn } = require('child_process');
router.post('/start-yolox-training', async (req, res) => { router.post('/start-yolox-training', async (req, res) => {
try { try {
const { project_id, training_id } = req.body; const { project_id, training_id } = req.body;
// Get project name // Get project name
const trainingProject = await TrainingProject.findByPk(project_id); const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`; const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
// Look up training row by id or project_details_id // Look up training row by id or project_details_id
const Training = require('../models/training.js'); const Training = require('../models/training.js');
let trainingRow = await Training.findByPk(training_id); let trainingRow = await Training.findByPk(training_id);
if (!trainingRow) { if (!trainingRow) {
trainingRow = await Training.findOne({ where: { project_details_id: training_id } }); trainingRow = await Training.findOne({ where: { project_details_id: training_id } });
} }
if (!trainingRow) { if (!trainingRow) {
return res.status(404).json({ error: `Training row not found for id or project_details_id ${training_id}` }); return res.status(404).json({ error: `Training row not found for id or project_details_id ${training_id}` });
} }
const project_details_id = trainingRow.project_details_id; const project_details_id = trainingRow.project_details_id;
// Use the generated exp.py from the correct project folder // Use the generated exp.py from the correct project folder
const outDir = path.join(__dirname, '..', projectName, String(project_details_id)); const outDir = path.join(__dirname, '..', projectName, String(project_details_id));
const yoloxMainDir = '/home/kitraining/Yolox/YOLOX-main'; const yoloxMainDir = '/home/kitraining/Yolox/YOLOX-main';
const expSrc = path.join(outDir, 'exp.py'); const expSrc = path.join(outDir, 'exp.py');
if (!fs.existsSync(expSrc)) { if (!fs.existsSync(expSrc)) {
return res.status(500).json({ error: `exp.py not found at ${expSrc}` }); return res.status(500).json({ error: `exp.py not found at ${expSrc}` });
} }
// Source venv and run YOLOX training in YOLOX-main folder // Source venv and run YOLOX training in YOLOX-main folder
const yoloxVenv = '/home/kitraining/Yolox/yolox_venv/bin/activate'; const yoloxVenv = '/home/kitraining/Yolox/yolox_venv/bin/activate';
// Determine model argument based on selected_model and transfer_learning // Determine model argument based on selected_model and transfer_learning
let modelArg = ''; let modelArg = '';
let cmd = ''; let cmd = '';
if ( if (
trainingRow.transfer_learning && trainingRow.transfer_learning &&
typeof trainingRow.transfer_learning === 'string' && typeof trainingRow.transfer_learning === 'string' &&
trainingRow.transfer_learning.toLowerCase() === 'coco' trainingRow.transfer_learning.toLowerCase() === 'coco'
) { ) {
// If transfer_learning is 'coco', add -o and modelArg // If transfer_learning is 'coco', add -o and modelArg
modelArg = ` -c /home/kitraining/Yolox/YOLOX-main/pretrained/${trainingRow.selected_model}`; modelArg = ` -c /home/kitraining/Yolox/YOLOX-main/pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`; cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else if ( } else if (
trainingRow.selected_model && trainingRow.selected_model &&
trainingRow.selected_model.toLowerCase() === 'coco' && trainingRow.selected_model.toLowerCase() === 'coco' &&
(!trainingRow.transfer_learning || trainingRow.transfer_learning === false) (!trainingRow.transfer_learning || trainingRow.transfer_learning === false)
) { ) {
// If selected_model is 'coco' and not transfer_learning, add modelArg only // If selected_model is 'coco' and not transfer_learning, add modelArg only
modelArg = ` -c /pretrained/${trainingRow.selected_model}`; modelArg = ` -c /pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`; cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else { } else {
// Default: no modelArg // Default: no modelArg
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16' --cache`; cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16' --cache`;
} }
console.log(cmd) console.log(cmd)
const child = spawn(cmd, { shell: true, cwd: yoloxMainDir }); const child = spawn(cmd, { shell: true, cwd: yoloxMainDir });
child.stdout.pipe(process.stdout); child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr); child.stderr.pipe(process.stderr);
res.json({ message: 'Training started' }); res.json({ message: 'Training started' });
} catch (err) { } catch (err) {
res.status(500).json({ error: 'Failed to start training', details: err.message }); res.status(500).json({ error: 'Failed to start training', details: err.message });
} }
}); });
// Get YOLOX training log // Get YOLOX training log
router.get('/training-log', async (req, res) => { router.get('/training-log', async (req, res) => {
try { try {
const { project_id, training_id } = req.query; const { project_id, training_id } = req.query;
const trainingProject = await TrainingProject.findByPk(project_id); const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`; const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
const outDir = path.join(__dirname, '..', projectName, String(training_id)); const outDir = path.join(__dirname, '..', projectName, String(training_id));
const logPath = path.join(outDir, 'training.log'); const logPath = path.join(outDir, 'training.log');
if (!fs.existsSync(logPath)) { if (!fs.existsSync(logPath)) {
return res.status(404).json({ error: 'Log not found' }); return res.status(404).json({ error: 'Log not found' });
} }
const logData = fs.readFileSync(logPath, 'utf8'); const logData = fs.readFileSync(logPath, 'utf8');
res.json({ log: logData }); res.json({ log: logData });
} catch (err) { } catch (err) {
res.status(500).json({ error: 'Failed to fetch log', details: err.message }); res.status(500).json({ error: 'Failed to fetch log', details: err.message });
} }
}); });
router.post('/training-projects', upload.single('project_image'), async (req, res) => { router.post('/training-projects', upload.single('project_image'), async (req, res) => {
try { try {
const { title, description } = req.body; const { title, description } = req.body;
const classes = JSON.parse(req.body.classes); const classes = JSON.parse(req.body.classes);
const project_image = req.file ? req.file.buffer : null; const project_image = req.file ? req.file.buffer : null;
const project_image_type = req.file ? req.file.mimetype : null; const project_image_type = req.file ? req.file.mimetype : null;
await TrainingProject.create({ await TrainingProject.create({
title, title,
description, description,
classes, classes,
project_image, project_image,
project_image_type project_image_type
}); });
res.json({ message: 'Project created!' }); res.json({ message: 'Project created!' });
} catch (error) { } catch (error) {
console.error('Error creating project:', error); console.error('Error creating project:', error);
res.status(500).json({ message: 'Failed to create project', error: error.message }); res.status(500).json({ message: 'Failed to create project', error: error.message });
} }
}); });
router.get('/training-projects', async (req, res) => { router.get('/training-projects', async (req, res) => {
try { try {
const projects = await TrainingProject.findAll(); const projects = await TrainingProject.findAll();
// Convert BLOB to base64 data URL for each project // Convert BLOB to base64 data URL for each project
const serialized = projects.map(project => { const serialized = projects.map(project => {
const plain = project.get({ plain: true }); const plain = project.get({ plain: true });
if (plain.project_image) { if (plain.project_image) {
const base64 = Buffer.from(plain.project_image).toString('base64'); const base64 = Buffer.from(plain.project_image).toString('base64');
const mimeType = plain.project_image_type || 'image/png'; const mimeType = plain.project_image_type || 'image/png';
plain.project_image = `data:${mimeType};base64,${base64}`; plain.project_image = `data:${mimeType};base64,${base64}`;
} }
return plain; return plain;
}); });
res.json(serialized); res.json(serialized);
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message }); res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
} }
}); });
router.get('/update-status', async (req, res) => { router.get('/update-status', async (req, res) => {
res.json(updateStatus) res.json(updateStatus)
}) })
router.get('/label-studio-projects', async (req, res) => { router.get('/label-studio-projects', async (req, res) => {
try { try {
const LabelStudioProject = require('../models/LabelStudioProject.js'); const LabelStudioProject = require('../models/LabelStudioProject.js');
const Image = require('../models/Images.js'); const Image = require('../models/Images.js');
const Annotation = require('../models/Annotation.js'); const Annotation = require('../models/Annotation.js');
const labelStudioProjects = await LabelStudioProject.findAll(); const labelStudioProjects = await LabelStudioProject.findAll();
const projectsWithCounts = await Promise.all(labelStudioProjects.map(async project => { const projectsWithCounts = await Promise.all(labelStudioProjects.map(async project => {
const plain = project.get({ plain: true }); const plain = project.get({ plain: true });
// Get all images for this project // Get all images for this project
const images = await Image.findAll({ where: { project_id: plain.project_id } }); const images = await Image.findAll({ where: { project_id: plain.project_id } });
let annotationCounts = {}; let annotationCounts = {};
if (images.length > 0) { if (images.length > 0) {
const imageIds = images.map(img => img.image_id); const imageIds = images.map(img => img.image_id);
// Get all annotations for these images // Get all annotations for these images
const annotations = await Annotation.findAll({ where: { image_id: imageIds } }); const annotations = await Annotation.findAll({ where: { image_id: imageIds } });
// Count by label // Count by label
for (const ann of annotations) { for (const ann of annotations) {
const label = ann.Label; const label = ann.Label;
annotationCounts[label] = (annotationCounts[label] || 0) + 1; annotationCounts[label] = (annotationCounts[label] || 0) + 1;
} }
} }
plain.annotationCounts = annotationCounts; plain.annotationCounts = annotationCounts;
return plain; return plain;
})); }));
res.json(projectsWithCounts); res.json(projectsWithCounts);
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message }); res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
} }
}) })
// POST endpoint to create TrainingProjectDetails with all fields // POST endpoint to create TrainingProjectDetails with all fields
router.post('/training-project-details', async (req, res) => { router.post('/training-project-details', async (req, res) => {
try { try {
const { const {
project_id, project_id,
annotation_projects, annotation_projects,
class_map, class_map,
description description
} = req.body; } = req.body;
if (!project_id || !annotation_projects) { if (!project_id || !annotation_projects) {
return res.status(400).json({ message: 'Missing required fields' }); return res.status(400).json({ message: 'Missing required fields' });
} }
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const created = await TrainingProjectDetails.create({ const created = await TrainingProjectDetails.create({
project_id, project_id,
annotation_projects, annotation_projects,
class_map: class_map || null, class_map: class_map || null,
description: description || null description: description || null
}); });
res.json({ message: 'TrainingProjectDetails created', details: created }); res.json({ message: 'TrainingProjectDetails created', details: created });
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to create TrainingProjectDetails', error: error.message }); res.status(500).json({ message: 'Failed to create TrainingProjectDetails', error: error.message });
} }
}); });
// GET endpoint to fetch all TrainingProjectDetails // GET endpoint to fetch all TrainingProjectDetails
router.get('/training-project-details', async (req, res) => { router.get('/training-project-details', async (req, res) => {
try { try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findAll(); const details = await TrainingProjectDetails.findAll();
res.json(details); res.json(details);
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to fetch TrainingProjectDetails', error: error.message }); res.status(500).json({ message: 'Failed to fetch TrainingProjectDetails', error: error.message });
} }
}); });
// PUT endpoint to update class_map and description in TrainingProjectDetails // PUT endpoint to update class_map and description in TrainingProjectDetails
router.put('/training-project-details', async (req, res) => { router.put('/training-project-details', async (req, res) => {
try { try {
const { project_id, class_map, description } = req.body; const { project_id, class_map, description } = req.body;
if (!project_id || !class_map || !description) { if (!project_id || !class_map || !description) {
return res.status(400).json({ message: 'Missing required fields' }); return res.status(400).json({ message: 'Missing required fields' });
} }
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id } }); const details = await TrainingProjectDetails.findOne({ where: { project_id } });
if (!details) { if (!details) {
return res.status(404).json({ message: 'TrainingProjectDetails not found' }); return res.status(404).json({ message: 'TrainingProjectDetails not found' });
} }
details.class_map = class_map; details.class_map = class_map;
details.description = description; details.description = description;
await details.save(); await details.save();
res.json({ message: 'Class map and description updated', details }); res.json({ message: 'Class map and description updated', details });
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to update class map or description', error: error.message }); res.status(500).json({ message: 'Failed to update class map or description', error: error.message });
} }
}); });
// POST endpoint to receive YOLOX settings and save to DB (handles multipart/form-data) // POST endpoint to receive YOLOX settings and save to DB (handles multipart/form-data)
router.post('/yolox-settings', upload.any(), async (req, res) => { router.post('/yolox-settings', upload.any(), async (req, res) => {
try { try {
const settings = req.body; const settings = req.body;
// Debug: Log all received fields and types // Debug: Log all received fields and types
console.log('--- YOLOX settings received ---'); console.log('--- YOLOX settings received ---');
console.log('settings:', settings); console.log('settings:', settings);
if (req.files && req.files.length > 0) { if (req.files && req.files.length > 0) {
console.log('Files received:', req.files.map(f => ({ fieldname: f.fieldname, originalname: f.originalname, size: f.size }))); console.log('Files received:', req.files.map(f => ({ fieldname: f.fieldname, originalname: f.originalname, size: f.size })));
} }
// Declare requiredFields once // Declare requiredFields once
const requiredFields = ['project_details_id', 'exp_name', 'max_epoch', 'depth', 'width', 'activation', 'train', 'valid', 'test', 'selected_model', 'transfer_learning']; const requiredFields = ['project_details_id', 'exp_name', 'max_epoch', 'depth', 'width', 'activation', 'train', 'valid', 'test', 'selected_model', 'transfer_learning'];
// Log types of required fields // Log types of required fields
requiredFields.forEach(field => { requiredFields.forEach(field => {
console.log(`Field '${field}': value='${settings[field]}', type='${typeof settings[field]}'`); console.log(`Field '${field}': value='${settings[field]}', type='${typeof settings[field]}'`);
}); });
// Map select_model to selected_model if present // Map select_model to selected_model if present
if (settings && settings.select_model && !settings.selected_model) { if (settings && settings.select_model && !settings.selected_model) {
settings.selected_model = settings.select_model; settings.selected_model = settings.select_model;
delete settings.select_model; delete settings.select_model;
} }
// Lookup project_details_id from project_id // Lookup project_details_id from project_id
if (!settings.project_id || isNaN(Number(settings.project_id))) { if (!settings.project_id || isNaN(Number(settings.project_id))) {
throw new Error('Missing or invalid project_id in request. Cannot assign training to a project.'); throw new Error('Missing or invalid project_id in request. Cannot assign training to a project.');
} }
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
let details = await TrainingProjectDetails.findOne({ where: { project_id: settings.project_id } }); let details = await TrainingProjectDetails.findOne({ where: { project_id: settings.project_id } });
if (!details) { if (!details) {
details = await TrainingProjectDetails.create({ details = await TrainingProjectDetails.create({
project_id: settings.project_id, project_id: settings.project_id,
annotation_projects: [], annotation_projects: [],
class_map: null, class_map: null,
description: null description: null
}); });
} }
settings.project_details_id = details.id; settings.project_details_id = details.id;
// Map 'act' from frontend to 'activation' for DB // Map 'act' from frontend to 'activation' for DB
if (settings.act !== undefined) { if (settings.act !== undefined) {
settings.activation = settings.act; settings.activation = settings.act;
delete settings.act; delete settings.act;
} }
// Type conversion for DB compatibility // Type conversion for DB compatibility
[ [
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr', 'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum', 'print_interval', 'eval_interval', 'test_conf', 'nmsthre', 'multiscale_range', 'degrees', 'translate', 'shear', 'train', 'valid', 'test' 'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr', 'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum', 'print_interval', 'eval_interval', 'test_conf', 'nmsthre', 'multiscale_range', 'degrees', 'translate', 'shear', 'train', 'valid', 'test'
].forEach(f => { ].forEach(f => {
if (settings[f] !== undefined) settings[f] = Number(settings[f]); if (settings[f] !== undefined) settings[f] = Number(settings[f]);
}); });
// Improved boolean conversion // Improved boolean conversion
['ema', 'enable_mixup', 'save_history_ckpt'].forEach(f => { ['ema', 'enable_mixup', 'save_history_ckpt'].forEach(f => {
if (settings[f] !== undefined) { if (settings[f] !== undefined) {
if (typeof settings[f] === 'string') { if (typeof settings[f] === 'string') {
settings[f] = settings[f].toLowerCase() === 'true'; settings[f] = settings[f].toLowerCase() === 'true';
} else { } else {
settings[f] = Boolean(settings[f]); settings[f] = Boolean(settings[f]);
} }
} }
}); });
// Improved array conversion // Improved array conversion
['mosaic_scale', 'mixup_scale', 'scale'].forEach(f => { ['mosaic_scale', 'mixup_scale', 'scale'].forEach(f => {
if (settings[f] && typeof settings[f] === 'string') { if (settings[f] && typeof settings[f] === 'string') {
settings[f] = settings[f] settings[f] = settings[f]
.split(',') .split(',')
.map(s => Number(s.trim())) .map(s => Number(s.trim()))
.filter(n => !isNaN(n)); .filter(n => !isNaN(n));
} }
}); });
// Trim all string fields // Trim all string fields
Object.keys(settings).forEach(f => { Object.keys(settings).forEach(f => {
if (typeof settings[f] === 'string') settings[f] = settings[f].trim(); if (typeof settings[f] === 'string') settings[f] = settings[f].trim();
}); });
// Set default for transfer_learning if missing // Set default for transfer_learning if missing
if (settings.transfer_learning === undefined) settings.transfer_learning = false; if (settings.transfer_learning === undefined) settings.transfer_learning = false;
// Convert empty string seed to null // Convert empty string seed to null
if ('seed' in settings && (settings.seed === '' || settings.seed === undefined)) { if ('seed' in settings && (settings.seed === '' || settings.seed === undefined)) {
settings.seed = null; settings.seed = null;
} }
// Validate required fields for training table // Validate required fields for training table
for (const field of requiredFields) { for (const field of requiredFields) {
if (settings[field] === undefined || settings[field] === null || settings[field] === '') { if (settings[field] === undefined || settings[field] === null || settings[field] === '') {
console.error('Missing required field:', field, 'Value:', settings[field]); console.error('Missing required field:', field, 'Value:', settings[field]);
throw new Error('Missing required field: ' + field); throw new Error('Missing required field: ' + field);
} }
} }
console.log('Received YOLOX settings:', settings); console.log('Received YOLOX settings:', settings);
// Handle uploaded model file (ckpt_upload) // Handle uploaded model file (ckpt_upload)
if (req.files && req.files.length > 0) { if (req.files && req.files.length > 0) {
const ckptFile = req.files.find(f => f.fieldname === 'ckpt_upload'); const ckptFile = req.files.find(f => f.fieldname === 'ckpt_upload');
if (ckptFile) { if (ckptFile) {
const uploadDir = path.join(__dirname, '..', 'uploads'); const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir); if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = ckptFile.originalname || `uploaded_model_${settings.project_id}.pth`; const filename = ckptFile.originalname || `uploaded_model_${settings.project_id}.pth`;
const filePath = path.join(uploadDir, filename); const filePath = path.join(uploadDir, filename);
fs.writeFileSync(filePath, ckptFile.buffer); fs.writeFileSync(filePath, ckptFile.buffer);
settings.model_upload = filePath; settings.model_upload = filePath;
} }
} }
// Save settings to DB only (no file) // Save settings to DB only (no file)
const { pushYoloxExpToDb } = require('../services/push-yolox-exp.js'); const { pushYoloxExpToDb } = require('../services/push-yolox-exp.js');
const training = await pushYoloxExpToDb(settings); const training = await pushYoloxExpToDb(settings);
res.json({ message: 'YOLOX settings saved to DB', training }); res.json({ message: 'YOLOX settings saved to DB', training });
} catch (error) { } catch (error) {
console.error('Error in /api/yolox-settings:', error.stack || error); console.error('Error in /api/yolox-settings:', error.stack || error);
res.status(500).json({ message: 'Failed to save YOLOX settings', error: error.message }); res.status(500).json({ message: 'Failed to save YOLOX settings', error: error.message });
} }
}); });
// POST endpoint to receive binary model file and save to disk (not DB) // POST endpoint to receive binary model file and save to disk (not DB)
router.post('/yolox-settings/upload', async (req, res) => { router.post('/yolox-settings/upload', async (req, res) => {
try { try {
const projectId = req.query.project_id; const projectId = req.query.project_id;
if (!projectId) return res.status(400).json({ message: 'Missing project_id in query' }); if (!projectId) return res.status(400).json({ message: 'Missing project_id in query' });
// Save file to disk // Save file to disk
const uploadDir = path.join(__dirname, '..', 'uploads'); const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir); if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = req.headers['x-upload-filename'] || `uploaded_model_${projectId}.pth`; const filename = req.headers['x-upload-filename'] || `uploaded_model_${projectId}.pth`;
const filePath = path.join(uploadDir, filename); const filePath = path.join(uploadDir, filename);
const chunks = []; const chunks = [];
req.on('data', chunk => chunks.push(chunk)); req.on('data', chunk => chunks.push(chunk));
req.on('end', async () => { req.on('end', async () => {
const buffer = Buffer.concat(chunks); const buffer = Buffer.concat(chunks);
fs.writeFile(filePath, buffer, async err => { fs.writeFile(filePath, buffer, async err => {
if (err) { if (err) {
console.error('Error saving file:', err); console.error('Error saving file:', err);
return res.status(500).json({ message: 'Failed to save model file', error: err.message }); return res.status(500).json({ message: 'Failed to save model file', error: err.message });
} }
// Update latest training row for this project with file path // Update latest training row for this project with file path
try { try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js'); const Training = require('../models/training.js');
// Find details row for this project // Find details row for this project
const details = await TrainingProjectDetails.findOne({ where: { project_id: projectId } }); const details = await TrainingProjectDetails.findOne({ where: { project_id: projectId } });
if (!details) return res.status(404).json({ message: 'No TrainingProjectDetails found for project_id' }); if (!details) return res.status(404).json({ message: 'No TrainingProjectDetails found for project_id' });
// Find latest training for this details row // Find latest training for this details row
const training = await Training.findOne({ where: { project_details_id: details.id }, order: [['createdAt', 'DESC']] }); const training = await Training.findOne({ where: { project_details_id: details.id }, order: [['createdAt', 'DESC']] });
if (!training) return res.status(404).json({ message: 'No training found for project_id' }); if (!training) return res.status(404).json({ message: 'No training found for project_id' });
// Save file path to model_upload field // Save file path to model_upload field
training.model_upload = filePath; training.model_upload = filePath;
await training.save(); await training.save();
res.json({ message: 'Model file uploaded and saved to disk', filename, trainingId: training.id }); res.json({ message: 'Model file uploaded and saved to disk', filename, trainingId: training.id });
} catch (dbErr) { } catch (dbErr) {
console.error('Error updating training with file path:', dbErr); console.error('Error updating training with file path:', dbErr);
res.status(500).json({ message: 'File saved but failed to update training row', error: dbErr.message }); res.status(500).json({ message: 'File saved but failed to update training row', error: dbErr.message });
} }
}); });
}); });
} catch (error) { } catch (error) {
console.error('Error in /api/yolox-settings/upload:', error.stack || error); console.error('Error in /api/yolox-settings/upload:', error.stack || error);
res.status(500).json({ message: 'Failed to upload model file', error: error.message }); res.status(500).json({ message: 'Failed to upload model file', error: error.message });
} }
}); });
// GET endpoint to fetch all trainings (optionally filtered by project_id) // GET endpoint to fetch all trainings (optionally filtered by project_id)
router.get('/trainings', async (req, res) => { router.get('/trainings', async (req, res) => {
try { try {
const project_id = req.query.project_id; const project_id = req.query.project_id;
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js'); const Training = require('../models/training.js');
if (project_id) { if (project_id) {
// Find all details rows for this project // Find all details rows for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } }); const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) return res.json([]); if (!detailsRows || detailsRows.length === 0) return res.json([]);
// Get all trainings linked to any details row for this project // Get all trainings linked to any details row for this project
const detailsIds = detailsRows.map(d => d.id); const detailsIds = detailsRows.map(d => d.id);
const trainings = await Training.findAll({ where: { project_details_id: detailsIds } }); const trainings = await Training.findAll({ where: { project_details_id: detailsIds } });
return res.json(trainings); return res.json(trainings);
} else { } else {
// Return all trainings if no project_id is specified // Return all trainings if no project_id is specified
const trainings = await Training.findAll(); const trainings = await Training.findAll();
return res.json(trainings); return res.json(trainings);
} }
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to fetch trainings', error: error.message }); res.status(500).json({ message: 'Failed to fetch trainings', error: error.message });
} }
}); });
// DELETE endpoint to remove a training by id // DELETE endpoint to remove a training by id
router.delete('/trainings/:id', async (req, res) => { router.delete('/trainings/:id', async (req, res) => {
try { try {
const Training = require('../models/training.js'); const Training = require('../models/training.js');
const id = req.params.id; const id = req.params.id;
const deleted = await Training.destroy({ where: { id } }); const deleted = await Training.destroy({ where: { id } });
if (deleted) { if (deleted) {
res.json({ message: 'Training deleted' }); res.json({ message: 'Training deleted' });
} else { } else {
res.status(404).json({ message: 'Training not found' }); res.status(404).json({ message: 'Training not found' });
} }
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to delete training', error: error.message }); res.status(500).json({ message: 'Failed to delete training', error: error.message });
} }
}); });
// DELETE endpoint to remove a training project and all related entries // DELETE endpoint to remove a training project and all related entries
router.delete('/training-projects/:id', async (req, res) => { router.delete('/training-projects/:id', async (req, res) => {
try { try {
const projectId = req.params.id; const projectId = req.params.id;
const TrainingProject = require('../models/TrainingProject.js'); const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js'); const Training = require('../models/training.js');
// Find details row(s) for this project // Find details row(s) for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id: projectId } }); const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id: projectId } });
const detailsIds = detailsRows.map(d => d.id); const detailsIds = detailsRows.map(d => d.id);
// Delete all trainings linked to these details // Delete all trainings linked to these details
if (detailsIds.length > 0) { if (detailsIds.length > 0) {
await Training.destroy({ where: { project_details_id: detailsIds } }); await Training.destroy({ where: { project_details_id: detailsIds } });
await TrainingProjectDetails.destroy({ where: { project_id: projectId } }); await TrainingProjectDetails.destroy({ where: { project_id: projectId } });
} }
// Delete the project itself // Delete the project itself
const deleted = await TrainingProject.destroy({ where: { project_id: projectId } }); const deleted = await TrainingProject.destroy({ where: { project_id: projectId } });
if (deleted) { if (deleted) {
res.json({ message: 'Training project and all related entries deleted' }); res.json({ message: 'Training project and all related entries deleted' });
} else { } else {
res.status(404).json({ message: 'Training project not found' }); res.status(404).json({ message: 'Training project not found' });
} }
} catch (error) { } catch (error) {
res.status(500).json({ message: 'Failed to delete training project', error: error.message }); res.status(500).json({ message: 'Failed to delete training project', error: error.message });
} }
}); });
module.exports = router; module.exports = router;

1398
backend/routes/api.py
View File

File diff suppressed because it is too large Load Diff

68
backend/server.js
View File

@@ -1,34 +1,34 @@
const express = require('express'); const express = require('express');
const cors = require('cors'); const cors = require('cors');
const path = require('path'); const path = require('path');
const sequelize = require('./database/database'); const sequelize = require('./database/database');
const app = express(); const app = express();
app.use(express.json()); app.use(express.json());
const port = 3000; const port = 3000;
const apiRouter = require('./routes/api.js'); const apiRouter = require('./routes/api.js');
app.use('/api', apiRouter); app.use('/api', apiRouter);
app.use(cors()); app.use(cors());
app.use(express.json()); app.use(express.json());
app.use(express.static(path.join(__dirname, '..'))); app.use(express.static(path.join(__dirname, '..')));
// Initialize DB and start server // Initialize DB and start server
(async () => { (async () => {
try { try {
await sequelize.authenticate(); await sequelize.authenticate();
console.log('DB connection established.'); console.log('DB connection established.');
await sequelize.sync(); // Only if you want Sequelize to ensure schema matches await sequelize.sync(); // Only if you want Sequelize to ensure schema matches
app.listen(port, '0.0.0.0', () => app.listen(port, '0.0.0.0', () =>
console.log(`Server running at http://0.0.0.0:${port}`) console.log(`Server running at http://0.0.0.0:${port}`)
); );
} catch (err) { } catch (err) {
console.error('Failed to start:', err); console.error('Failed to start:', err);
} }
})(); })();

2
backend/services/__init__.py
View File

@@ -1 +1 @@
# Services module # Services module

184
backend/services/fetch-labelstudio.js
View File

@@ -1,92 +1,92 @@
const API_URL = 'http://192.168.1.19:8080/api'; const API_URL = 'http://192.168.1.19:8080/api';
const API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f'; const API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f';
const fetch = require('node-fetch'); const fetch = require('node-fetch');
async function fetchLableStudioProject(projectid) { async function fetchLableStudioProject(projectid) {
// 1. Trigger export // 1. Trigger export
const exportUrl = `${API_URL}/projects/${projectid}/export?exportType=JSON_MIN`; const exportUrl = `${API_URL}/projects/${projectid}/export?exportType=JSON_MIN`;
const headers = { Authorization: `Token ${API_TOKEN}` }; const headers = { Authorization: `Token ${API_TOKEN}` };
let res = await fetch(exportUrl, { headers }); let res = await fetch(exportUrl, { headers });
if (!res.ok) { if (!res.ok) {
let errorText = await res.text().catch(() => ''); let errorText = await res.text().catch(() => '');
console.error(`Failed to trigger export: ${res.status} ${res.statusText} - ${errorText}`); console.error(`Failed to trigger export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to trigger export: ${res.status} ${res.statusText}`); throw new Error(`Failed to trigger export: ${res.status} ${res.statusText}`);
} }
let data = await res.json(); let data = await res.json();
// If data is an array, it's ready // If data is an array, it's ready
if (Array.isArray(data)) return data; if (Array.isArray(data)) return data;
// If not, poll for the export file // If not, poll for the export file
let fileUrl = data.download_url || data.url || null; let fileUrl = data.download_url || data.url || null;
let tries = 0; let tries = 0;
while (!fileUrl && tries < 20) { while (!fileUrl && tries < 20) {
await new Promise(r => setTimeout(r, 2000)); await new Promise(r => setTimeout(r, 2000));
res = await fetch(exportUrl, { headers }); res = await fetch(exportUrl, { headers });
if (!res.ok) { if (!res.ok) {
let errorText = await res.text().catch(() => ''); let errorText = await res.text().catch(() => '');
console.error(`Failed to poll export: ${res.status} ${res.statusText} - ${errorText}`); console.error(`Failed to poll export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to poll export: ${res.status} ${res.statusText}`); throw new Error(`Failed to poll export: ${res.status} ${res.statusText}`);
} }
data = await res.json(); data = await res.json();
fileUrl = data.download_url || data.url || null; fileUrl = data.download_url || data.url || null;
tries++; tries++;
} }
if (!fileUrl) throw new Error('Label Studio export did not become ready'); if (!fileUrl) throw new Error('Label Studio export did not become ready');
// 2. Download the export file // 2. Download the export file
res = await fetch(fileUrl.startsWith('http') ? fileUrl : `${API_URL.replace('/api','')}${fileUrl}`, { headers }); res = await fetch(fileUrl.startsWith('http') ? fileUrl : `${API_URL.replace('/api','')}${fileUrl}`, { headers });
if (!res.ok) { if (!res.ok) {
let errorText = await res.text().catch(() => ''); let errorText = await res.text().catch(() => '');
console.error(`Failed to download export: ${res.status} ${res.statusText} - ${errorText}`); console.error(`Failed to download export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to download export: ${res.status} ${res.statusText}`); throw new Error(`Failed to download export: ${res.status} ${res.statusText}`);
} }
return await res.json(); return await res.json();
} }
async function fetchProjectIdsAndTitles() { async function fetchProjectIdsAndTitles() {
try { try {
const response = await fetch(`${API_URL}/projects/`, { const response = await fetch(`${API_URL}/projects/`, {
headers: { headers: {
'Authorization': `Token ${API_TOKEN}`, 'Authorization': `Token ${API_TOKEN}`,
'Content-Type': 'application/json' 'Content-Type': 'application/json'
} }
}); });
if (!response.ok) { if (!response.ok) {
let errorText = await response.text().catch(() => ''); let errorText = await response.text().catch(() => '');
console.error(`Failed to fetch projects: ${response.status} ${response.statusText} - ${errorText}`); console.error(`Failed to fetch projects: ${response.status} ${response.statusText} - ${errorText}`);
throw new Error(`HTTP error! status: ${response.status}`); throw new Error(`HTTP error! status: ${response.status}`);
} }
const data = await response.json(); const data = await response.json();
if (!data.results || !Array.isArray(data.results)) { if (!data.results || !Array.isArray(data.results)) {
throw new Error('API response does not contain results array'); throw new Error('API response does not contain results array');
} }
// Extract id and title from each project // Extract id and title from each project
const projects = data.results.map(project => ({ const projects = data.results.map(project => ({
id: project.id, id: project.id,
title: project.title title: project.title
})); }));
console.log(projects) console.log(projects)
return projects; return projects;
} catch (error) { } catch (error) {
console.error('Failed to fetch projects:', error); console.error('Failed to fetch projects:', error);
return []; return [];
} }
} }
module.exports = { fetchLableStudioProject, fetchProjectIdsAndTitles }; module.exports = { fetchLableStudioProject, fetchProjectIdsAndTitles };
//getLableStudioProject(20) //getLableStudioProject(20)
//fetchProjectIdsAndTitles() //fetchProjectIdsAndTitles()

178
backend/services/fetch_labelstudio.py
View File

@@ -1,85 +1,93 @@
import requests import requests
import time import time
from services.settings_service import get_setting
API_URL = 'http://192.168.1.19:8080/api'
API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f' def get_api_credentials():
"""Get Label Studio API credentials from settings"""
def fetch_label_studio_project(project_id): api_url = get_setting('labelstudio_api_url', 'http://192.168.1.19:8080/api')
"""Fetch Label Studio project annotations""" api_token = get_setting('labelstudio_api_token', 'c1cef980b7c73004f4ee880a42839313b863869f')
export_url = f'{API_URL}/projects/{project_id}/export?exportType=JSON_MIN' return api_url, api_token
headers = {'Authorization': f'Token {API_TOKEN}'}
def fetch_label_studio_project(project_id):
# Trigger export """Fetch Label Studio project annotations"""
res = requests.get(export_url, headers=headers) API_URL, API_TOKEN = get_api_credentials()
if not res.ok:
error_text = res.text if res.text else '' export_url = f'{API_URL}/projects/{project_id}/export?exportType=JSON_MIN'
print(f'Failed to trigger export: {res.status_code} {res.reason} - {error_text}') headers = {'Authorization': f'Token {API_TOKEN}'}
raise Exception(f'Failed to trigger export: {res.status_code} {res.reason}')
# Trigger export
data = res.json() res = requests.get(export_url, headers=headers)
if not res.ok:
# If data is an array, it's ready error_text = res.text if res.text else ''
if isinstance(data, list): print(f'Failed to trigger export: {res.status_code} {res.reason} - {error_text}')
return data raise Exception(f'Failed to trigger export: {res.status_code} {res.reason}')
# If not, poll for the export file data = res.json()
file_url = data.get('download_url') or data.get('url')
tries = 0 # If data is an array, it's ready
if isinstance(data, list):
while not file_url and tries < 20: return data
time.sleep(2)
res = requests.get(export_url, headers=headers) # If not, poll for the export file
if not res.ok: file_url = data.get('download_url') or data.get('url')
error_text = res.text if res.text else '' tries = 0
print(f'Failed to poll export: {res.status_code} {res.reason} - {error_text}')
raise Exception(f'Failed to poll export: {res.status_code} {res.reason}') while not file_url and tries < 20:
time.sleep(2)
data = res.json() res = requests.get(export_url, headers=headers)
file_url = data.get('download_url') or data.get('url') if not res.ok:
tries += 1 error_text = res.text if res.text else ''
print(f'Failed to poll export: {res.status_code} {res.reason} - {error_text}')
if not file_url: raise Exception(f'Failed to poll export: {res.status_code} {res.reason}')
raise Exception('Label Studio export did not become ready')
data = res.json()
# Download the export file file_url = data.get('download_url') or data.get('url')
full_url = file_url if file_url.startswith('http') else f"{API_URL.replace('/api', '')}{file_url}" tries += 1
res = requests.get(full_url, headers=headers)
if not res.ok: if not file_url:
error_text = res.text if res.text else '' raise Exception('Label Studio export did not become ready')
print(f'Failed to download export: {res.status_code} {res.reason} - {error_text}')
raise Exception(f'Failed to download export: {res.status_code} {res.reason}') # Download the export file
full_url = file_url if file_url.startswith('http') else f"{API_URL.replace('/api', '')}{file_url}"
return res.json() res = requests.get(full_url, headers=headers)
if not res.ok:
def fetch_project_ids_and_titles(): error_text = res.text if res.text else ''
"""Fetch all Label Studio project IDs and titles""" print(f'Failed to download export: {res.status_code} {res.reason} - {error_text}')
try: raise Exception(f'Failed to download export: {res.status_code} {res.reason}')
response = requests.get(
f'{API_URL}/projects/', return res.json()
headers={
'Authorization': f'Token {API_TOKEN}', def fetch_project_ids_and_titles():
'Content-Type': 'application/json' """Fetch all Label Studio project IDs and titles"""
} API_URL, API_TOKEN = get_api_credentials()
)
try:
if not response.ok: response = requests.get(
error_text = response.text if response.text else '' f'{API_URL}/projects/',
print(f'Failed to fetch projects: {response.status_code} {response.reason} - {error_text}') headers={
raise Exception(f'HTTP error! status: {response.status_code}') 'Authorization': f'Token {API_TOKEN}',
'Content-Type': 'application/json'
data = response.json() }
)
if 'results' not in data or not isinstance(data['results'], list):
raise Exception('API response does not contain results array') if not response.ok:
error_text = response.text if response.text else ''
# Extract id and title from each project print(f'Failed to fetch projects: {response.status_code} {response.reason} - {error_text}')
projects = [ raise Exception(f'HTTP error! status: {response.status_code}')
{'id': project['id'], 'title': project['title']}
for project in data['results'] data = response.json()
]
print(projects) if 'results' not in data or not isinstance(data['results'], list):
return projects raise Exception('API response does not contain results array')
except Exception as error: # Extract id and title from each project
print(f'Failed to fetch projects: {error}') projects = [
return [] {'id': project['id'], 'title': project['title']}
for project in data['results']
]
print(projects)
return projects
except Exception as error:
print(f'Failed to fetch projects: {error}')
return []

350
backend/services/generate-json-yolox.js
View File

@@ -1,176 +1,176 @@
const TrainingProject = require('../models/TrainingProject.js'); const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js') const TrainingProjectDetails = require('../models/TrainingProjectDetails.js')
const LabelStudioProject = require('../models/LabelStudioProject.js') const LabelStudioProject = require('../models/LabelStudioProject.js')
const Annotation = require('../models/Annotation.js') const Annotation = require('../models/Annotation.js')
const Images = require('../models/Images.js') const Images = require('../models/Images.js')
const fs = require('fs'); const fs = require('fs');
async function generateTrainingJson(trainingId){ async function generateTrainingJson(trainingId){
// trainingId is now project_details_id // trainingId is now project_details_id
const trainingProjectDetails = await TrainingProjectDetails.findByPk(trainingId); const trainingProjectDetails = await TrainingProjectDetails.findByPk(trainingId);
if (!trainingProjectDetails) throw new Error('No TrainingProjectDetails found for project_details_id ' + trainingId); if (!trainingProjectDetails) throw new Error('No TrainingProjectDetails found for project_details_id ' + trainingId);
const detailsObj = trainingProjectDetails.get({ plain: true }); const detailsObj = trainingProjectDetails.get({ plain: true });
// Get parent project for name // Get parent project for name
const trainingProject = await TrainingProject.findByPk(detailsObj.project_id); const trainingProject = await TrainingProject.findByPk(detailsObj.project_id);
// Get split percentages (assume they are stored as train_percent, valid_percent, test_percent) // Get split percentages (assume they are stored as train_percent, valid_percent, test_percent)
const trainPercent = detailsObj.train_percent || 85; const trainPercent = detailsObj.train_percent || 85;
const validPercent = detailsObj.valid_percent || 10; const validPercent = detailsObj.valid_percent || 10;
const testPercent = detailsObj.test_percent || 5; const testPercent = detailsObj.test_percent || 5;
let cocoImages = []; let cocoImages = [];
let cocoAnnotations = []; let cocoAnnotations = [];
let cocoCategories = []; let cocoCategories = [];
let categoryMap = {}; let categoryMap = {};
let categoryId = 0; let categoryId = 0;
let imageid = 0; let imageid = 0;
let annotationid = 0; let annotationid = 0;
for (const cls of detailsObj.class_map) { for (const cls of detailsObj.class_map) {
const asgMap = []; const asgMap = [];
const listAsg = cls[1]; const listAsg = cls[1];
for(const asg of listAsg){ for(const asg of listAsg){
asgMap.push ({ original: asg[0], mapped: asg[1] }); asgMap.push ({ original: asg[0], mapped: asg[1] });
// Build category list and mapping // Build category list and mapping
if (asg[1] && !(asg[1] in categoryMap)) { if (asg[1] && !(asg[1] in categoryMap)) {
categoryMap[asg[1]] = categoryId; categoryMap[asg[1]] = categoryId;
cocoCategories.push({ id: categoryId, name: asg[1], supercategory: '' }); cocoCategories.push({ id: categoryId, name: asg[1], supercategory: '' });
categoryId++; categoryId++;
} }
} }
const images = await Images.findAll({ where: { project_id: cls[0] } }); const images = await Images.findAll({ where: { project_id: cls[0] } });
for(const image of images){ for(const image of images){
imageid += 1; imageid += 1;
let fileName = image.image_path; let fileName = image.image_path;
if (fileName.includes('%20')) { if (fileName.includes('%20')) {
fileName = fileName.replace(/%20/g, ' '); fileName = fileName.replace(/%20/g, ' ');
} }
if (fileName && fileName.startsWith('/data/local-files/?d=')) { if (fileName && fileName.startsWith('/data/local-files/?d=')) {
fileName = fileName.replace('/data/local-files/?d=', ''); fileName = fileName.replace('/data/local-files/?d=', '');
fileName = fileName.replace('/home/kitraining/home/kitraining/', ''); fileName = fileName.replace('/home/kitraining/home/kitraining/', '');
} }
if (fileName && fileName.startsWith('home/kitraining/To_Annotate/')) { if (fileName && fileName.startsWith('home/kitraining/To_Annotate/')) {
fileName = fileName.replace('home/kitraining/To_Annotate/',''); fileName = fileName.replace('home/kitraining/To_Annotate/','');
} }
// Get annotations for this image // Get annotations for this image
const annotations = await Annotation.findAll({ where: { image_id: image.image_id } }); const annotations = await Annotation.findAll({ where: { image_id: image.image_id } });
// Use image.width and image.height from DB (populated from original_width/original_height) // Use image.width and image.height from DB (populated from original_width/original_height)
cocoImages.push({ cocoImages.push({
id: imageid, id: imageid,
file_name: fileName, file_name: fileName,
width: image.width || 0, width: image.width || 0,
height: image.height || 0 height: image.height || 0
}); });
for (const annotation of annotations) { for (const annotation of annotations) {
// Translate class name using asgMap // Translate class name using asgMap
let mappedClass = annotation.Label; let mappedClass = annotation.Label;
for (const mapEntry of asgMap) { for (const mapEntry of asgMap) {
if (annotation.Label === mapEntry.original) { if (annotation.Label === mapEntry.original) {
mappedClass = mapEntry.mapped; mappedClass = mapEntry.mapped;
break; break;
} }
} }
// Only add annotation if mappedClass is valid // Only add annotation if mappedClass is valid
if (mappedClass && mappedClass in categoryMap) { if (mappedClass && mappedClass in categoryMap) {
annotationid += 1; annotationid += 1;
let area = 0; let area = 0;
if (annotation.width && annotation.height) { if (annotation.width && annotation.height) {
area = annotation.width * annotation.height; area = annotation.width * annotation.height;
} }
cocoAnnotations.push({ cocoAnnotations.push({
id: annotationid, id: annotationid,
image_id: imageid, image_id: imageid,
category_id: categoryMap[mappedClass], category_id: categoryMap[mappedClass],
bbox: [annotation.x, annotation.y, annotation.width, annotation.height], bbox: [annotation.x, annotation.y, annotation.width, annotation.height],
area: area, area: area,
iscrowd: annotation.iscrowd || 0 iscrowd: annotation.iscrowd || 0
}); });
} }
} }
} }
} }
// Shuffle images for random split using seed // Shuffle images for random split using seed
function seededRandom(seed) { function seededRandom(seed) {
let x = Math.sin(seed++) * 10000; let x = Math.sin(seed++) * 10000;
return x - Math.floor(x); return x - Math.floor(x);
} }
function shuffle(array, seed) { function shuffle(array, seed) {
for (let i = array.length - 1; i > 0; i--) { for (let i = array.length - 1; i > 0; i--) {
const j = Math.floor(seededRandom(seed + i) * (i + 1)); const j = Math.floor(seededRandom(seed + i) * (i + 1));
[array[i], array[j]] = [array[j], array[i]]; [array[i], array[j]] = [array[j], array[i]];
} }
} }
// Use seed from detailsObj if present, else default to 42 // Use seed from detailsObj if present, else default to 42
const splitSeed = detailsObj.seed !== undefined && detailsObj.seed !== null ? Number(detailsObj.seed) : 42; const splitSeed = detailsObj.seed !== undefined && detailsObj.seed !== null ? Number(detailsObj.seed) : 42;
shuffle(cocoImages, splitSeed); shuffle(cocoImages, splitSeed);
// Split images // Split images
const totalImages = cocoImages.length; const totalImages = cocoImages.length;
const trainCount = Math.floor(totalImages * trainPercent / 100); const trainCount = Math.floor(totalImages * trainPercent / 100);
const validCount = Math.floor(totalImages * validPercent / 100); const validCount = Math.floor(totalImages * validPercent / 100);
const testCount = totalImages - trainCount - validCount; const testCount = totalImages - trainCount - validCount;
const trainImages = cocoImages.slice(0, trainCount); const trainImages = cocoImages.slice(0, trainCount);
const validImages = cocoImages.slice(trainCount, trainCount + validCount); const validImages = cocoImages.slice(trainCount, trainCount + validCount);
const testImages = cocoImages.slice(trainCount + validCount); const testImages = cocoImages.slice(trainCount + validCount);
// Helper to get image ids for each split // Helper to get image ids for each split
const trainImageIds = new Set(trainImages.map(img => img.id)); const trainImageIds = new Set(trainImages.map(img => img.id));
const validImageIds = new Set(validImages.map(img => img.id)); const validImageIds = new Set(validImages.map(img => img.id));
const testImageIds = new Set(testImages.map(img => img.id)); const testImageIds = new Set(testImages.map(img => img.id));
// Split annotations // Split annotations
const trainAnnotations = cocoAnnotations.filter(ann => trainImageIds.has(ann.image_id)); const trainAnnotations = cocoAnnotations.filter(ann => trainImageIds.has(ann.image_id));
const validAnnotations = cocoAnnotations.filter(ann => validImageIds.has(ann.image_id)); const validAnnotations = cocoAnnotations.filter(ann => validImageIds.has(ann.image_id));
const testAnnotations = cocoAnnotations.filter(ann => testImageIds.has(ann.image_id)); const testAnnotations = cocoAnnotations.filter(ann => testImageIds.has(ann.image_id));
// Build final COCO JSONs with info section // Build final COCO JSONs with info section
const buildCocoJson = (images, annotations, categories) => ({ const buildCocoJson = (images, annotations, categories) => ({
images, images,
annotations, annotations,
categories categories
}); });
// Build COCO JSONs with info section // Build COCO JSONs with info section
const trainJson = buildCocoJson(trainImages, trainAnnotations, cocoCategories); const trainJson = buildCocoJson(trainImages, trainAnnotations, cocoCategories);
const validJson = buildCocoJson(validImages, validAnnotations, cocoCategories); const validJson = buildCocoJson(validImages, validAnnotations, cocoCategories);
const testJson = buildCocoJson(testImages, testAnnotations, cocoCategories); const testJson = buildCocoJson(testImages, testAnnotations, cocoCategories);
// Create output directory: projectname/trainingid/annotations // Create output directory: projectname/trainingid/annotations
const projectName = trainingProject && trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${detailsObj.project_id}`; const projectName = trainingProject && trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${detailsObj.project_id}`;
const outDir = `${projectName}/${trainingId}`; const outDir = `${projectName}/${trainingId}`;
const annotationsDir = `/home/kitraining/To_Annotate/annotations`; const annotationsDir = `/home/kitraining/To_Annotate/annotations`;
if (!fs.existsSync(annotationsDir)) { if (!fs.existsSync(annotationsDir)) {
fs.mkdirSync(annotationsDir, { recursive: true }); fs.mkdirSync(annotationsDir, { recursive: true });
} }
// Write to files in the annotations directory // Write to files in the annotations directory
const trainPath = `${annotationsDir}/coco_project_${trainingId}_train.json`; const trainPath = `${annotationsDir}/coco_project_${trainingId}_train.json`;
const validPath = `${annotationsDir}/coco_project_${trainingId}_valid.json`; const validPath = `${annotationsDir}/coco_project_${trainingId}_valid.json`;
const testPath = `${annotationsDir}/coco_project_${trainingId}_test.json`; const testPath = `${annotationsDir}/coco_project_${trainingId}_test.json`;
fs.writeFileSync(trainPath, JSON.stringify(trainJson, null, 2)); fs.writeFileSync(trainPath, JSON.stringify(trainJson, null, 2));
fs.writeFileSync(validPath, JSON.stringify(validJson, null, 2)); fs.writeFileSync(validPath, JSON.stringify(validJson, null, 2));
fs.writeFileSync(testPath, JSON.stringify(testJson, null, 2)); fs.writeFileSync(testPath, JSON.stringify(testJson, null, 2));
console.log(`COCO JSON splits written to ${annotationsDir} for trainingId ${trainingId}`); console.log(`COCO JSON splits written to ${annotationsDir} for trainingId ${trainingId}`);
// Also generate inference exp.py in the same output directory as exp.py (project folder in workspace) // Also generate inference exp.py in the same output directory as exp.py (project folder in workspace)
const { generateYoloxInferenceExp } = require('./generate-yolox-exp'); const { generateYoloxInferenceExp } = require('./generate-yolox-exp');
const path = require('path'); const path = require('path');
const projectFolder = path.join(__dirname, '..', projectName, String(trainingId)); const projectFolder = path.join(__dirname, '..', projectName, String(trainingId));
if (!fs.existsSync(projectFolder)) { if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true }); fs.mkdirSync(projectFolder, { recursive: true });
} }
const inferenceExpPath = path.join(projectFolder, 'exp_infer.py'); const inferenceExpPath = path.join(projectFolder, 'exp_infer.py');
generateYoloxInferenceExp(trainingId).then(expContent => { generateYoloxInferenceExp(trainingId).then(expContent => {
fs.writeFileSync(inferenceExpPath, expContent); fs.writeFileSync(inferenceExpPath, expContent);
console.log(`Inference exp.py written to ${inferenceExpPath}`); console.log(`Inference exp.py written to ${inferenceExpPath}`);
}).catch(err => { }).catch(err => {
console.error('Failed to generate inference exp.py:', err); console.error('Failed to generate inference exp.py:', err);
}); });
} }
module.exports = {generateTrainingJson}; module.exports = {generateTrainingJson};

268
backend/services/generate-yolox-exp.js
View File

@@ -1,135 +1,135 @@
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const Training = require('../models/training.js'); const Training = require('../models/training.js');
const TrainingProject = require('../models/TrainingProject.js'); const TrainingProject = require('../models/TrainingProject.js');
// Remove Python comments and legacy code // Remove Python comments and legacy code
const exp_names = [ const exp_names = [
'YOLOX-s', 'YOLOX-s',
'YOLOX-m', 'YOLOX-m',
'YOLOX-l', 'YOLOX-l',
'YOLOX-x', 'YOLOX-x',
'YOLOX-Darknet53', //todo 'YOLOX-Darknet53', //todo
'YOLOX-Nano', 'YOLOX-Nano',
'YOLOX-Tiny' 'YOLOX-Tiny'
] ]
//TODO: Clean up generation of exp_names.py and remove second exp creation!!! //TODO: Clean up generation of exp_names.py and remove second exp creation!!!
// Refactored: Accept trainingId, fetch info from DB // Refactored: Accept trainingId, fetch info from DB
async function generateYoloxExp(trainingId) { async function generateYoloxExp(trainingId) {
// Fetch training row from DB by project_details_id if not found by PK // Fetch training row from DB by project_details_id if not found by PK
let training = await Training.findByPk(trainingId); let training = await Training.findByPk(trainingId);
if (!training) { if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } }); training = await Training.findOne({ where: { project_details_id: trainingId } });
} }
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId); if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// If transfer_learning is 'coco', just return the path to the default exp.py // If transfer_learning is 'coco', just return the path to the default exp.py
if (training.transfer_learning === 'coco') { if (training.transfer_learning === 'coco') {
const selectedModel = training.selected_model.toLowerCase().replace('-', '_'); const selectedModel = training.selected_model.toLowerCase().replace('-', '_');
const expSourcePath = `/home/kitraining/Yolox/YOLOX-main/exps/default/${selectedModel}.py`; const expSourcePath = `/home/kitraining/Yolox/YOLOX-main/exps/default/${selectedModel}.py`;
if (!fs.existsSync(expSourcePath)) { if (!fs.existsSync(expSourcePath)) {
throw new Error(`Default exp.py not found for model: ${selectedModel} at ${expSourcePath}`); throw new Error(`Default exp.py not found for model: ${selectedModel} at ${expSourcePath}`);
} }
// Copy to project folder (e.g., /home/kitraining/coco_tool/backend/project_XX/YY/exp.py) // Copy to project folder (e.g., /home/kitraining/coco_tool/backend/project_XX/YY/exp.py)
const projectDetailsId = training.project_details_id; const projectDetailsId = training.project_details_id;
const projectFolder = path.resolve(__dirname, `../project_23/${projectDetailsId}`); const projectFolder = path.resolve(__dirname, `../project_23/${projectDetailsId}`);
if (!fs.existsSync(projectFolder)) { if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true }); fs.mkdirSync(projectFolder, { recursive: true });
} }
const expDestPath = path.join(projectFolder, 'exp.py'); const expDestPath = path.join(projectFolder, 'exp.py');
fs.copyFileSync(expSourcePath, expDestPath); fs.copyFileSync(expSourcePath, expDestPath);
return { type: 'default', expPath: expDestPath }; return { type: 'default', expPath: expDestPath };
} }
// If transfer_learning is 'sketch', generate a custom exp.py as before // If transfer_learning is 'sketch', generate a custom exp.py as before
if (training.transfer_learning === 'sketch') { if (training.transfer_learning === 'sketch') {
// ...existing custom exp.py generation logic here (copy from previous implementation)... // ...existing custom exp.py generation logic here (copy from previous implementation)...
// For brevity, you can call generateYoloxInferenceExp or similar here, or inline the logic. // For brevity, you can call generateYoloxInferenceExp or similar here, or inline the logic.
// Example: // Example:
const expContent = await generateYoloxInferenceExp(trainingId); const expContent = await generateYoloxInferenceExp(trainingId);
return { type: 'custom', expContent }; return { type: 'custom', expContent };
} }
throw new Error('Unknown transfer_learning type: ' + training.transfer_learning); throw new Error('Unknown transfer_learning type: ' + training.transfer_learning);
} }
async function saveYoloxExp(trainingId, outPath) { async function saveYoloxExp(trainingId, outPath) {
const expResult = await generateYoloxExp(trainingId); const expResult = await generateYoloxExp(trainingId);
if (expResult.type === 'custom' && expResult.expContent) { if (expResult.type === 'custom' && expResult.expContent) {
fs.writeFileSync(outPath, expResult.expContent); fs.writeFileSync(outPath, expResult.expContent);
return outPath; return outPath;
} else if (expResult.type === 'default' && expResult.expPath) { } else if (expResult.type === 'default' && expResult.expPath) {
// Optionally copy the file if outPath is different // Optionally copy the file if outPath is different
if (expResult.expPath !== outPath) { if (expResult.expPath !== outPath) {
fs.copyFileSync(expResult.expPath, outPath); fs.copyFileSync(expResult.expPath, outPath);
} }
return outPath; return outPath;
} else { } else {
throw new Error('Unknown expResult type or missing content'); throw new Error('Unknown expResult type or missing content');
} }
} }
// Generate a second exp.py for inference, using the provided template and DB values // Generate a second exp.py for inference, using the provided template and DB values
async function generateYoloxInferenceExp(trainingId, options = {}) { async function generateYoloxInferenceExp(trainingId, options = {}) {
let training = await Training.findByPk(trainingId); let training = await Training.findByPk(trainingId);
if (!training) { if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } }); training = await Training.findOne({ where: { project_details_id: trainingId } });
} }
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId); if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// Always use the trainingId (project_details_id) for annotation file names // Always use the trainingId (project_details_id) for annotation file names
const projectDetailsId = training.project_details_id; const projectDetailsId = training.project_details_id;
const dataDir = options.data_dir || '/home/kitraining/To_Annotate/'; const dataDir = options.data_dir || '/home/kitraining/To_Annotate/';
const trainAnn = options.train_ann || `coco_project_${trainingId}_train.json`; const trainAnn = options.train_ann || `coco_project_${trainingId}_train.json`;
const valAnn = options.val_ann || `coco_project_${trainingId}_valid.json`; const valAnn = options.val_ann || `coco_project_${trainingId}_valid.json`;
const testAnn = options.test_ann || `coco_project_${trainingId}_test.json`; const testAnn = options.test_ann || `coco_project_${trainingId}_test.json`;
// Get num_classes from TrainingProject.classes JSON // Get num_classes from TrainingProject.classes JSON
let numClasses = 80; let numClasses = 80;
try { try {
const trainingProject = await TrainingProject.findByPk(projectDetailsId); const trainingProject = await TrainingProject.findByPk(projectDetailsId);
if (trainingProject && trainingProject.classes) { if (trainingProject && trainingProject.classes) {
let classesArr = trainingProject.classes; let classesArr = trainingProject.classes;
if (typeof classesArr === 'string') { if (typeof classesArr === 'string') {
classesArr = JSON.parse(classesArr); classesArr = JSON.parse(classesArr);
} }
if (Array.isArray(classesArr)) { if (Array.isArray(classesArr)) {
numClasses = classesArr.filter(c => c !== null && c !== undefined && c !== '').length; numClasses = classesArr.filter(c => c !== null && c !== undefined && c !== '').length;
} else if (typeof classesArr === 'object' && classesArr !== null) { } else if (typeof classesArr === 'object' && classesArr !== null) {
numClasses = Object.keys(classesArr).filter(k => classesArr[k] !== null && classesArr[k] !== undefined && classesArr[k] !== '').length; numClasses = Object.keys(classesArr).filter(k => classesArr[k] !== null && classesArr[k] !== undefined && classesArr[k] !== '').length;
} }
} }
} catch (e) { } catch (e) {
console.warn('Could not determine num_classes from TrainingProject.classes:', e); console.warn('Could not determine num_classes from TrainingProject.classes:', e);
} }
const depth = options.depth || training.depth || 1.00; const depth = options.depth || training.depth || 1.00;
const width = options.width || training.width || 1.00; const width = options.width || training.width || 1.00;
const inputSize = options.input_size || training.input_size || [640, 640]; const inputSize = options.input_size || training.input_size || [640, 640];
const mosaicScale = options.mosaic_scale || training.mosaic_scale || [0.1, 2]; const mosaicScale = options.mosaic_scale || training.mosaic_scale || [0.1, 2];
const randomSize = options.random_size || training.random_size || [10, 20]; const randomSize = options.random_size || training.random_size || [10, 20];
const testSize = options.test_size || training.test_size || [640, 640]; const testSize = options.test_size || training.test_size || [640, 640];
const expName = options.exp_name || 'inference_exp'; const expName = options.exp_name || 'inference_exp';
const enableMixup = options.enable_mixup !== undefined ? options.enable_mixup : false; const enableMixup = options.enable_mixup !== undefined ? options.enable_mixup : false;
let expContent = ''; let expContent = '';
expContent += `#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.data_dir = "${dataDir}"\n self.train_ann = "${trainAnn}"\n self.val_ann = "${valAnn}"\n self.test_ann = "coco_project_${trainingId}_test.json"\n self.num_classes = ${numClasses}\n`; expContent += `#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.data_dir = "${dataDir}"\n self.train_ann = "${trainAnn}"\n self.val_ann = "${valAnn}"\n self.test_ann = "coco_project_${trainingId}_test.json"\n self.num_classes = ${numClasses}\n`;
// Set pretrained_ckpt if transfer_learning is 'coco' // Set pretrained_ckpt if transfer_learning is 'coco'
if (training.transfer_learning && typeof training.transfer_learning === 'string' && training.transfer_learning.toLowerCase() === 'coco') { if (training.transfer_learning && typeof training.transfer_learning === 'string' && training.transfer_learning.toLowerCase() === 'coco') {
const yoloxBaseDir = '/home/kitraining/Yolox/YOLOX-main'; const yoloxBaseDir = '/home/kitraining/Yolox/YOLOX-main';
const selectedModel = training.selected_model ? training.selected_model.replace(/\.pth$/i, '') : ''; const selectedModel = training.selected_model ? training.selected_model.replace(/\.pth$/i, '') : '';
if (selectedModel) { if (selectedModel) {
expContent += ` self.pretrained_ckpt = r'${yoloxBaseDir}/pretrained/${selectedModel}.pth'\n`; expContent += ` self.pretrained_ckpt = r'${yoloxBaseDir}/pretrained/${selectedModel}.pth'\n`;
} }
} }
expContent += ` self.depth = ${depth}\n self.width = ${width}\n self.input_size = (${Array.isArray(inputSize) ? inputSize.join(', ') : inputSize})\n self.mosaic_scale = (${Array.isArray(mosaicScale) ? mosaicScale.join(', ') : mosaicScale})\n self.random_size = (${Array.isArray(randomSize) ? randomSize.join(', ') : randomSize})\n self.test_size = (${Array.isArray(testSize) ? testSize.join(', ') : testSize})\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]\n self.enable_mixup = ${enableMixup ? 'True' : 'False'}\n`; expContent += ` self.depth = ${depth}\n self.width = ${width}\n self.input_size = (${Array.isArray(inputSize) ? inputSize.join(', ') : inputSize})\n self.mosaic_scale = (${Array.isArray(mosaicScale) ? mosaicScale.join(', ') : mosaicScale})\n self.random_size = (${Array.isArray(randomSize) ? randomSize.join(', ') : randomSize})\n self.test_size = (${Array.isArray(testSize) ? testSize.join(', ') : testSize})\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]\n self.enable_mixup = ${enableMixup ? 'True' : 'False'}\n`;
return expContent; return expContent;
} }
// Save inference exp.py to a custom path // Save inference exp.py to a custom path
async function saveYoloxInferenceExp(trainingId, outPath, options = {}) { async function saveYoloxInferenceExp(trainingId, outPath, options = {}) {
const expContent = await generateYoloxInferenceExp(trainingId, options); const expContent = await generateYoloxInferenceExp(trainingId, options);
fs.writeFileSync(outPath, expContent); fs.writeFileSync(outPath, expContent);
return outPath; return outPath;
} }
module.exports = { generateYoloxExp, saveYoloxExp, generateYoloxInferenceExp, saveYoloxInferenceExp }; module.exports = { generateYoloxExp, saveYoloxExp, generateYoloxInferenceExp, saveYoloxInferenceExp };

467
backend/services/generate_json_yolox.py
View File

@@ -1,179 +1,288 @@
import json import json
import os import os
import math import math
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
def generate_training_json(training_id): def generate_training_json(training_id):
"""Generate COCO JSON for training, validation, and test sets""" """Generate COCO JSON for training, validation, and test sets"""
# training_id is now project_details_id # training_id is now project_details_id
training_project_details = TrainingProjectDetails.query.get(training_id) training_project_details = TrainingProjectDetails.query.get(training_id)
if not training_project_details: if not training_project_details:
raise Exception(f'No TrainingProjectDetails found for project_details_id {training_id}') raise Exception(f'No TrainingProjectDetails found for project_details_id {training_id}')
details_obj = training_project_details.to_dict() details_obj = training_project_details.to_dict()
# Get parent project for name # Get parent project for name
training_project = TrainingProject.query.get(details_obj['project_id']) training_project = TrainingProject.query.get(details_obj['project_id'])
# Get split percentages (default values if not set) # Get the data directory setting for image paths
train_percent = details_obj.get('train_percent', 85) from services.settings_service import get_setting
valid_percent = details_obj.get('valid_percent', 10) data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
test_percent = details_obj.get('test_percent', 5)
# Fix UNC path if it's missing the \\ prefix
coco_images = [] # Check if it looks like a UNC path without proper prefix (e.g., "192.168.1.19\...")
coco_annotations = [] if data_dir and not data_dir.startswith('\\\\') and not data_dir.startswith('/'):
coco_categories = [] # Check if it starts with an IP address pattern
category_map = {} import re
category_id = 0 if re.match(r'^\d+\.\d+\.\d+\.\d+[/\\]', data_dir):
image_id = 0 data_dir = '\\\\' + data_dir
annotation_id = 0
# Ensure data_dir ends with separator
for cls in details_obj['class_map']: if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
asg_map = [] data_dir += os.sep
list_asg = cls[1]
# Get split percentages (default values if not set)
for asg in list_asg: train_percent = details_obj.get('train_percent', 85)
asg_map.append({'original': asg[0], 'mapped': asg[1]}) valid_percent = details_obj.get('valid_percent', 10)
# Build category list and mapping test_percent = details_obj.get('test_percent', 5)
if asg[1] and asg[1] not in category_map:
category_map[asg[1]] = category_id coco_images = []
coco_categories.append({'id': category_id, 'name': asg[1], 'supercategory': ''}) coco_annotations = []
category_id += 1 coco_categories = []
category_map = {}
# Get images for this project category_id = 0
images = Image.query.filter_by(project_id=cls[0]).all() image_id = 0
annotation_id = 0
for image in images:
image_id += 1 # Build category list and mapping from class_map dictionary {source: target}
file_name = image.image_path class_map = details_obj.get('class_map', {})
# Clean up file path for source_class, target_class in class_map.items():
if '%20' in file_name: if target_class and target_class not in category_map:
file_name = file_name.replace('%20', ' ') category_map[target_class] = category_id
if file_name and file_name.startswith('/data/local-files/?d='): coco_categories.append({'id': category_id, 'name': target_class, 'supercategory': ''})
file_name = file_name.replace('/data/local-files/?d=', '') category_id += 1
file_name = file_name.replace('/home/kitraining/home/kitraining/', '')
if file_name and file_name.startswith('home/kitraining/To_Annotate/'): # Get all annotation projects (Label Studio project IDs)
file_name = file_name.replace('home/kitraining/To_Annotate/', '') annotation_projects = details_obj.get('annotation_projects', [])
# Get annotations for this image # Get class mappings from database grouped by Label Studio project
annotations = Annotation.query.filter_by(image_id=image.image_id).all() from models.ClassMapping import ClassMapping
all_mappings = ClassMapping.query.filter_by(project_details_id=training_id).all()
coco_images.append({
'id': image_id, # Group mappings by Label Studio project ID
'file_name': file_name, mappings_by_project = {}
'width': image.width or 0, for mapping in all_mappings:
'height': image.height or 0 ls_proj_id = mapping.label_studio_project_id
}) if ls_proj_id not in mappings_by_project:
mappings_by_project[ls_proj_id] = {}
for annotation in annotations: mappings_by_project[ls_proj_id][mapping.source_class] = mapping.target_class
# Translate class name using asg_map
mapped_class = annotation.Label # Also add target class to category map if not present
for map_entry in asg_map: if mapping.target_class and mapping.target_class not in category_map:
if annotation.Label == map_entry['original']: category_map[mapping.target_class] = category_id
mapped_class = map_entry['mapped'] coco_categories.append({'id': category_id, 'name': mapping.target_class, 'supercategory': ''})
break category_id += 1
# Only add annotation if mapped_class is valid # Iterate through each annotation project to collect images and annotations
if mapped_class and mapped_class in category_map: for ls_project_id in annotation_projects:
annotation_id += 1 # Get images for this Label Studio project
area = 0 images = Image.query.filter_by(project_id=ls_project_id).all()
if annotation.width and annotation.height:
area = annotation.width * annotation.height for image in images:
image_id += 1
coco_annotations.append({ file_name = image.image_path
'id': annotation_id,
'image_id': image_id, # Clean up file path from Label Studio format
'category_id': category_map[mapped_class], if '%20' in file_name:
'bbox': [annotation.x, annotation.y, annotation.width, annotation.height], file_name = file_name.replace('%20', ' ')
'area': area, if file_name and file_name.startswith('/data/local-files/?d='):
'iscrowd': 0 file_name = file_name.replace('/data/local-files/?d=', '')
})
# Remove any Label Studio prefixes but keep full path
# Shuffle images for random split using seed # Common Label Studio patterns
def seeded_random(seed): prefixes_to_remove = [
x = math.sin(seed) * 10000 '//192.168.1.19/home/kitraining/To_Annotate/',
return x - math.floor(x) '192.168.1.19/home/kitraining/To_Annotate/',
'/home/kitraining/home/kitraining/',
def shuffle(array, seed): 'home/kitraining/To_Annotate/',
for i in range(len(array) - 1, 0, -1): '/home/kitraining/To_Annotate/',
j = int(seeded_random(seed + i) * (i + 1)) ]
array[i], array[j] = array[j], array[i]
# Try each prefix
# Use seed from details_obj if present, else default to 42 for prefix in prefixes_to_remove:
split_seed = details_obj.get('seed', 42) if file_name.startswith(prefix):
if split_seed is not None: file_name = file_name[len(prefix):]
split_seed = int(split_seed) break
else:
split_seed = 42 # Construct ABSOLUTE path using data_dir
# Detect platform for proper path handling
shuffle(coco_images, split_seed) import platform
is_windows = platform.system() == 'Windows'
# Split images
total_images = len(coco_images) # Normalize path separators in file_name to forward slashes first (OS-agnostic)
train_count = int(total_images * train_percent / 100) file_name = file_name.replace('\\', '/')
valid_count = int(total_images * valid_percent / 100)
test_count = total_images - train_count - valid_count # Normalize data_dir to use forward slashes
normalized_data_dir = data_dir.rstrip('/\\').replace('\\', '/')
train_images = coco_images[0:train_count]
valid_images = coco_images[train_count:train_count + valid_count] # Check if file_name is already an absolute path
test_images = coco_images[train_count + valid_count:] is_absolute = False
if is_windows:
# Helper to get image ids for each split # Windows: Check for drive letter (C:/) or UNC path (//server/)
train_image_ids = {img['id'] for img in train_images} is_absolute = (len(file_name) > 1 and file_name[1] == ':') or file_name.startswith('//')
valid_image_ids = {img['id'] for img in valid_images} else:
test_image_ids = {img['id'] for img in test_images} # Linux/Mac: Check for leading /
is_absolute = file_name.startswith('/')
# Split annotations
train_annotations = [ann for ann in coco_annotations if ann['image_id'] in train_image_ids] if not is_absolute:
valid_annotations = [ann for ann in coco_annotations if ann['image_id'] in valid_image_ids] # It's a relative path, combine with data_dir
test_annotations = [ann for ann in coco_annotations if ann['image_id'] in test_image_ids] if normalized_data_dir.startswith('//'):
# UNC path on Windows
# Build final COCO JSONs file_name = normalized_data_dir + '/' + file_name
def build_coco_json(images, annotations, categories): else:
return { # Regular path - use os.path.join but with forward slashes
'images': images, file_name = os.path.join(normalized_data_dir, file_name).replace('\\', '/')
'annotations': annotations,
'categories': categories # Final OS-specific normalization
} if is_windows:
# Convert to Windows-style backslashes
train_json = build_coco_json(train_images, train_annotations, coco_categories) file_name = file_name.replace('/', '\\')
valid_json = build_coco_json(valid_images, valid_annotations, coco_categories) else:
test_json = build_coco_json(test_images, test_annotations, coco_categories) # Keep as forward slashes for Linux/Mac
file_name = file_name.replace('\\', '/')
# Create output directory
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}' # Get annotations for this image
annotations_dir = '/home/kitraining/To_Annotate/annotations' annotations = Annotation.query.filter_by(image_id=image.image_id).all()
os.makedirs(annotations_dir, exist_ok=True)
# Ensure width and height are integers and valid
# Write to files # If missing or invalid, skip this image or use default dimensions
train_path = f'{annotations_dir}/coco_project_{training_id}_train.json' img_width = int(image.width) if image.width else 0
valid_path = f'{annotations_dir}/coco_project_{training_id}_valid.json' img_height = int(image.height) if image.height else 0
test_path = f'{annotations_dir}/coco_project_{training_id}_test.json'
# Skip images with invalid dimensions
with open(train_path, 'w') as f: if img_width <= 0 or img_height <= 0:
json.dump(train_json, f, indent=2) print(f'Warning: Skipping image {file_name} with invalid dimensions: {img_width}x{img_height}')
with open(valid_path, 'w') as f: continue
json.dump(valid_json, f, indent=2)
with open(test_path, 'w') as f: coco_images.append({
json.dump(test_json, f, indent=2) 'id': image_id,
'file_name': file_name, # Use absolute path
print(f'COCO JSON splits written to {annotations_dir} for trainingId {training_id}') 'width': img_width,
'height': img_height
# Also generate inference exp.py })
from services.generate_yolox_exp import generate_yolox_inference_exp
project_folder = os.path.join(os.path.dirname(__file__), '..', project_name, str(training_id)) for annotation in annotations:
os.makedirs(project_folder, exist_ok=True) # Translate class name using class_map for this specific Label Studio project
original_class = annotation.Label
inference_exp_path = os.path.join(project_folder, 'exp_infer.py') project_class_map = mappings_by_project.get(ls_project_id, {})
try: mapped_class = project_class_map.get(original_class, original_class)
exp_content = generate_yolox_inference_exp(training_id)
with open(inference_exp_path, 'w') as f: # Only add annotation if mapped_class is valid
f.write(exp_content) if mapped_class and mapped_class in category_map:
print(f'Inference exp.py written to {inference_exp_path}') annotation_id += 1
except Exception as err: area = 0
print(f'Failed to generate inference exp.py: {err}') if annotation.width and annotation.height:
area = annotation.width * annotation.height
coco_annotations.append({
'id': annotation_id,
'image_id': image_id,
'category_id': category_map[mapped_class],
'bbox': [annotation.x, annotation.y, annotation.width, annotation.height],
'area': area,
'iscrowd': 0
})
# Shuffle images for random split using seed
def seeded_random(seed):
x = math.sin(seed) * 10000
return x - math.floor(x)
def shuffle(array, seed):
for i in range(len(array) - 1, 0, -1):
j = int(seeded_random(seed + i) * (i + 1))
array[i], array[j] = array[j], array[i]
# Use seed from details_obj if present, else default to 42
split_seed = details_obj.get('seed', 42)
if split_seed is not None:
split_seed = int(split_seed)
else:
split_seed = 42
shuffle(coco_images, split_seed)
# Split images
total_images = len(coco_images)
train_count = int(total_images * train_percent / 100)
valid_count = int(total_images * valid_percent / 100)
test_count = total_images - train_count - valid_count
train_images = coco_images[0:train_count]
valid_images = coco_images[train_count:train_count + valid_count]
test_images = coco_images[train_count + valid_count:]
# Helper to get image ids for each split
train_image_ids = {img['id'] for img in train_images}
valid_image_ids = {img['id'] for img in valid_images}
test_image_ids = {img['id'] for img in test_images}
# Split annotations
train_annotations = [ann for ann in coco_annotations if ann['image_id'] in train_image_ids]
valid_annotations = [ann for ann in coco_annotations if ann['image_id'] in valid_image_ids]
test_annotations = [ann for ann in coco_annotations if ann['image_id'] in test_image_ids]
# Build final COCO JSONs
def build_coco_json(images, annotations, categories):
return {
'images': images,
'annotations': annotations,
'categories': categories
}
train_json = build_coco_json(train_images, train_annotations, coco_categories)
valid_json = build_coco_json(valid_images, valid_annotations, coco_categories)
test_json = build_coco_json(test_images, test_annotations, coco_categories)
# Create output directory
from services.settings_service import get_setting
from models.training import Training
output_base_path = get_setting('yolox_output_path', './backend')
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}'
# Get training record to use its name for folder
training_record = Training.query.filter_by(project_details_id=training_id).first()
training_folder_name = f"{training_record.exp_name or training_record.training_name or 'training'}_{training_record.id}" if training_record else str(training_id)
training_folder_name = training_folder_name.replace(' ', '_')
# Use training_record.id for file names to match what generate_yolox_exp expects
training_file_id = training_record.id if training_record else training_id
# Save annotations to the configured output folder
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
os.makedirs(annotations_dir, exist_ok=True)
# Write to files
train_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_train.json')
valid_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_valid.json')
test_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_test.json')
with open(train_path, 'w') as f:
json.dump(train_json, f, indent=2)
with open(valid_path, 'w') as f:
json.dump(valid_json, f, indent=2)
with open(test_path, 'w') as f:
json.dump(test_json, f, indent=2)
print(f'COCO JSON splits written to {annotations_dir} for trainingId {training_id}')
# Also generate inference exp.py
from services.generate_yolox_exp import generate_yolox_inference_exp
project_folder = os.path.join(output_base_path, project_name, str(training_id))
os.makedirs(project_folder, exist_ok=True)
inference_exp_path = os.path.join(project_folder, 'exp_infer.py')
try:
exp_content = generate_yolox_inference_exp(training_id)
with open(inference_exp_path, 'w') as f:
f.write(exp_content)
print(f'Inference exp.py written to {inference_exp_path}')
except Exception as err:
print(f'Failed to generate inference exp.py: {err}')

557
backend/services/generate_yolox_exp.py
View File

@@ -1,228 +1,329 @@
import os import os
import shutil import shutil
import importlib.util import importlib.util
from models.training import Training from models.training import Training
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
def load_base_config(selected_model): def load_base_config(selected_model):
"""Load base configuration for a specific YOLOX model""" """Load base configuration for a specific YOLOX model"""
model_name = selected_model.lower().replace('-', '_').replace('.pth', '') model_name = selected_model.lower().replace('-', '_').replace('.pth', '')
base_config_path = os.path.join(os.path.dirname(__file__), '..', 'data', f'{model_name}.py') base_config_path = os.path.join(os.path.dirname(__file__), '..', 'data', f'{model_name}.py')
if not os.path.exists(base_config_path): if not os.path.exists(base_config_path):
raise Exception(f'Base configuration not found for model: {model_name} at {base_config_path}') raise Exception(f'Base configuration not found for model: {model_name} at {base_config_path}')
# Load the module dynamically # Load the module dynamically
spec = importlib.util.spec_from_file_location(f"base_config_{model_name}", base_config_path) spec = importlib.util.spec_from_file_location(f"base_config_{model_name}", base_config_path)
module = importlib.util.module_from_spec(spec) module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) spec.loader.exec_module(module)
# Extract all attributes from BaseExp class # Extract all attributes from BaseExp class
base_exp = module.BaseExp() base_exp = module.BaseExp()
base_config = {} base_config = {}
for attr in dir(base_exp): for attr in dir(base_exp):
if not attr.startswith('_'): if not attr.startswith('_'):
base_config[attr] = getattr(base_exp, attr) base_config[attr] = getattr(base_exp, attr)
return base_config return base_config
def generate_yolox_exp(training_id): def generate_yolox_exp(training_id):
"""Generate YOLOX exp.py file""" """Generate YOLOX exp.py file"""
# Fetch training row from DB # Fetch training row from DB
training = Training.query.get(training_id) training = Training.query.get(training_id)
if not training: if not training:
training = Training.query.filter_by(project_details_id=training_id).first() training = Training.query.filter_by(project_details_id=training_id).first()
if not training: if not training:
raise Exception(f'Training not found for trainingId or project_details_id: {training_id}') raise Exception(f'Training not found for trainingId or project_details_id: {training_id}')
# If transfer_learning is 'coco', generate exp using base config + custom settings # If transfer_learning is 'coco', generate exp using base config + custom settings
if training.transfer_learning == 'coco': if training.transfer_learning == 'coco':
exp_content = generate_yolox_inference_exp(training_id, use_base_config=True) exp_content = generate_yolox_inference_exp(training_id, use_base_config=True)
return {'type': 'custom', 'expContent': exp_content} return {'type': 'custom', 'expContent': exp_content}
# If transfer_learning is 'sketch', generate custom exp.py # If transfer_learning is 'sketch', generate custom exp.py
if training.transfer_learning == 'sketch': if training.transfer_learning == 'sketch':
exp_content = generate_yolox_inference_exp(training_id, use_base_config=False) exp_content = generate_yolox_inference_exp(training_id, use_base_config=False)
return {'type': 'custom', 'expContent': exp_content} return {'type': 'custom', 'expContent': exp_content}
raise Exception(f'Unknown transfer_learning type: {training.transfer_learning}') raise Exception(f'Unknown transfer_learning type: {training.transfer_learning}')
def save_yolox_exp(training_id, out_path): def save_yolox_exp(training_id, out_path):
"""Save YOLOX exp.py to specified path""" """Save YOLOX exp.py to specified path"""
exp_result = generate_yolox_exp(training_id) exp_result = generate_yolox_exp(training_id)
if exp_result['type'] == 'custom' and 'expContent' in exp_result: if exp_result['type'] == 'custom' and 'expContent' in exp_result:
with open(out_path, 'w') as f: with open(out_path, 'w') as f:
f.write(exp_result['expContent']) f.write(exp_result['expContent'])
return out_path return out_path
elif exp_result['type'] == 'default' and 'expPath' in exp_result: elif exp_result['type'] == 'default' and 'expPath' in exp_result:
# Optionally copy the file if outPath is different # Optionally copy the file if outPath is different
if exp_result['expPath'] != out_path: if exp_result['expPath'] != out_path:
shutil.copyfile(exp_result['expPath'], out_path) shutil.copyfile(exp_result['expPath'], out_path)
return out_path return out_path
else: else:
raise Exception('Unknown expResult type or missing content') raise Exception('Unknown expResult type or missing content')
def generate_yolox_inference_exp(training_id, options=None, use_base_config=False): def generate_yolox_inference_exp(training_id, options=None, use_base_config=False):
"""Generate inference exp.py using DB values """Generate inference exp.py using DB values
Args: Args:
training_id: The training/project_details ID training_id: The training/project_details ID
options: Optional overrides for data paths options: Optional overrides for data paths
use_base_config: If True, load base config and only override with user-defined values use_base_config: If True, load base config and only override with user-defined values
""" """
if options is None: if options is None:
options = {} options = {}
training = Training.query.get(training_id) training = Training.query.get(training_id)
if not training: if not training:
training = Training.query.filter_by(project_details_id=training_id).first() training = Training.query.filter_by(project_details_id=training_id).first()
if not training: if not training:
raise Exception(f'Training not found for trainingId or project_details_id: {training_id}') raise Exception(f'Training not found for trainingId or project_details_id: {training_id}')
# Always use the training_id (project_details_id) for annotation file names # Always use the project_details_id for annotation file names and paths
project_details_id = training.project_details_id project_details_id = training.project_details_id
data_dir = options.get('data_dir', '/home/kitraining/To_Annotate/') # Get annotation file names from options or use defaults
train_ann = options.get('train_ann', f'coco_project_{training_id}_train.json') # Use training.id (not project_details_id) for consistency with generate_training_json
val_ann = options.get('val_ann', f'coco_project_{training_id}_valid.json') train_ann = options.get('train_ann', f'coco_project_{training_id}_train.json')
test_ann = options.get('test_ann', f'coco_project_{training_id}_test.json') val_ann = options.get('val_ann', f'coco_project_{training_id}_valid.json')
test_ann = options.get('test_ann', f'coco_project_{training_id}_test.json')
# Get num_classes from TrainingProject.classes JSON
num_classes = 80 # Get data_dir - this should point to where IMAGES are located (not annotations)
try: # YOLOX will combine data_dir + file_name from COCO JSON to find images
training_project = TrainingProject.query.get(project_details_id) # The annotations are in a separate location (output folder)
if training_project and training_project.classes: from services.settings_service import get_setting
classes_arr = training_project.classes from models.TrainingProjectDetails import TrainingProjectDetails
if isinstance(classes_arr, str):
import json if 'data_dir' in options:
classes_arr = json.loads(classes_arr) data_dir = options['data_dir']
else:
if isinstance(classes_arr, list): # Use the yolox_data_dir setting - this is where training images are stored
num_classes = len([c for c in classes_arr if c not in [None, '']]) data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
elif isinstance(classes_arr, dict): # Ensure it ends with a separator
num_classes = len([k for k, v in classes_arr.items() if v not in [None, '']]) if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
except Exception as e: data_dir += os.sep
print(f'Could not determine num_classes from TrainingProject.classes: {e}')
# Get num_classes from ProjectClass table (3NF)
# Initialize config dictionary num_classes = 80
config = {} try:
from models.ProjectClass import ProjectClass
# If using base config (transfer learning from COCO), load protected parameters first training_project = TrainingProject.query.get(project_details_id)
if use_base_config and training.selected_model: if training_project:
try: # Count classes from ProjectClass table
base_config = load_base_config(training.selected_model) class_count = ProjectClass.query.filter_by(project_id=training_project.project_id).count()
config.update(base_config) if class_count > 0:
print(f'Loaded base config for {training.selected_model}: {list(base_config.keys())}') num_classes = class_count
except Exception as e: except Exception as e:
print(f'Warning: Could not load base config for {training.selected_model}: {e}') print(f'Could not determine num_classes from ProjectClass: {e}')
print('Falling back to custom settings only')
# Initialize config dictionary
# Override with user-defined values from training table (only if they exist and are not None) config = {}
user_overrides = {
'depth': training.depth, # If using base config (transfer learning from COCO), load protected parameters first
'width': training.width, if use_base_config and training.selected_model:
'input_size': training.input_size, try:
'mosaic_scale': training.mosaic_scale, base_config = load_base_config(training.selected_model)
'test_size': training.test_size, config.update(base_config)
'enable_mixup': training.enable_mixup, print(f'Loaded base config for {training.selected_model}: {list(base_config.keys())}')
'max_epoch': training.max_epoch, except Exception as e:
'warmup_epochs': training.warmup_epochs, print(f'Warning: Could not load base config for {training.selected_model}: {e}')
'warmup_lr': training.warmup_lr, print('Falling back to custom settings only')
'basic_lr_per_img': training.basic_lr_per_img,
'scheduler': training.scheduler, # Get size arrays from TrainingSize table (3NF)
'no_aug_epochs': training.no_aug_epochs, from models.TrainingSize import TrainingSize
'min_lr_ratio': training.min_lr_ratio,
'ema': training.ema, def get_size_array(training_id, size_type):
'weight_decay': training.weight_decay, """Helper to get size array from TrainingSize table"""
'momentum': training.momentum, sizes = TrainingSize.query.filter_by(
'print_interval': training.print_interval, training_id=training_id,
'eval_interval': training.eval_interval, size_type=size_type
'test_conf': training.test_conf, ).order_by(TrainingSize.value_order).all()
'nms_thre': training.nms_thre, return [s.value for s in sizes] if sizes else None
'mosaic_prob': training.mosaic_prob,
'mixup_prob': training.mixup_prob, input_size = get_size_array(training.id, 'input_size')
'hsv_prob': training.hsv_prob, test_size = get_size_array(training.id, 'test_size')
'flip_prob': training.flip_prob, mosaic_scale = get_size_array(training.id, 'mosaic_scale')
'degrees': training.degrees, mixup_scale = get_size_array(training.id, 'mixup_scale')
'translate': training.translate,
'shear': training.shear, # Override with user-defined values from training table (only if they exist and are not None)
'mixup_scale': training.mixup_scale, user_overrides = {
'activation': training.activation, 'depth': training.depth,
} 'width': training.width,
'input_size': input_size,
# Only override if value is explicitly set (not None) 'mosaic_scale': mosaic_scale,
for key, value in user_overrides.items(): 'test_size': test_size,
if value is not None: 'enable_mixup': training.enable_mixup,
config[key] = value 'max_epoch': training.max_epoch,
'warmup_epochs': training.warmup_epochs,
# Apply any additional options overrides 'warmup_lr': training.warmup_lr,
config.update(options) 'basic_lr_per_img': training.basic_lr_per_img,
'scheduler': training.scheduler,
# Set defaults for any missing required parameters 'no_aug_epochs': training.no_aug_epochs,
config.setdefault('depth', 1.00) 'min_lr_ratio': training.min_lr_ratio,
config.setdefault('width', 1.00) 'ema': training.ema,
config.setdefault('input_size', [640, 640]) 'weight_decay': training.weight_decay,
config.setdefault('mosaic_scale', [0.1, 2]) 'momentum': training.momentum,
config.setdefault('random_size', [10, 20]) 'print_interval': training.print_interval,
config.setdefault('test_size', [640, 640]) 'eval_interval': training.eval_interval,
config.setdefault('enable_mixup', False) 'test_conf': training.test_conf,
config.setdefault('exp_name', 'inference_exp') 'nms_thre': training.nms_thre,
'mosaic_prob': training.mosaic_prob,
# Build exp content 'mixup_prob': training.mixup_prob,
exp_content = f'''#!/usr/bin/env python3 'hsv_prob': training.hsv_prob,
# -*- coding:utf-8 -*- 'flip_prob': training.flip_prob,
# Copyright (c) Megvii, Inc. and its affiliates. # Convert single values to tuples for YOLOX augmentation parameters
'degrees': (training.degrees, training.degrees) if training.degrees is not None and not isinstance(training.degrees, (list, tuple)) else training.degrees,
import os 'translate': (training.translate, training.translate) if training.translate is not None and not isinstance(training.translate, (list, tuple)) else training.translate,
'shear': (training.shear, training.shear) if training.shear is not None and not isinstance(training.shear, (list, tuple)) else training.shear,
from yolox.exp import Exp as MyExp 'mixup_scale': mixup_scale,
'activation': training.activation,
}
class Exp(MyExp):
def __init__(self): # Only override if value is explicitly set (not None)
super(Exp, self).__init__() for key, value in user_overrides.items():
self.data_dir = "{data_dir}" if value is not None:
self.train_ann = "{train_ann}" config[key] = value
self.val_ann = "{val_ann}"
self.test_ann = "{test_ann}" # Apply any additional options overrides
self.num_classes = {num_classes} config.update(options)
'''
# Set defaults for any missing required parameters
# Set pretrained_ckpt if transfer_learning is 'coco' config.setdefault('depth', 1.00)
if training.transfer_learning and isinstance(training.transfer_learning, str) and training.transfer_learning.lower() == 'coco': config.setdefault('width', 1.00)
yolox_base_dir = '/home/kitraining/Yolox/YOLOX-main' config.setdefault('input_size', [640, 640])
selected_model = training.selected_model.replace('.pth', '') if training.selected_model else '' config.setdefault('mosaic_scale', [0.1, 2])
if selected_model: config.setdefault('random_size', [10, 20])
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n" config.setdefault('test_size', [640, 640])
config.setdefault('enable_mixup', False)
# Format arrays config.setdefault('exp_name', 'inference_exp')
def format_value(val):
if isinstance(val, (list, tuple)): # Prepare data_dir for template - escape backslashes and remove trailing separator
return '(' + ', '.join(map(str, val)) + ')' data_dir_clean = data_dir.rstrip('/\\')
elif isinstance(val, bool): data_dir_escaped = data_dir_clean.replace('\\', '\\\\')
return str(val)
elif isinstance(val, str): # Calculate annotations directory (where JSON files are stored)
return f'"{val}"' # This is in the output folder, not with the images
else: from models.TrainingProjectDetails import TrainingProjectDetails
return str(val) details = TrainingProjectDetails.query.get(project_details_id)
if details:
# Add all config parameters to exp training_project = TrainingProject.query.get(details.project_id)
for key, value in config.items(): project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details.project_id}'
if key not in ['exp_name']: # exp_name is handled separately else:
exp_content += f" self.{key} = {format_value(value)}\n" project_name = f'project_{project_details_id}'
# Add exp_name at the end (uses dynamic path) training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
exp_content += f''' self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0] training_folder_name = training_folder_name.replace(' ', '_')
'''
output_base_path = get_setting('yolox_output_path', './backend')
return exp_content annotations_parent_dir = os.path.join(output_base_path, project_name, training_folder_name)
annotations_parent_escaped = annotations_parent_dir.replace('\\', '\\\\')
def save_yolox_inference_exp(training_id, out_path, options=None):
"""Save inference exp.py to custom path""" # Build exp content
exp_content = generate_yolox_inference_exp(training_id, options, use_base_config=False) exp_content = f'''#!/usr/bin/env python3
with open(out_path, 'w') as f: # -*- coding:utf-8 -*-
f.write(exp_content) # Copyright (c) Megvii, Inc. and its affiliates.
return out_path
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "{data_dir_escaped}" # Where images are located
self.annotations_dir = "{annotations_parent_escaped}" # Where annotation JSONs are located
self.train_ann = "{train_ann}"
self.val_ann = "{val_ann}"
self.test_ann = "{test_ann}"
self.num_classes = {num_classes}
# Disable train2017 subdirectory - our images are directly in data_dir
self.name = ""
# Set data workers for training
self.data_num_workers = 8
'''
# Set pretrained_ckpt if transfer_learning is 'coco'
if training.transfer_learning and isinstance(training.transfer_learning, str) and training.transfer_learning.lower() == 'coco':
yolox_base_dir = '/home/kitraining/Yolox/YOLOX-main'
selected_model = training.selected_model.replace('.pth', '') if training.selected_model else ''
if selected_model:
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n"
# Format arrays
def format_value(val):
if isinstance(val, (list, tuple)):
# Convert float values to int for size-related parameters
formatted_items = []
for item in val:
# Convert to int if it's a whole number float
if isinstance(item, float) and item.is_integer():
formatted_items.append(str(int(item)))
else:
formatted_items.append(str(item))
return '(' + ', '.join(formatted_items) + ')'
elif isinstance(val, bool):
return str(val)
elif isinstance(val, str):
return f'"{val}"'
elif isinstance(val, float) and val.is_integer():
# Convert whole number floats to ints
return str(int(val))
else:
return str(val)
# Add all config parameters to exp
for key, value in config.items():
if key not in ['exp_name']: # exp_name is handled separately
exp_content += f" self.{key} = {format_value(value)}\n"
# Add get_dataset override using name parameter for image directory
exp_content += '''
def get_dataset(self, cache=False, cache_type="ram"):
"""Override to use name parameter for images directory"""
from yolox.data import COCODataset
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
# Use empty string for name since we have absolute paths in JSON
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.train_ann,
name="",
img_size=self.input_size,
preproc=self.preproc if hasattr(self, 'preproc') else None,
cache=cache,
cache_type=cache_type,
)
def get_eval_dataset(self, **kwargs):
"""Override eval dataset using name parameter"""
from yolox.data import COCODataset
testdev = kwargs.get("testdev", False)
legacy = kwargs.get("legacy", False)
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.val_ann if not testdev else self.test_ann,
name="",
img_size=self.test_size,
preproc=None, # No preprocessing for evaluation
)
'''
# Add exp_name at the end (uses dynamic path)
exp_content += f''' self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
'''
return exp_content
def save_yolox_inference_exp(training_id, out_path, options=None):
"""Save inference exp.py to custom path"""
exp_content = generate_yolox_inference_exp(training_id, options, use_base_config=False)
with open(out_path, 'w') as f:
f.write(exp_content)
return out_path

94
backend/services/push-yolox-exp.js
View File

@@ -1,48 +1,48 @@
const Training = require('../models/training.js'); const Training = require('../models/training.js');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
async function pushYoloxExpToDb(settings) { async function pushYoloxExpToDb(settings) {
// Normalize boolean and array fields for DB // Normalize boolean and array fields for DB
const normalized = { ...settings }; const normalized = { ...settings };
// Map 'act' from frontend to 'activation' for DB // Map 'act' from frontend to 'activation' for DB
if (normalized.act !== undefined) { if (normalized.act !== undefined) {
normalized.activation = normalized.act; normalized.activation = normalized.act;
delete normalized.act; delete normalized.act;
} }
// Convert 'on'/'off' to boolean for save_history_ckpt // Convert 'on'/'off' to boolean for save_history_ckpt
if (typeof normalized.save_history_ckpt === 'string') { if (typeof normalized.save_history_ckpt === 'string') {
normalized.save_history_ckpt = normalized.save_history_ckpt === 'on' ? true : false; normalized.save_history_ckpt = normalized.save_history_ckpt === 'on' ? true : false;
} }
// Convert comma-separated strings to arrays for input_size, test_size, mosaic_scale, mixup_scale // Convert comma-separated strings to arrays for input_size, test_size, mosaic_scale, mixup_scale
['input_size', 'test_size', 'mosaic_scale', 'mixup_scale'].forEach(key => { ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale'].forEach(key => {
if (typeof normalized[key] === 'string') { if (typeof normalized[key] === 'string') {
const arr = normalized[key].split(',').map(v => parseFloat(v.trim())); const arr = normalized[key].split(',').map(v => parseFloat(v.trim()));
normalized[key] = arr.length === 1 ? arr[0] : arr; normalized[key] = arr.length === 1 ? arr[0] : arr;
} }
}); });
// Find TrainingProjectDetails for this project // Find TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js'); const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id: normalized.project_id } }); const details = await TrainingProjectDetails.findOne({ where: { project_id: normalized.project_id } });
if (!details) throw new Error('TrainingProjectDetails not found for project_id ' + normalized.project_id); if (!details) throw new Error('TrainingProjectDetails not found for project_id ' + normalized.project_id);
normalized.project_details_id = details.id; normalized.project_details_id = details.id;
// Create DB row // Create DB row
const training = await Training.create(normalized); const training = await Training.create(normalized);
return training; return training;
} }
async function generateYoloxExpFromDb(trainingId) { async function generateYoloxExpFromDb(trainingId) {
// Fetch training row from DB // Fetch training row from DB
const training = await Training.findByPk(trainingId); const training = await Training.findByPk(trainingId);
if (!training) throw new Error('Training not found'); if (!training) throw new Error('Training not found');
// Template for exp.py // Template for exp.py
const expTemplate = `#!/usr/bin/env python3\n# Copyright (c) Megvii Inc. All rights reserved.\n\nimport os\nimport random\n\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nfrom .base_exp import BaseExp\n\n__all__ = [\"Exp\", \"check_exp_value\"]\n\nclass Exp(BaseExp):\n def __init__(self):\n super().__init__()\n\n # ---------------- model config ---------------- #\n self.num_classes = ${training.num_classes || 80}\n self.depth = ${training.depth || 1.00}\n self.width = ${training.width || 1.00}\n self.act = \"${training.activation || training.act || 'silu'}\"\n\n # ---------------- dataloader config ---------------- #\n self.data_num_workers = ${training.data_num_workers || 4}\n self.input_size = (${Array.isArray(training.input_size) ? training.input_size.join(', ') : '640, 640'})\n self.multiscale_range = ${training.multiscale_range || 5}\n self.data_dir = ${training.data_dir ? `\"${training.data_dir}\"` : 'None'}\n self.train_ann = \"${training.train_ann || 'instances_train2017.json'}\"\n self.val_ann = \"${training.val_ann || 'instances_val2017.json'}\"\n self.test_ann = \"${training.test_ann || 'instances_test2017.json'}\"\n\n # --------------- transform config ----------------- #\n self.mosaic_prob = ${training.mosaic_prob !== undefined ? training.mosaic_prob : 1.0}\n self.mixup_prob = ${training.mixup_prob !== undefined ? training.mixup_prob : 1.0}\n self.hsv_prob = ${training.hsv_prob !== undefined ? training.hsv_prob : 1.0}\n self.flip_prob = ${training.flip_prob !== undefined ? training.flip_prob : 0.5}\n self.degrees = ${training.degrees !== undefined ? training.degrees : 10.0}\n self.translate = ${training.translate !== undefined ? training.translate : 0.1}\n self.mosaic_scale = (${Array.isArray(training.mosaic_scale) ? training.mosaic_scale.join(', ') : '0.1, 2'})\n self.enable_mixup = ${training.enable_mixup !== undefined ? training.enable_mixup : true}\n self.mixup_scale = (${Array.isArray(training.mixup_scale) ? training.mixup_scale.join(', ') : '0.5, 1.5'})\n self.shear = ${training.shear !== undefined ? training.shear : 2.0}\n\n # -------------- training config --------------------- #\n self.warmup_epochs = ${training.warmup_epochs !== undefined ? training.warmup_epochs : 5}\n self.max_epoch = ${training.max_epoch !== undefined ? training.max_epoch : 300}\n self.warmup_lr = ${training.warmup_lr !== undefined ? training.warmup_lr : 0}\n self.min_lr_ratio = ${training.min_lr_ratio !== undefined ? training.min_lr_ratio : 0.05}\n self.basic_lr_per_img = ${training.basic_lr_per_img !== undefined ? training.basic_lr_per_img : 0.01 / 64.0}\n self.scheduler = \"${training.scheduler || 'yoloxwarmcos'}\"\n self.no_aug_epochs = ${training.no_aug_epochs !== undefined ? training.no_aug_epochs : 15}\n self.ema = ${training.ema !== undefined ? training.ema : true}\n self.weight_decay = ${training.weight_decay !== undefined ? training.weight_decay : 5e-4}\n self.momentum = ${training.momentum !== undefined ? training.momentum : 0.9}\n self.print_interval = ${training.print_interval !== undefined ? training.print_interval : 10}\n self.eval_interval = ${training.eval_interval !== undefined ? training.eval_interval : 10}\n self.save_history_ckpt = ${training.save_history_ckpt !== undefined ? training.save_history_ckpt : true}\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n\n # ----------------- testing config ------------------ #\n self.test_size = (${Array.isArray(training.test_size) ? training.test_size.join(', ') : '640, 640'})\n self.test_conf = ${training.test_conf !== undefined ? training.test_conf : 0.01}\n self.nmsthre = ${training.nmsthre !== undefined ? training.nmsthre : 0.65}\n\n # ... rest of the template ...\n\ndef check_exp_value(exp: Exp):\n h, w = exp.input_size\n assert h % 32 == 0 and w % 32 == 0, \"input size must be multiples of 32\"\n`; const expTemplate = `#!/usr/bin/env python3\n# Copyright (c) Megvii Inc. All rights reserved.\n\nimport os\nimport random\n\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nfrom .base_exp import BaseExp\n\n__all__ = [\"Exp\", \"check_exp_value\"]\n\nclass Exp(BaseExp):\n def __init__(self):\n super().__init__()\n\n # ---------------- model config ---------------- #\n self.num_classes = ${training.num_classes || 80}\n self.depth = ${training.depth || 1.00}\n self.width = ${training.width || 1.00}\n self.act = \"${training.activation || training.act || 'silu'}\"\n\n # ---------------- dataloader config ---------------- #\n self.data_num_workers = ${training.data_num_workers || 4}\n self.input_size = (${Array.isArray(training.input_size) ? training.input_size.join(', ') : '640, 640'})\n self.multiscale_range = ${training.multiscale_range || 5}\n self.data_dir = ${training.data_dir ? `\"${training.data_dir}\"` : 'None'}\n self.train_ann = \"${training.train_ann || 'instances_train2017.json'}\"\n self.val_ann = \"${training.val_ann || 'instances_val2017.json'}\"\n self.test_ann = \"${training.test_ann || 'instances_test2017.json'}\"\n\n # --------------- transform config ----------------- #\n self.mosaic_prob = ${training.mosaic_prob !== undefined ? training.mosaic_prob : 1.0}\n self.mixup_prob = ${training.mixup_prob !== undefined ? training.mixup_prob : 1.0}\n self.hsv_prob = ${training.hsv_prob !== undefined ? training.hsv_prob : 1.0}\n self.flip_prob = ${training.flip_prob !== undefined ? training.flip_prob : 0.5}\n self.degrees = ${training.degrees !== undefined ? training.degrees : 10.0}\n self.translate = ${training.translate !== undefined ? training.translate : 0.1}\n self.mosaic_scale = (${Array.isArray(training.mosaic_scale) ? training.mosaic_scale.join(', ') : '0.1, 2'})\n self.enable_mixup = ${training.enable_mixup !== undefined ? training.enable_mixup : true}\n self.mixup_scale = (${Array.isArray(training.mixup_scale) ? training.mixup_scale.join(', ') : '0.5, 1.5'})\n self.shear = ${training.shear !== undefined ? training.shear : 2.0}\n\n # -------------- training config --------------------- #\n self.warmup_epochs = ${training.warmup_epochs !== undefined ? training.warmup_epochs : 5}\n self.max_epoch = ${training.max_epoch !== undefined ? training.max_epoch : 300}\n self.warmup_lr = ${training.warmup_lr !== undefined ? training.warmup_lr : 0}\n self.min_lr_ratio = ${training.min_lr_ratio !== undefined ? training.min_lr_ratio : 0.05}\n self.basic_lr_per_img = ${training.basic_lr_per_img !== undefined ? training.basic_lr_per_img : 0.01 / 64.0}\n self.scheduler = \"${training.scheduler || 'yoloxwarmcos'}\"\n self.no_aug_epochs = ${training.no_aug_epochs !== undefined ? training.no_aug_epochs : 15}\n self.ema = ${training.ema !== undefined ? training.ema : true}\n self.weight_decay = ${training.weight_decay !== undefined ? training.weight_decay : 5e-4}\n self.momentum = ${training.momentum !== undefined ? training.momentum : 0.9}\n self.print_interval = ${training.print_interval !== undefined ? training.print_interval : 10}\n self.eval_interval = ${training.eval_interval !== undefined ? training.eval_interval : 10}\n self.save_history_ckpt = ${training.save_history_ckpt !== undefined ? training.save_history_ckpt : true}\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n\n # ----------------- testing config ------------------ #\n self.test_size = (${Array.isArray(training.test_size) ? training.test_size.join(', ') : '640, 640'})\n self.test_conf = ${training.test_conf !== undefined ? training.test_conf : 0.01}\n self.nmsthre = ${training.nmsthre !== undefined ? training.nmsthre : 0.65}\n\n # ... rest of the template ...\n\ndef check_exp_value(exp: Exp):\n h, w = exp.input_size\n assert h % 32 == 0 and w % 32 == 0, \"input size must be multiples of 32\"\n`;
// Save to file in output directory // Save to file in output directory
const outDir = path.join(__dirname, '../../', training.project_id ? `project_${training.project_id}/${trainingId}` : 'exp_files'); const outDir = path.join(__dirname, '../../', training.project_id ? `project_${training.project_id}/${trainingId}` : 'exp_files');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true }); if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
const filePath = path.join(outDir, 'exp.py'); const filePath = path.join(outDir, 'exp.py');
fs.writeFileSync(filePath, expTemplate); fs.writeFileSync(filePath, expTemplate);
return filePath; return filePath;
} }
module.exports = { pushYoloxExpToDb, generateYoloxExpFromDb }; module.exports = { pushYoloxExpToDb, generateYoloxExpFromDb };

205
backend/services/push_yolox_exp.py
View File

@@ -1,92 +1,113 @@
from models.training import Training from models.training import Training
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from database.database import db from models.TrainingSize import TrainingSize
from database.database import db
def push_yolox_exp_to_db(settings):
"""Save YOLOX settings to database""" def push_yolox_exp_to_db(settings):
normalized = dict(settings) """Save YOLOX settings to database"""
normalized = dict(settings)
# Map common frontend aliases to DB column names
alias_map = { # Map common frontend aliases to DB column names
'act': 'activation', alias_map = {
'nmsthre': 'nms_thre', 'act': 'activation',
'select_model': 'selected_model' 'nmsthre': 'nms_thre',
} 'select_model': 'selected_model'
for a, b in alias_map.items(): }
if a in normalized and b not in normalized: for a, b in alias_map.items():
normalized[b] = normalized.pop(a) if a in normalized and b not in normalized:
normalized[b] = normalized.pop(a)
# Convert 'on'/'off' or 'true'/'false' strings to boolean for known boolean fields
for bool_field in ['save_history_ckpt', 'ema', 'enable_mixup']: # Convert 'on'/'off' or 'true'/'false' strings to boolean for known boolean fields
if bool_field in normalized: for bool_field in ['save_history_ckpt', 'ema', 'enable_mixup']:
val = normalized[bool_field] if bool_field in normalized:
if isinstance(val, str): val = normalized[bool_field]
normalized[bool_field] = val.lower() in ('1', 'true', 'on') if isinstance(val, str):
else: normalized[bool_field] = val.lower() in ('1', 'true', 'on')
normalized[bool_field] = bool(val) else:
normalized[bool_field] = bool(val)
# Convert comma-separated strings to arrays for JSON fields
for key in ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale']: # Extract size arrays for separate TrainingSize table (3NF)
if key in normalized and isinstance(normalized[key], str): size_arrays = {}
parts = [p.strip() for p in normalized[key].split(',') if p.strip()] for key in ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale']:
try: if key in normalized:
arr = [float(p) for p in parts] if isinstance(normalized[key], str):
except Exception: parts = [p.strip() for p in normalized[key].split(',') if p.strip()]
arr = parts try:
normalized[key] = arr[0] if len(arr) == 1 else arr arr = [float(p) for p in parts]
except Exception:
# Ensure we have a TrainingProjectDetails row for project_id arr = parts
project_id = normalized.get('project_id') size_arrays[key] = arr[0] if len(arr) == 1 else (arr if isinstance(arr, list) else [arr])
if not project_id: elif isinstance(normalized[key], list):
raise Exception('Missing project_id in settings') size_arrays[key] = normalized[key]
details = TrainingProjectDetails.query.filter_by(project_id=project_id).first() elif normalized[key] is not None:
if not details: size_arrays[key] = [float(normalized[key])]
raise Exception(f'TrainingProjectDetails not found for project_id {project_id}') # Remove from normalized dict since it won't be stored in training table
normalized['project_details_id'] = details.id del normalized[key]
# Filter normalized to only columns that exist on the Training model # Ensure we have a TrainingProjectDetails row for project_id
valid_cols = {c.name: c for c in Training.__table__.columns} project_id = normalized.get('project_id')
filtered = {} if not project_id:
for k, v in normalized.items(): raise Exception('Missing project_id in settings')
if k in valid_cols: details = TrainingProjectDetails.query.filter_by(project_id=project_id).first()
col_type = valid_cols[k].type.__class__.__name__ if not details:
# Try to coerce types for numeric/boolean columns raise Exception(f'TrainingProjectDetails not found for project_id {project_id}')
try: normalized['project_details_id'] = details.id
if 'Integer' in col_type:
if v is None or v == '': # Filter normalized to only columns that exist on the Training model
filtered[k] = None valid_cols = {c.name: c for c in Training.__table__.columns}
else: filtered = {}
filtered[k] = int(float(v)) for k, v in normalized.items():
elif 'Float' in col_type: if k in valid_cols:
if v is None or v == '': col_type = valid_cols[k].type.__class__.__name__
filtered[k] = None # Try to coerce types for numeric/boolean columns
else: try:
filtered[k] = float(v) if 'Integer' in col_type:
elif 'Boolean' in col_type: if v is None or v == '':
if isinstance(v, str): filtered[k] = None
filtered[k] = v.lower() in ('1', 'true', 'on') else:
else: filtered[k] = int(float(v))
filtered[k] = bool(v) elif 'Float' in col_type:
elif 'JSON' in col_type: if v is None or v == '':
filtered[k] = v filtered[k] = None
elif 'LargeBinary' in col_type: else:
# If a file path was passed, store its bytes; otherwise store raw bytes filtered[k] = float(v)
if isinstance(v, str): elif 'Boolean' in col_type:
try: if isinstance(v, str):
filtered[k] = v.encode('utf-8') filtered[k] = v.lower() in ('1', 'true', 'on')
except Exception: else:
filtered[k] = None filtered[k] = bool(v)
else: elif 'LargeBinary' in col_type:
filtered[k] = v # If a file path was passed, store its bytes; otherwise store raw bytes
else: if isinstance(v, str):
filtered[k] = v try:
except Exception: filtered[k] = v.encode('utf-8')
# If conversion fails, just assign raw value except Exception:
filtered[k] = v filtered[k] = None
else:
# Create DB row filtered[k] = v
training = Training(**filtered) else:
db.session.add(training) filtered[k] = v
db.session.commit() except Exception:
# If conversion fails, just assign raw value
return training filtered[k] = v
# Create DB row
training = Training(**filtered)
db.session.add(training)
db.session.flush() # Get training.id
# Save size arrays to TrainingSize table (3NF)
for size_type, values in size_arrays.items():
if values and isinstance(values, list):
for order, value in enumerate(values):
size_record = TrainingSize(
training_id=training.id,
size_type=size_type,
value_order=order,
value=float(value)
)
db.session.add(size_record)
db.session.commit()
return training

240
backend/services/seed-label-studio.js
View File

@@ -1,120 +1,120 @@
const sequelize = require('../database/database.js'); const sequelize = require('../database/database.js');
const { Project, Img, Ann } = require('../models'); const { Project, Img, Ann } = require('../models');
const { fetchLableStudioProject, fetchProjectIdsAndTitles } = require('./fetch-labelstudio.js'); const { fetchLableStudioProject, fetchProjectIdsAndTitles } = require('./fetch-labelstudio.js');
const updateStatus = { running: false }; const updateStatus = { running: false };
async function seedLabelStudio() { async function seedLabelStudio() {
updateStatus.running = true; updateStatus.running = true;
console.log('Seeding started'); console.log('Seeding started');
try { try {
await sequelize.sync(); await sequelize.sync();
const projects = await fetchProjectIdsAndTitles(); const projects = await fetchProjectIdsAndTitles();
for (const project of projects) { for (const project of projects) {
console.log(`Processing project ${project.id} (${project.title})`); console.log(`Processing project ${project.id} (${project.title})`);
// Upsert project in DB // Upsert project in DB
await Project.upsert({ project_id: project.id, title: project.title }); await Project.upsert({ project_id: project.id, title: project.title });
// Fetch project data (annotations array) // Fetch project data (annotations array)
const data = await fetchLableStudioProject(project.id); const data = await fetchLableStudioProject(project.id);
if (!Array.isArray(data) || data.length === 0) { if (!Array.isArray(data) || data.length === 0) {
console.log(`No annotation data for project ${project.id}`); console.log(`No annotation data for project ${project.id}`);
continue; continue;
} }
// Remove old images and annotations for this project // Remove old images and annotations for this project
const oldImages = await Img.findAll({ where: { project_id: project.id } }); const oldImages = await Img.findAll({ where: { project_id: project.id } });
const oldImageIds = oldImages.map(img => img.image_id); const oldImageIds = oldImages.map(img => img.image_id);
if (oldImageIds.length > 0) { if (oldImageIds.length > 0) {
await Ann.destroy({ where: { image_id: oldImageIds } }); await Ann.destroy({ where: { image_id: oldImageIds } });
await Img.destroy({ where: { project_id: project.id } }); await Img.destroy({ where: { project_id: project.id } });
console.log(`Deleted ${oldImageIds.length} old images and their annotations for project ${project.id}`); console.log(`Deleted ${oldImageIds.length} old images and their annotations for project ${project.id}`);
} }
// Prepare arrays // Prepare arrays
const imagesBulk = []; const imagesBulk = [];
const annsBulk = []; const annsBulk = [];
for (const ann of data) { for (const ann of data) {
// Extract width/height // Extract width/height
let width = null; let width = null;
let height = null; let height = null;
if (Array.isArray(ann.label_rectangles) && ann.label_rectangles.length > 0) { if (Array.isArray(ann.label_rectangles) && ann.label_rectangles.length > 0) {
width = ann.label_rectangles[0].original_width; width = ann.label_rectangles[0].original_width;
height = ann.label_rectangles[0].original_height; height = ann.label_rectangles[0].original_height;
} else if (Array.isArray(ann.label) && ann.label.length > 0 && ann.label[0].original_width && ann.label[0].original_height) { } else if (Array.isArray(ann.label) && ann.label.length > 0 && ann.label[0].original_width && ann.label[0].original_height) {
width = ann.label[0].original_width; width = ann.label[0].original_width;
height = ann.label[0].original_height; height = ann.label[0].original_height;
} }
// Only push image and annotations if width and height are valid // Only push image and annotations if width and height are valid
if (width && height) { if (width && height) {
imagesBulk.push({ imagesBulk.push({
project_id: project.id, project_id: project.id,
image_path: ann.image, image_path: ann.image,
width, width,
height height
}); });
// Handle multiple annotations per image // Handle multiple annotations per image
if (Array.isArray(ann.label_rectangles)) { if (Array.isArray(ann.label_rectangles)) {
for (const ann_detail of ann.label_rectangles) { for (const ann_detail of ann.label_rectangles) {
annsBulk.push({ annsBulk.push({
image_path: ann.image, image_path: ann.image,
x: (ann_detail.x * width) / 100, x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100, y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100, width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100, height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown') Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
}); });
} }
} else if (Array.isArray(ann.label)) { } else if (Array.isArray(ann.label)) {
for (const ann_detail of ann.label) { for (const ann_detail of ann.label) {
annsBulk.push({ annsBulk.push({
image_path: ann.image, image_path: ann.image,
x: (ann_detail.x * width) / 100, x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100, y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100, width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100, height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown') Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
}); });
} }
} }
} }
} }
// 1) Insert images and get generated IDs // 1) Insert images and get generated IDs
const insertedImages = await Img.bulkCreate(imagesBulk, { returning: true }); const insertedImages = await Img.bulkCreate(imagesBulk, { returning: true });
// 2) Map image_path -> image_id // 2) Map image_path -> image_id
const imageMap = {}; const imageMap = {};
for (const img of insertedImages) { for (const img of insertedImages) {
imageMap[img.image_path] = img.image_id; imageMap[img.image_path] = img.image_id;
} }
// 3) Assign correct image_id to each annotation // 3) Assign correct image_id to each annotation
for (const ann of annsBulk) { for (const ann of annsBulk) {
ann.image_id = imageMap[ann.image_path]; ann.image_id = imageMap[ann.image_path];
delete ann.image_path; // cleanup delete ann.image_path; // cleanup
} }
// 4) Insert annotations // 4) Insert annotations
await Ann.bulkCreate(annsBulk); await Ann.bulkCreate(annsBulk);
console.log(`Inserted ${imagesBulk.length} images and ${annsBulk.length} annotations for project ${project.id}`); console.log(`Inserted ${imagesBulk.length} images and ${annsBulk.length} annotations for project ${project.id}`);
} }
console.log('Seeding done'); console.log('Seeding done');
return { success: true, message: 'Data inserted successfully!' }; return { success: true, message: 'Data inserted successfully!' };
} catch (error) { } catch (error) {
console.error('Error inserting data:', error); console.error('Error inserting data:', error);
return { success: false, message: error.message }; return { success: false, message: error.message };
} finally { } finally {
updateStatus.running = false; updateStatus.running = false;
console.log('updateStatus.running set to false'); console.log('updateStatus.running set to false');
} }
} }
module.exports = { seedLabelStudio, updateStatus }; module.exports = { seedLabelStudio, updateStatus };

298
backend/services/seed_label_studio.py
View File

@@ -1,149 +1,149 @@
from database.database import db from database.database import db
from models.LabelStudioProject import LabelStudioProject from models.LabelStudioProject import LabelStudioProject
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
from services.fetch_labelstudio import fetch_label_studio_project, fetch_project_ids_and_titles from services.fetch_labelstudio import fetch_label_studio_project, fetch_project_ids_and_titles
update_status = {"running": False} update_status = {"running": False}
def seed_label_studio(): def seed_label_studio():
"""Seed database with Label Studio project data""" """Seed database with Label Studio project data"""
update_status["running"] = True update_status["running"] = True
print('Seeding started') print('Seeding started')
try: try:
projects = fetch_project_ids_and_titles() projects = fetch_project_ids_and_titles()
for project in projects: for project in projects:
print(f"Processing project {project['id']} ({project['title']})") print(f"Processing project {project['id']} ({project['title']})")
# Upsert project in DB # Upsert project in DB
existing_project = LabelStudioProject.query.filter_by(project_id=project['id']).first() existing_project = LabelStudioProject.query.filter_by(project_id=project['id']).first()
if existing_project: if existing_project:
existing_project.title = project['title'] existing_project.title = project['title']
else: else:
new_project = LabelStudioProject(project_id=project['id'], title=project['title']) new_project = LabelStudioProject(project_id=project['id'], title=project['title'])
db.session.add(new_project) db.session.add(new_project)
db.session.commit() db.session.commit()
# Fetch project data (annotations array) # Fetch project data (annotations array)
data = fetch_label_studio_project(project['id']) data = fetch_label_studio_project(project['id'])
if not isinstance(data, list) or len(data) == 0: if not isinstance(data, list) or len(data) == 0:
print(f"No annotation data for project {project['id']}") print(f"No annotation data for project {project['id']}")
continue continue
# Remove old images and annotations for this project # Remove old images and annotations for this project
old_images = Image.query.filter_by(project_id=project['id']).all() old_images = Image.query.filter_by(project_id=project['id']).all()
old_image_ids = [img.image_id for img in old_images] old_image_ids = [img.image_id for img in old_images]
if old_image_ids: if old_image_ids:
Annotation.query.filter(Annotation.image_id.in_(old_image_ids)).delete(synchronize_session=False) Annotation.query.filter(Annotation.image_id.in_(old_image_ids)).delete(synchronize_session=False)
Image.query.filter_by(project_id=project['id']).delete() Image.query.filter_by(project_id=project['id']).delete()
db.session.commit() db.session.commit()
print(f"Deleted {len(old_image_ids)} old images and their annotations for project {project['id']}") print(f"Deleted {len(old_image_ids)} old images and their annotations for project {project['id']}")
# Prepare arrays # Prepare arrays
images_bulk = [] images_bulk = []
anns_bulk = [] anns_bulk = []
for ann in data: for ann in data:
# Extract width/height # Extract width/height
width = None width = None
height = None height = None
if isinstance(ann.get('label_rectangles'), list) and len(ann['label_rectangles']) > 0: if isinstance(ann.get('label_rectangles'), list) and len(ann['label_rectangles']) > 0:
width = ann['label_rectangles'][0].get('original_width') width = ann['label_rectangles'][0].get('original_width')
height = ann['label_rectangles'][0].get('original_height') height = ann['label_rectangles'][0].get('original_height')
elif isinstance(ann.get('label'), list) and len(ann['label']) > 0: elif isinstance(ann.get('label'), list) and len(ann['label']) > 0:
if ann['label'][0].get('original_width') and ann['label'][0].get('original_height'): if ann['label'][0].get('original_width') and ann['label'][0].get('original_height'):
width = ann['label'][0]['original_width'] width = ann['label'][0]['original_width']
height = ann['label'][0]['original_height'] height = ann['label'][0]['original_height']
# Only process if width and height are valid # Only process if width and height are valid
if width and height: if width and height:
image_data = { image_data = {
'project_id': project['id'], 'project_id': project['id'],
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'width': width, 'width': int(width), # Ensure integer
'height': height 'height': int(height) # Ensure integer
} }
images_bulk.append(image_data) images_bulk.append(image_data)
# Handle multiple annotations per image # Handle multiple annotations per image
if isinstance(ann.get('label_rectangles'), list): if isinstance(ann.get('label_rectangles'), list):
for ann_detail in ann['label_rectangles']: for ann_detail in ann['label_rectangles']:
# Get label safely # Get label safely
rectanglelabels = ann_detail.get('rectanglelabels', []) rectanglelabels = ann_detail.get('rectanglelabels', [])
if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0: if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0:
label = rectanglelabels[0] label = rectanglelabels[0]
elif isinstance(rectanglelabels, str): elif isinstance(rectanglelabels, str):
label = rectanglelabels label = rectanglelabels
else: else:
label = 'unknown' label = 'unknown'
ann_data = { ann_data = {
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'x': (ann_detail['x'] * width) / 100, 'x': (ann_detail['x'] * width) / 100,
'y': (ann_detail['y'] * height) / 100, 'y': (ann_detail['y'] * height) / 100,
'width': (ann_detail['width'] * width) / 100, 'width': (ann_detail['width'] * width) / 100,
'height': (ann_detail['height'] * height) / 100, 'height': (ann_detail['height'] * height) / 100,
'Label': label 'Label': label
} }
anns_bulk.append(ann_data) anns_bulk.append(ann_data)
elif isinstance(ann.get('label'), list): elif isinstance(ann.get('label'), list):
for ann_detail in ann['label']: for ann_detail in ann['label']:
# Get label safely # Get label safely
rectanglelabels = ann_detail.get('rectanglelabels', []) rectanglelabels = ann_detail.get('rectanglelabels', [])
if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0: if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0:
label = rectanglelabels[0] label = rectanglelabels[0]
elif isinstance(rectanglelabels, str): elif isinstance(rectanglelabels, str):
label = rectanglelabels label = rectanglelabels
else: else:
label = 'unknown' label = 'unknown'
ann_data = { ann_data = {
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'x': (ann_detail['x'] * width) / 100, 'x': (ann_detail['x'] * width) / 100,
'y': (ann_detail['y'] * height) / 100, 'y': (ann_detail['y'] * height) / 100,
'width': (ann_detail['width'] * width) / 100, 'width': (ann_detail['width'] * width) / 100,
'height': (ann_detail['height'] * height) / 100, 'height': (ann_detail['height'] * height) / 100,
'Label': label 'Label': label
} }
anns_bulk.append(ann_data) anns_bulk.append(ann_data)
# Insert images and get generated IDs # Insert images and get generated IDs
inserted_images = [] inserted_images = []
for img_data in images_bulk: for img_data in images_bulk:
new_image = Image(**img_data) new_image = Image(**img_data)
db.session.add(new_image) db.session.add(new_image)
db.session.flush() # Flush to get the ID db.session.flush() # Flush to get the ID
inserted_images.append(new_image) inserted_images.append(new_image)
db.session.commit() db.session.commit()
# Map image_path -> image_id # Map image_path -> image_id
image_map = {img.image_path: img.image_id for img in inserted_images} image_map = {img.image_path: img.image_id for img in inserted_images}
# Assign correct image_id to each annotation # Assign correct image_id to each annotation
for ann_data in anns_bulk: for ann_data in anns_bulk:
ann_data['image_id'] = image_map.get(ann_data['image_path']) ann_data['image_id'] = image_map.get(ann_data['image_path'])
del ann_data['image_path'] del ann_data['image_path']
# Insert annotations # Insert annotations
for ann_data in anns_bulk: for ann_data in anns_bulk:
new_annotation = Annotation(**ann_data) new_annotation = Annotation(**ann_data)
db.session.add(new_annotation) db.session.add(new_annotation)
db.session.commit() db.session.commit()
print(f"Inserted {len(images_bulk)} images and {len(anns_bulk)} annotations for project {project['id']}") print(f"Inserted {len(images_bulk)} images and {len(anns_bulk)} annotations for project {project['id']}")
print('Seeding done') print('Seeding done')
return {'success': True, 'message': 'Data inserted successfully!'} return {'success': True, 'message': 'Data inserted successfully!'}
except Exception as error: except Exception as error:
print(f'Error inserting data: {error}') print(f'Error inserting data: {error}')
db.session.rollback() db.session.rollback()
return {'success': False, 'message': str(error)} return {'success': False, 'message': str(error)}
finally: finally:
update_status["running"] = False update_status["running"] = False
print('updateStatus.running set to false') print('updateStatus.running set to false')

71
backend/services/settings_service.py Normal file
View File

@@ -0,0 +1,71 @@
"""
Settings Service - Manages global application settings
"""
from models.Settings import Settings
from database.database import db
def get_setting(key, default=None):
"""Get a setting value by key"""
setting = Settings.query.filter_by(key=key).first()
return setting.value if setting else default
def set_setting(key, value, description=None):
"""Set a setting value"""
setting = Settings.query.filter_by(key=key).first()
if setting:
setting.value = value
if description:
setting.description = description
else:
setting = Settings(key=key, value=value, description=description)
db.session.add(setting)
db.session.commit()
return setting
def get_all_settings():
"""Get all settings as a dictionary"""
settings = Settings.query.all()
return {s.key: s.value for s in settings}
def get_all_settings_detailed():
"""Get all settings with full details"""
settings = Settings.query.all()
return [s.to_dict() for s in settings]
def initialize_default_settings():
"""Initialize default settings if they don't exist"""
defaults = {
'labelstudio_api_url': {
'value': 'http://192.168.1.19:8080/api',
'description': 'Label Studio API URL'
},
'labelstudio_api_token': {
'value': 'c1cef980b7c73004f4ee880a42839313b863869f',
'description': 'Label Studio API Token'
},
'yolox_path': {
'value': 'C:/YOLOX',
'description': 'Path to YOLOX installation directory'
},
'yolox_venv_path': {
'value': '/home/kitraining/Yolox/yolox_venv/bin/activate',
'description': 'Path to YOLOX virtual environment activation script'
},
'yolox_output_path': {
'value': './backend',
'description': 'Output folder for YOLOX experiment files and JSONs'
},
'yolox_data_dir': {
'value': '/home/kitraining/To_Annotate/',
'description': 'Data directory path for YOLOX training (where images are located)'
}
}
for key, data in defaults.items():
existing = Settings.query.filter_by(key=key).first()
if not existing:
setting = Settings(key=key, value=data['value'], description=data['description'])
db.session.add(setting)
db.session.commit()

28
backend/start.py
View File

@@ -1,14 +1,14 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Start the Flask backend server Start the Flask backend server
""" """
import sys import sys
import os import os
# Add the backend directory to Python path # Add the backend directory to Python path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from app import app from app import app
if __name__ == '__main__': if __name__ == '__main__':
app.run(host='0.0.0.0', port=3000, debug=True) app.run(host='0.0.0.0', port=3000, debug=True)

47
backend/test/7/exp.py Normal file
View File

@@ -0,0 +1,47 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_3_train.json"
self.val_ann = "coco_project_3_valid.json"
self.test_ann = "coco_project_3_test.json"
self.num_classes = 2
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

BIN
documentation/Projektdoku.pdf Normal file
View File

Binary file not shown.

764
documentation/Projektdokumentation.md Normal file
View File

@@ -0,0 +1,764 @@
# Projektdokumentation: mb ai Trainer
**Autor:** [Ihr Name]
**Datum:** 1. Dezember 2025
**Projekt:** mb ai Trainer - Webbasiertes YOLOX Trainings-Management-System
---
## Inhaltsverzeichnis
1. [Einleitung](#1-einleitung)
- 1.1 [Das Unternehmen](#11-das-unternehmen)
- 1.2 [Projektumfeld](#12-projektumfeld)
- 1.3 [Projektziel](#13-projektziel)
- 1.4 [Projektbegründung](#14-projektbegründung)
2. [Grundlagen](#2-grundlagen)
- 2.1 [Verwendete Technologien](#21-verwendete-technologien)
- 2.2 [YOLOX Object Detection](#22-yolox-object-detection)
- 2.3 [Label Studio](#23-label-studio)
3. [Ist-Analyse](#3-ist-analyse)
- 3.1 [Ausgangssituation](#31-ausgangssituation)
- 3.2 [Problembeschreibung](#32-problembeschreibung)
4. [Soll-Konzept](#4-soll-konzept)
- 4.1 [Anforderungsanalyse](#41-anforderungsanalyse)
- 4.2 [Funktionale Anforderungen](#42-funktionale-anforderungen)
- 4.3 [Nicht-funktionale Anforderungen](#43-nicht-funktionale-anforderungen)
5. [Systemarchitektur](#5-systemarchitektur)
- 5.1 [Gesamtarchitektur](#51-gesamtarchitektur)
- 5.2 [Backend-Architektur](#52-backend-architektur)
- 5.3 [Frontend-Architektur](#53-frontend-architektur)
- 5.4 [Datenbankdesign](#54-datenbankdesign)
- 5.5 [API-Design](#55-api-design)
6. [Implementierung](#6-implementierung)
- 6.1 [Backend-Entwicklung](#61-backend-entwicklung)
- 6.2 [Frontend-Entwicklung](#62-frontend-entwicklung)
- 6.3 [Datenbank-Implementierung](#63-datenbank-implementierung)
- 6.4 [Label Studio Integration](#64-label-studio-integration)
- 6.5 [Transfer Learning System](#65-transfer-learning-system)
- 6.6 [Training-Workflow](#66-training-workflow)
7. [Testing und Qualitätssicherung](#7-testing-und-qualitätssicherung)
- 7.1 [Teststrategie](#71-teststrategie)
- 7.2 [Unit Tests](#72-unit-tests)
- 7.3 [Integrationstests](#73-integrationstests)
- 7.4 [Systemtests](#74-systemtests)
8. [Deployment und Betrieb](#8-deployment-und-betrieb)
- 8.1 [Systemanforderungen](#81-systemanforderungen)
- 8.2 [Installation](#82-installation)
- 8.3 [Konfiguration](#83-konfiguration)
9. [Fazit und Ausblick](#9-fazit-und-ausblick)
- 9.1 [Zusammenfassung](#91-zusammenfassung)
- 9.2 [Erreichte Ziele](#92-erreichte-ziele)
- 9.3 [Lessons Learned](#93-lessons-learned)
- 9.4 [Ausblick](#94-ausblick)
10. [Anhang](#10-anhang)
- 10.1 [Glossar](#101-glossar)
- 10.2 [Literaturverzeichnis](#102-literaturverzeichnis)
- 10.3 [Abbildungsverzeichnis](#103-abbildungsverzeichnis)
- 10.4 [Code-Beispiele](#104-code-beispiele)
---
## 1. Einleitung
### 1.1 Das Unternehmen
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Firmenbeschreibung, Standort, Mitarbeiter, Branche, etc.]
### 1.2 Projektumfeld
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Übergeordnetes Projekt, Teamstruktur, Projektorganisation, etc.]
### 1.3 Projektziel
Der **mb ai Trainer** ist eine webbasierte Anwendung zur vollständigen Verwaltung und Durchführung von YOLOX-Modelltrainings im Bereich Computer Vision. Das System wurde entwickelt, um den gesamten Machine Learning Workflow - von der Datenannotation über das Training bis zur Modell-Evaluierung - in einer einheitlichen Plattform zu integrieren.
**Hauptziele des Projekts:**
- Entwicklung einer benutzerfreundlichen Weboberfläche zur Verwaltung von Trainingsprojekten
- Integration von Label Studio für professionelle Bildannotation
- Implementierung einer robusten Datenverwaltung mit MySQL
- Automatisierung des YOLOX-Trainingsprozesses
- Bereitstellung eines Transfer Learning Systems mit vordefinierten Basis-Konfigurationen
- Zentrale Verwaltung aller Trainingsparameter und -verläufe
### 1.4 Projektbegründung
Im Bereich des Machine Learning für Computer Vision existieren zahlreiche isolierte Tools für verschiedene Aufgaben (Annotation, Training, Evaluierung). Dies führt zu:
- **Hohem Verwaltungsaufwand:** Daten müssen zwischen verschiedenen Tools transferiert werden
- **Fehlerpotential:** Manuelle Konfiguration von Trainingsskripten ist fehleranfällig
- **Keine Nachvollziehbarkeit:** Trainingsverläufe und verwendete Parameter sind nicht zentral dokumentiert
- **Steile Lernkurve:** Neue Teammitglieder müssen mehrere Tools erlernen
Der mb ai Trainer löst diese Probleme durch eine integrierte Plattform, die alle notwendigen Funktionen vereint.
---
## 2. Grundlagen
### 2.1 Verwendete Technologien
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Grundlegende Technologien, allgemeiner Kontext]
#### 2.1.1 Backend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **Python** | 3.x | Hauptsprache für Backend-Entwicklung |
| **Flask** | 3.0.0 | Lightweight Web-Framework für REST API |
| **Flask-CORS** | Latest | Cross-Origin Resource Sharing Support |
| **Flask-SQLAlchemy** | Latest | ORM-Integration in Flask |
| **SQLAlchemy** | Latest | Object-Relational Mapper für Datenbankzugriff |
| **PyMySQL** | Latest | MySQL Database Connector |
| **MySQL** | 8.0+ | Relationale Datenbank für persistente Speicherung |
| **requests** | Latest | HTTP-Client für externe API-Calls (Label Studio) |
**Begründung der Technologiewahl - Backend:**
- **Python:** Dominante Sprache im ML-Bereich, umfangreiches Ökosystem
- **Flask:** Lightweight, flexibel, kein unnötiger Overhead
- **SQLAlchemy:** Mächtiges ORM mit ausgezeichneter MySQL-Unterstützung
- **MySQL:** Bewährtes RDBMS, gut für strukturierte Trainingsdaten
#### 2.1.2 Frontend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **HTML5** | - | Strukturierung der Weboberfläche |
| **CSS3** | - | Styling, Layout, Responsive Design |
| **Vanilla JavaScript** | ES6+ | Client-seitige Logik ohne Frameworks |
**Begründung der Technologiewahl - Frontend:**
- **Vanilla JavaScript:** Keine Framework-Abhängigkeiten, volle Kontrolle, geringe Dateigröße
- **Kein Build-System:** Einfache Entwicklung und Deployment
- **Moderne ES6+ Features:** Async/Await, Fetch API, Arrow Functions
#### 2.1.3 Externe Services und Tools
| Service/Tool | Verwendung |
|--------------|------------|
| **Label Studio** | Open-Source Annotationstool für Bilddaten |
| **YOLOX** | State-of-the-art Object Detection Framework |
### 2.2 YOLOX Object Detection
YOLOX ist ein modernes, hochperformantes Object Detection Framework, das auf der YOLO-Familie aufbaut.
**Kernmerkmale:**
- **Anchor-free:** Keine vordefinierten Anchor-Boxen notwendig
- **Decoupled Head:** Getrennte Heads für Klassifikation und Lokalisierung
- **SimOTA:** Optimiertes Label Assignment
- **Verschiedene Modellgrößen:** YOLOX-S, -M, -L, -X für unterschiedliche Anforderungen
**Modell-Varianten:**
| Modell | Depth | Width | Parameter | Use Case |
|--------|-------|-------|-----------|----------|
| YOLOX-S | 0.33 | 0.50 | ~9M | Schnelle Inferenz, Edge Devices |
| YOLOX-M | 0.67 | 0.75 | ~25M | Balance Performance/Speed |
| YOLOX-L | 1.00 | 1.00 | ~54M | Hohe Genauigkeit |
| YOLOX-X | 1.33 | 1.25 | ~99M | Maximale Genauigkeit |
### 2.3 Label Studio
Label Studio ist eine Open-Source Daten-Annotationsplattform, die verschiedene Datentypen unterstützt.
**Funktionen:**
- **Multi-Format Support:** Bilder, Text, Audio, Video
- **Flexible Annotation:** Bounding Boxes, Polygone, Segmentierung
- **REST API:** Programmatischer Zugriff auf Projekte und Annotationen
- **Collaboration:** Multi-User Support für Team-Annotationen
**Integration im mb ai Trainer:**
- Import von Bildern aus Label Studio Projekten
- Synchronisation von Annotationen in MySQL-Datenbank
- Export von Annotationen im COCO JSON Format für YOLOX-Training
---
## 3. Ist-Analyse
### 3.1 Ausgangssituation
Vor der Entwicklung des mb ai Trainer existierte ein System basierend auf Node.js, Express und Sequelize ORM. Dieses System wies folgende Charakteristika auf:
**Technischer Stack (Alt):**
- Backend: Node.js mit Express.js
- ORM: Sequelize
- Datenbank: MySQL
- Frontend: HTML/CSS/JavaScript (static)
**Funktionsumfang (Alt):**
- Grundlegende Projektverwaltung
- Einfache Label Studio Anbindung
- Manuelle YOLOX-Konfiguration
### 3.2 Problembeschreibung
Das bestehende System hatte mehrere Limitierungen:
**1. Technologische Inkonsistenz**
- Backend in JavaScript, ML-Framework in Python
- Komplexe Interprozesskommunikation notwendig
- Doppelte Abhängigkeiten (Node.js + Python)
**2. Fehlende Funktionalität**
- Kein Transfer Learning Support
- Keine Basis-Konfigurationen für YOLOX-Modelle
- Manuelle Parameterkonfiguration fehleranfällig
- Keine Validierung von Trainingsparametern
**3. Wartbarkeit**
- Zwei getrennte Technologie-Stacks erschweren Wartung
- Keine einheitliche Codebasis
- Hoher Einarbeitungsaufwand für neue Entwickler
**4. Benutzerfreundlichkeit**
- Keine visuelle Unterscheidung zwischen Pflicht- und optionalen Parametern
- Keine Hilfestellungen für YOLOX-Anfänger
- Kein Schutz vor Fehlkonfigurationen
**Entscheidung:** Komplette Migration des Backends zu Python mit Erweiterung um Transfer Learning Funktionalität.
---
## 4. Soll-Konzept
### 4.1 Anforderungsanalyse
Aus der Ist-Analyse wurden folgende Anforderungen abgeleitet:
### 4.2 Funktionale Anforderungen
#### FA-1: Projektverwaltung
- Anlegen, Bearbeiten, Löschen von Trainingsprojekten
- Upload von Projektbildern
- Verknüpfung mit Label Studio Projekten
#### FA-2: Label Studio Integration
- Abruf von Label Studio Projekten via REST API
- Import von Annotationen
- Synchronisation von Bildern und Annotations in MySQL
- Export im COCO JSON Format
#### FA-3: Training-Konfiguration
- Auswahl von YOLOX-Modellen (S, M, L, X)
- Konfiguration aller Trainingsparameter
- Drei Modi: "Train from scratch", "Train on coco", "Custom weights"
- Validierung von Eingabeparametern
#### FA-4: Transfer Learning System
- Vordefinierte Basis-Konfigurationen für jedes YOLOX-Modell
- Automatisches Laden bei "Train on coco" Modus
- Schutz von Basis-Parametern vor Überschreibung
- Visuelle Kennzeichnung geschützter Felder
#### FA-5: Training-Durchführung
- Generierung von YOLOX exp.py Dateien
- Start des Trainingsprozesses
- Logging von Trainingsverläufen
#### FA-6: Datenbank-Management
- Persistente Speicherung aller Projekte
- Speicherung von Trainingseinstellungen
- Versionierung von Konfigurationen
### 4.3 Nicht-funktionale Anforderungen
#### NFA-1: Performance
- Laden von Basis-Konfigurationen < 100ms
- Label Studio Sync < 5s für 1000 Bilder
- API Response Times < 200ms (95th Percentile)
#### NFA-2: Usability
- Intuitive Benutzeroberfläche ohne Schulung nutzbar
- Klare visuelle Unterscheidung (enabled/disabled Felder)
- Hilfreiche Tooltips und Fehlermeldungen
#### NFA-3: Wartbarkeit
- Modularer Aufbau (Services, Models, Routes getrennt)
- Umfassende Code-Dokumentation
- Einheitlicher Technologie-Stack (Python)
#### NFA-4: Erweiterbarkeit
- Neue YOLOX-Modelle ohne Code-Änderung hinzufügbar
- Basis-Konfigurationen als separate Dateien
- Plugin-Architektur für zukünftige Frameworks
#### NFA-5: Zuverlässigkeit
- Fehlerbehandlung bei API-Calls
- Transaktionale Datenbankoperationen
- Rollback bei fehlgeschlagenen Operationen
---
## 5. Systemarchitektur
### 3.1 Anforderungsanalyse
#### 3.1.1 Funktionale Anforderungen
Das Transfer Learning Feature sollte folgende Funktionen bereitstellen:
1. **Basis-Konfiguration Verwaltung**
- Vordefinierte Konfigurationen für YOLOX-S, -M, -L, -X Modelle
- Speicherung als Python-Klassen für einfache Wartbarkeit
- Dynamisches Laden zur Laufzeit
2. **Parameter-Schutz**
- Bestimmte Parameter (depth, width, etc.) aus Basis-Konfiguration
- Benutzer kann geschützte Parameter nicht überschreiben
- Visuelle Kennzeichnung im Frontend
3. **Merge-Strategie**
- Basis-Konfiguration als Grundlage
- Benutzer-definierte Werte für nicht-geschützte Parameter
- Fallback auf Default-Werte
#### 3.1.2 Nicht-funktionale Anforderungen
- **Performance:** Basis-Konfiguration muss < 100ms laden
- **Usability:** Klare visuelle Unterscheidung zwischen geschützten und editierbaren Feldern
- **Wartbarkeit:** Neue Modelle sollen einfach hinzugefügt werden können
- **Erweiterbarkeit:** System muss für zukünftige YOLOX-Versionen erweiterbar sein
### 3.2 Systemarchitektur
#### 3.2.1 Backend-Architektur
Das Backend des mb ai Trainer ist als modulare Python/Flask-Anwendung aufgebaut:
```
backend/
├── app.py # Hauptanwendung, Flask-Server
├── requirements.txt # Python-Abhängigkeiten
├── database/
│ ├── database.py # SQLAlchemy Konfiguration
│ └── myapp.sql # Datenbankschema
├── models/ # SQLAlchemy Models
│ ├── TrainingProject.py # Trainingsprojekte
│ ├── TrainingProjectDetails.py
│ ├── training.py # Training-Settings
│ ├── LabelStudioProject.py
│ ├── Images.py
│ └── Annotation.js
├── data/ # Basis-Konfigurationen (NEU)
│ ├── yolox_s.py # YOLOX-Small Base Config
│ ├── yolox_m.py # YOLOX-Medium Base Config
│ ├── yolox_l.py # YOLOX-Large Base Config
│ ├── yolox_x.py # YOLOX-XLarge Base Config
│ └── README.md # Dokumentation
├── services/ # Business Logic
│ ├── generate_yolox_exp.py # Exp-Generierung mit Base Config
│ ├── fetch_labelstudio.py # Label Studio API Client
│ ├── seed_label_studio.py # Daten-Seeding
│ ├── generate_json_yolox.py # COCO JSON Export
│ └── push_yolox_exp.py # Settings speichern
└── routes/
└── api.py # REST API Endpoints
```
**Datenfluss:**
1. **Projektanlage:** Frontend → POST `/api/training-projects` → MySQL
2. **Label Studio Sync:** Backend → Label Studio API → MySQL (Images, Annotations)
3. **Transfer Learning:**
- Frontend wählt "Train on coco" + Model
- GET `/api/base-config/<model_name>` → Basis-Config laden
- Frontend sperrt geschützte Felder
4. **Training Start:**
- POST `/api/yolox-settings` → Settings speichern
- POST `/api/start-yolox-training` → Exp.py generieren
- YOLOX Training starten
#### 3.2.2 Basis-Konfiguration Format
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters (protected)
depth = 0.33
width = 0.50
# Training parameters (protected)
scheduler = "yoloxwarmcos"
activation = "silu"
# ... weitere Parameter
```
#### 3.2.3 API Design
**Endpoint:** `GET /api/base-config/<model_name>`
**Response:**
```json
{
"depth": 0.33,
"width": 0.50,
"activation": "silu",
"scheduler": "yoloxwarmcos",
// ... weitere Parameter
}
```
### 3.3 Implementierung
#### 3.3.1 Backend: Dynamic Module Loading
Die Basis-Konfigurationen werden zur Laufzeit dynamisch geladen:
```python
import importlib.util
import os
def load_base_config(selected_model):
"""Dynamically load base configuration for selected model"""
model_map = {
'yolox-s': 'yolox_s.py',
'yolox-m': 'yolox_m.py',
'yolox-l': 'yolox_l.py',
'yolox-x': 'yolox_x.py'
}
file_name = model_map.get(selected_model.lower())
if not file_name:
return None
base_dir = os.path.join(os.path.dirname(__file__), '..', 'data')
file_path = os.path.join(base_dir, file_name)
# Dynamic import
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module.BaseExp
```
**Vorteile:**
- Keine hardcodierten Imports
- Neue Modelle können ohne Code-Änderung hinzugefügt werden
- Klare Trennung von Konfiguration und Logik
#### 3.3.2 Backend: Configuration Merging
Die Merge-Strategie kombiniert drei Quellen:
```python
def generate_yolox_inference_exp(training_id, options, use_base_config=True):
"""Generate exp.py with base config support"""
# 1. Load base configuration
base_config = load_base_config(selected_model) if use_base_config else None
# 2. Load user-defined values from database
training = db.session.get(Training, training_id)
# 3. Merge with priority: base > user > defaults
final_config = {}
# Protected parameters from base config
protected_params = ['depth', 'width', 'activation', 'scheduler', ...]
for param in protected_params:
if base_config and hasattr(base_config, param):
final_config[param] = getattr(base_config, param)
elif hasattr(training, param):
final_config[param] = getattr(training, param)
else:
final_config[param] = DEFAULT_VALUES.get(param)
return final_config
```
**Priorität:**
1. **Base Config** (höchste Priorität bei COCO-Mode)
2. **User Values** (für nicht-geschützte Parameter)
3. **Default Values** (Fallback)
#### 3.3.3 Frontend: Field Locking
Das Frontend zeigt geschützte Felder visuell an:
```javascript
const protectedFields = [
'depth', 'width', 'activation', 'scheduler', 'nmsthre',
'momentum', 'weight_decay', 'warmup_epochs', 'max_epoch',
// ... weitere Felder
];
function applyBaseConfig(config, isCocoMode) {
protectedFields.forEach(field => {
const input = document.querySelector(`[name="${field}"]`);
if (input && config.hasOwnProperty(field)) {
// Set value
input.value = config[field];
if (isCocoMode) {
// Lock field
input.disabled = true;
input.style.backgroundColor = '#d3d3d3';
input.style.cursor = 'not-allowed';
input.title = 'This parameter is locked by base configuration';
}
}
});
// Show/hide info banner
const banner = document.getElementById('base-config-info');
banner.style.display = isCocoMode ? 'block' : 'none';
}
```
**UI-Features:**
- Grauer Hintergrund für gesperrte Felder
- Cursor: `not-allowed` für visuelle Rückmeldung
- Tooltip erklärt warum Feld gesperrt ist
- Grüner Info-Banner zeigt aktive Basis-Konfiguration
#### 3.3.4 Frontend: Field Name Mapping
Alias-Handling für Backend/Frontend Unterschiede:
```javascript
const fieldNameMap = {
'activation': 'act', // Backend: activation, Frontend: act
'nms_thre': 'nmsthre' // Backend: nms_thre, Frontend: nmsthre
};
function applyBaseConfig(config, isCocoMode) {
for (const [backendName, value] of Object.entries(config)) {
// Try direct match first
let input = document.querySelector(`[name="${backendName}"]`);
// Try mapped name if direct match fails
if (!input && fieldNameMap[backendName]) {
const frontendName = fieldNameMap[backendName];
input = document.querySelector(`[name="${frontendName}"]`);
}
if (input) {
input.value = value;
// ... weitere Logik
}
}
}
```
### 3.4 Testing und Validierung
#### 3.4.1 Unit Tests
Erstellte Test-Suite für Basis-Konfigurationen:
```python
# backend/data/test_base_configs.py
import importlib.util
import os
def test_base_config(file_name, expected_depth, expected_width):
"""Test if base config loads correctly"""
file_path = os.path.join(os.path.dirname(__file__), file_name)
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
config = module.BaseExp
assert config.depth == expected_depth
assert config.width == expected_width
# ... weitere Assertions
# Run tests
test_base_config('yolox_s.py', 0.33, 0.50)
test_base_config('yolox_m.py', 0.67, 0.75)
# ...
```
**Ergebnis:** Alle 4 Modell-Konfigurationen laden erfolgreich ✓
#### 3.4.2 Integration Tests
**Testfall 1: COCO Transfer Learning Flow**
1. Dropdown "Train on coco" auswählen → ✓
2. Model "yolox-s" auswählen → ✓
3. Basis-Konfiguration lädt via API → ✓
4. Felder werden ausgefüllt und gesperrt → ✓
5. Info-Banner erscheint → ✓
6. Formular abschicken → ✓
7. Exp.py wird mit gemergten Werten generiert → ✓
**Testfall 2: Sketch Mode**
1. Dropdown "Train from sketch" auswählen → ✓
2. Model-Auswahl wird versteckt → ✓
3. Alle Felder editierbar → ✓
4. Keine Basis-Konfiguration geladen → ✓
**Testfall 3: Field Name Alias**
1. Basis-Config enthält `activation: "silu"` → ✓
2. Frontend-Feld `act` wird korrekt befüllt → ✓
3. Feld wird korrekt gesperrt → ✓
#### 3.4.3 Performance Messungen
| Operation | Durchschnitt | Max |
|-----------|--------------|-----|
| Base Config laden (API) | 45ms | 78ms |
| Frontend Field Update | 12ms | 23ms |
| Exp.py Generierung | 156ms | 234ms |
**Fazit:** Alle Performance-Anforderungen erfüllt ✓
---
## 4. Fazit
### 4.1 Erreichte Ziele
Das Transfer Learning Feature für den mb ai Trainer wurde erfolgreich implementiert und erfüllt alle Anforderungen:
✓ **Basis-Konfiguration System** - 4 YOLOX-Modelle (S, M, L, X) vordefiniert
✓ **Parameter-Schutz** - 24 geschützte Parameter implementiert
✓ **Frontend Integration** - Visuelle Rückmeldung, Field Locking mit HTML/CSS/JS
✓ **Python Backend** - Flask REST API mit SQLAlchemy ORM
✓ **MySQL Integration** - Vollständige Datenbankanbindung
✓ **Label Studio Integration** - Nahtlose Anbindung für Annotationen
✓ **Performance** - Alle Ladezeiten < 100ms
✓ **Dokumentation** - Umfassende Dokumentation erstellt
### 4.2 Herausforderungen
**Problem 1: Backend-Migration von Node.js zu Python**
- Ursprüngliches System war in Node.js/Express/Sequelize entwickelt
- Anforderung: Komplette Migration zu Python/Flask/SQLAlchemy
- Lösung: Vollständige Backend-Konvertierung mit allen Models, Services und Routes
**Problem 2: Label Studio API Integration**
- Synchrone Kommunikation mit externem Service
- Lösung: `requests` Library mit Fehlerbehandlung und Retry-Logik
**Problem 3: Field Name Aliasing**
- Backend nutzt `activation`, Frontend nutzt `act`
- Backend nutzt `nms_thre`, Frontend nutzt `nmsthre`
- Lösung: Mapping-Dictionary für bidirektionales Alias-Handling
**Problem 4: Form Submission mit Disabled Fields**
- HTML disabled Felder werden nicht in FormData inkludiert
- Training würde ohne geschützte Parameter starten
- Lösung: Temporäres Enable vor Submit, dann Re-disable nach Submit
**Problem 5: MySQL Schema-Erweiterung**
- Fehlende Spalten `width` und `height` in `image` Tabelle
- Lösung: ALTER TABLE Statement zur Laufzeit
### 4.3 Mögliche Erweiterungen
1. **Weitere Modelle:** YOLOv8, YOLOv9, EfficientDet Basis-Configs
2. **Custom Base Configs:** Benutzer können eigene Basis-Konfigurationen hochladen
3. **Versionierung:** Historie von Basis-Config Änderungen mit Git-Integration
4. **A/B Testing:** Vergleich verschiedener Basis-Konfigurationen
5. **Cloud-Integration:** Speicherung von Modellen in Cloud Storage
6. **Multi-Tenancy:** Mehrere Benutzer mit separaten Projekten
### 4.4 Persönliches Fazit
Die Implementierung des Transfer Learning Features für den mb ai Trainer war ein erfolgreiches Projekt, das meine Kenntnisse in mehreren Bereichen vertieft hat:
**Technische Kompetenzen:**
- **Python Backend-Entwicklung:** Flask, SQLAlchemy, ORM-Design
- **REST API Design:** Endpoint-Planung, Request/Response-Handling
- **Frontend ohne Framework:** Vanilla JavaScript, DOM-Manipulation, Event-Handling
- **Datenbank-Design:** MySQL Schema-Design, Migrations
- **Externe API-Integration:** Label Studio REST API
- **Merge-Strategien:** Konfigurationsmanagement, Prioritäts-Logik
**Methodische Kompetenzen:**
- Anforderungsanalyse und Systemdesign
- Modulare Architektur und Code-Organisation
- Testing und Validierung
- Technische Dokumentation
Besonders wertvoll war die Erfahrung mit dynamischem Python Module Loading und der Entwicklung einer benutzerfreundlichen UI ausschließlich mit Vanilla JavaScript ohne externe Frameworks.
---
## 5. Anhang
### 5.1 Basis-Konfiguration Beispiel (YOLOX-S)
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters
depth = 0.33
width = 0.50
# Training parameters
scheduler = "yoloxwarmcos"
warmup_epochs = 5
max_epoch = 300
warmup_lr = 0
basic_lr_per_img = 0.01 / 64.0
# Optimizer parameters
momentum = 0.9
weight_decay = 5e-4
# Augmentation parameters
hsv_prob = 1.0
flip_prob = 0.5
degrees = 10.0
translate = 0.1
mosaic_prob = 1.0
# NMS parameters
nms_thre = 0.65
# Activation
activation = "silu"
```
### 5.2 API Dokumentation
**GET /api/base-config/<model_name>**
Lädt Basis-Konfiguration für angegebenes Modell.
**Parameter:**
- `model_name` (string): yolox-s, yolox-m, yolox-l, oder yolox-x
**Response:** JSON-Objekt mit Konfigurationsparametern
**Status Codes:**
- 200: Erfolg
- 404: Modell nicht gefunden
- 500: Server-Fehler
### 5.3 Verwendete Ressourcen
- Flask Documentation: https://flask.palletsprojects.com/
- Python importlib: https://docs.python.org/3/library/importlib.html
- YOLOX Repository: https://github.com/Megvii-BaseDetection/YOLOX
- MDN Web Docs (FormData): https://developer.mozilla.org/en-US/docs/Web/API/FormData
### 5.4 Glossar
| Begriff | Bedeutung |
|---------|-----------|
| Transfer Learning | Verwendung vortrainierter Modelle als Ausgangspunkt |
| Base Config | Vordefinierte Basis-Konfiguration für Modell |
| Protected Parameters | Parameter die nicht vom Benutzer geändert werden können |
| COCO | Common Objects in Context Dataset |
| YOLOX | State-of-the-art Object Detection Modell |
| Merge Strategy | Logik zum Kombinieren mehrerer Konfigurationsquellen |
---
**Ende der Dokumentation**

1221
edit-training.html
View File

File diff suppressed because it is too large Load Diff

40
globals.css
View File

@@ -1,20 +1,20 @@
@import url("https://cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.min.css"); @import url("https://cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.min.css");
* { * {
-webkit-font-smoothing: antialiased; -webkit-font-smoothing: antialiased;
box-sizing: border-box; box-sizing: border-box;
} }
html, html,
body { body {
height: 100%; height: 100%;
background-color: #f9fafb; background-color: #f9fafb;
} }
/* a blue color as a generic focus style */ /* a blue color as a generic focus style */
button:focus-visible { button:focus-visible {
outline: 2px solid #4a90e2 !important; outline: 2px solid #4a90e2 !important;
outline: -webkit-focus-ring-color auto 5px !important; outline: -webkit-focus-ring-color auto 5px !important;
} }
a { a {
text-decoration: none; text-decoration: none;
} }

259
index.html
View File

@@ -1,96 +1,165 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button> Training Project</button>
<button id="seed-db-btn" class="button"> <button id="seed-db-btn" class="button">
Seed Database Seed Database
<div class="loader" id="loader" style="display: none"></div> <div class="loader" id="loader" style="display: none"></div>
</button> </button>
</div> <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">
⚙️
</div> </button>
<div id="projects-list"> </div>
<script src="js/dashboard.js"></script>
</div> </div>
<script> <div id="projects-list">
document.getElementById('seed-db-btn').addEventListener('click', function () { <script src="js/dashboard.js"></script>
const elLoader = document.getElementById("loader") </div>
elLoader.style.display = "inherit" <script>
document.getElementById('seed-db-btn').addEventListener('click', function () {
fetch('/api/seed') const elLoader = document.getElementById("loader")
.finally(() => { elLoader.style.display = "inherit"
// Instead of hiding loader immediately, poll /api/update-status until done
function pollStatus() { fetch('/api/seed')
fetch('/api/update-status') .finally(() => {
.then(res => res.json()) // Instead of hiding loader immediately, poll /api/update-status until done
.then(status => { function pollStatus() {
if (status && status.running) { fetch('/api/update-status')
// Still running, poll again after short delay .then(res => res.json())
setTimeout(pollStatus, 5000); .then(status => {
} else { if (status && status.running) {
elLoader.style.display = "none"; // Still running, poll again after short delay
} setTimeout(pollStatus, 5000);
}) } else {
.catch(() => { elLoader.style.display = "none";
elLoader.style.display = "none"; }
}); })
} .catch(() => {
pollStatus(); elLoader.style.display = "none";
}) });
}); }
pollStatus();
// Show loader if backend is still processing on page load })
});
function pollStatus() {
const elLoader = document.getElementById("loader"); // Show loader if backend is still processing on page load
fetch('/api/update-status')
.then(res => res.json()) function pollStatus() {
const elLoader = document.getElementById("loader");
.then(status => { fetch('/api/update-status')
if (status && status.running) { .then(res => res.json())
elLoader.style.display = "inherit";
setTimeout(pollStatus, 5000); .then(status => {
} else { if (status && status.running) {
elLoader.style.display = "none"; elLoader.style.display = "inherit";
} setTimeout(pollStatus, 5000);
}) } else {
.catch(() => { elLoader.style.display = "none";
elLoader.style.display = "none"; }
}); })
} .catch(() => {
elLoader.style.display = "none";
</script> });
}
</div>
</body> </script>
</div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html> </html>

308
js/add-class.js
View File

@@ -1,154 +1,154 @@
export function addClass() { export function addClass() {
const input_class = document.querySelector('.add-category input.div-wrapper'); const input_class = document.querySelector('.add-category input.div-wrapper');
let existingClasses; let existingClasses;
const input_project_name = document.getElementById('project_name_input') const input_project_name = document.getElementById('project_name_input')
const description = document.getElementById('project_description_input'); const description = document.getElementById('project_description_input');
const button_addClass = document.querySelector('.add-category .upload-button-text-wrapper'); const button_addClass = document.querySelector('.add-category .upload-button-text-wrapper');
const button_addProject = document.querySelector('.popup .confirm-button-datasetcreation') const button_addProject = document.querySelector('.popup .confirm-button-datasetcreation')
const classWrapper = document.querySelector('.add-class-wrapper'); const classWrapper = document.querySelector('.add-class-wrapper');
button_addProject.addEventListener('click', () => { button_addProject.addEventListener('click', () => {
const title = input_project_name.value.trim(); const title = input_project_name.value.trim();
const descriptionText = description.value.trim(); const descriptionText = description.value.trim();
const classes = Array.from(classWrapper.querySelectorAll('.overlap-group')).map(el => el.textContent.trim()); const classes = Array.from(classWrapper.querySelectorAll('.overlap-group')).map(el => el.textContent.trim());
const formData = new FormData(); const formData = new FormData();
formData.append('title', title); formData.append('title', title);
formData.append('description', descriptionText); formData.append('description', descriptionText);
formData.append('classes', JSON.stringify(classes)); formData.append('classes', JSON.stringify(classes));
if (imgBlob) { if (imgBlob) {
formData.append('project_image', imgBlob, 'project_image.png'); // or the correct file type formData.append('project_image', imgBlob, 'project_image.png'); // or the correct file type
} }
fetch('/api/training-projects', { fetch('/api/training-projects', {
method: 'POST', method: 'POST',
body: formData body: formData
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert(data.message || 'Project created!'); alert(data.message || 'Project created!');
window.location.href = '/index.html'; window.location.href = '/index.html';
}) })
.catch(err => alert('Error: ' + err)); .catch(err => alert('Error: ' + err));
}); });
button_addClass.addEventListener('click', () => { button_addClass.addEventListener('click', () => {
const className = input_class.value.trim(); const className = input_class.value.trim();
if (!className) { if (!className) {
alert('Please enter a class name'); alert('Please enter a class name');
return; return;
} }
existingClasses = classWrapper.querySelectorAll('.overlap-group'); existingClasses = classWrapper.querySelectorAll('.overlap-group');
for (const el of existingClasses) { for (const el of existingClasses) {
if (el.textContent.trim().toLowerCase() === className.toLowerCase()) { if (el.textContent.trim().toLowerCase() === className.toLowerCase()) {
alert(`Class name "${className}" already exists.`); alert(`Class name "${className}" already exists.`);
return; return;
} }
} }
const newClassDiv = document.createElement('div'); const newClassDiv = document.createElement('div');
newClassDiv.classList.add('add-class'); newClassDiv.classList.add('add-class');
newClassDiv.style.position = 'relative'; newClassDiv.style.position = 'relative';
newClassDiv.style.width = '335px'; newClassDiv.style.width = '335px';
newClassDiv.style.height = '25px'; newClassDiv.style.height = '25px';
newClassDiv.style.marginBottom = '5px'; newClassDiv.style.marginBottom = '5px';
const overlapGroup = document.createElement('div'); const overlapGroup = document.createElement('div');
overlapGroup.classList.add('overlap-group'); overlapGroup.classList.add('overlap-group');
overlapGroup.style.position = 'absolute'; overlapGroup.style.position = 'absolute';
overlapGroup.style.width = '275px'; overlapGroup.style.width = '275px';
overlapGroup.style.height = '25px'; overlapGroup.style.height = '25px';
overlapGroup.style.top = '0'; overlapGroup.style.top = '0';
overlapGroup.style.left = '0'; overlapGroup.style.left = '0';
overlapGroup.style.backgroundColor = '#30bffc80'; overlapGroup.style.backgroundColor = '#30bffc80';
overlapGroup.style.borderRadius = '5px'; overlapGroup.style.borderRadius = '5px';
overlapGroup.style.display = 'flex'; overlapGroup.style.display = 'flex';
overlapGroup.style.alignItems = 'center'; overlapGroup.style.alignItems = 'center';
overlapGroup.style.paddingLeft = '10px'; overlapGroup.style.paddingLeft = '10px';
overlapGroup.style.color = '#000'; overlapGroup.style.color = '#000';
overlapGroup.style.fontFamily = 'var(--m3-body-small-font-family)'; overlapGroup.style.fontFamily = 'var(--m3-body-small-font-family)';
overlapGroup.style.fontWeight = 'var(--m3-body-small-font-weight)'; overlapGroup.style.fontWeight = 'var(--m3-body-small-font-weight)';
overlapGroup.style.fontSize = 'var(--m3-body-small-font-size)'; overlapGroup.style.fontSize = 'var(--m3-body-small-font-size)';
overlapGroup.style.letterSpacing = 'var(--m3-body-small-letter-spacing)'; overlapGroup.style.letterSpacing = 'var(--m3-body-small-letter-spacing)';
overlapGroup.style.lineHeight = 'var(--m3-body-small-line-height)'; overlapGroup.style.lineHeight = 'var(--m3-body-small-line-height)';
overlapGroup.textContent = className; overlapGroup.textContent = className;
const overlap = document.createElement('div'); const overlap = document.createElement('div');
overlap.classList.add('overlap'); overlap.classList.add('overlap');
overlap.style.position = 'absolute'; overlap.style.position = 'absolute';
overlap.style.width = '50px'; overlap.style.width = '50px';
overlap.style.height = '25px'; overlap.style.height = '25px';
overlap.style.top = '0'; overlap.style.top = '0';
overlap.style.left = '285px'; overlap.style.left = '285px';
const rectangle = document.createElement('div'); const rectangle = document.createElement('div');
rectangle.classList.add('rectangle'); rectangle.classList.add('rectangle');
rectangle.style.width = '50px'; rectangle.style.width = '50px';
rectangle.style.height = '25px'; rectangle.style.height = '25px';
rectangle.style.backgroundColor = '#ff0f43'; rectangle.style.backgroundColor = '#ff0f43';
rectangle.style.borderRadius = '5px'; rectangle.style.borderRadius = '5px';
rectangle.style.display = 'flex'; rectangle.style.display = 'flex';
rectangle.style.alignItems = 'center'; rectangle.style.alignItems = 'center';
rectangle.style.justifyContent = 'center'; rectangle.style.justifyContent = 'center';
rectangle.style.cursor = 'pointer'; rectangle.style.cursor = 'pointer';
rectangle.addEventListener('mouseenter', () => { rectangle.addEventListener('mouseenter', () => {
rectangle.style.backgroundColor = '#bb032b'; rectangle.style.backgroundColor = '#bb032b';
}); });
rectangle.addEventListener('mouseleave', () => { rectangle.addEventListener('mouseleave', () => {
rectangle.style.backgroundColor = '#ff0f43'; rectangle.style.backgroundColor = '#ff0f43';
}); });
const minusText = document.createElement('div'); const minusText = document.createElement('div');
minusText.classList.add('text-wrapper-4'); minusText.classList.add('text-wrapper-4');
minusText.style.position = 'absolute'; minusText.style.position = 'absolute';
minusText.style.top = '-18px'; minusText.style.top = '-18px';
minusText.style.left = '18px'; minusText.style.left = '18px';
minusText.style.fontFamily = 'var(--m3-display-large-font-family)'; minusText.style.fontFamily = 'var(--m3-display-large-font-family)';
minusText.style.fontWeight = 'var(--m3-display-large-font-weight)'; minusText.style.fontWeight = 'var(--m3-display-large-font-weight)';
minusText.style.color = '#000000'; minusText.style.color = '#000000';
minusText.style.fontSize = 'var(--minus-for-button-size)'; minusText.style.fontSize = 'var(--minus-for-button-size)';
minusText.style.letterSpacing = 'var(--m3-display-large-letter-spacing)'; minusText.style.letterSpacing = 'var(--m3-display-large-letter-spacing)';
minusText.style.lineHeight = 'var(--m3-display-large-line-height)'; minusText.style.lineHeight = 'var(--m3-display-large-line-height)';
minusText.style.whiteSpace = 'nowrap'; minusText.style.whiteSpace = 'nowrap';
minusText.style.cursor = 'pointer'; minusText.style.cursor = 'pointer';
minusText.style.fontStyle = 'var(--m3-display-large-font-style)'; minusText.style.fontStyle = 'var(--m3-display-large-font-style)';
minusText.textContent = '_'; minusText.textContent = '_';
rectangle.appendChild(minusText); rectangle.appendChild(minusText);
rectangle.addEventListener('click', () => { rectangle.addEventListener('click', () => {
classWrapper.removeChild(newClassDiv); classWrapper.removeChild(newClassDiv);
document.dispatchEvent(new CustomEvent('classListUpdated')); document.dispatchEvent(new CustomEvent('classListUpdated'));
}); });
overlap.appendChild(rectangle); overlap.appendChild(rectangle);
newClassDiv.appendChild(overlapGroup); newClassDiv.appendChild(overlapGroup);
newClassDiv.appendChild(overlap); newClassDiv.appendChild(overlap);
classWrapper.appendChild(newClassDiv); classWrapper.appendChild(newClassDiv);
input_class.value = ''; input_class.value = '';
}); });
} }

76
js/add-image.js
View File

@@ -1,38 +1,38 @@
//Global Variable //Global Variable
var imgBlob; var imgBlob;
var imgMimeType var imgMimeType
// Create a hidden file input dynamically // Create a hidden file input dynamically
const fileInput = document.createElement('input'); const fileInput = document.createElement('input');
fileInput.type = 'file'; fileInput.type = 'file';
fileInput.accept = 'image/*'; fileInput.accept = 'image/*';
fileInput.style.display = 'none'; fileInput.style.display = 'none';
document.body.appendChild(fileInput); document.body.appendChild(fileInput);
function uploadButtonHandler() { function uploadButtonHandler() {
fileInput.click(); fileInput.click();
}; };
fileInput.addEventListener('change', () => { fileInput.addEventListener('change', () => {
const imageDiv = document.querySelector('.popup .image'); const imageDiv = document.querySelector('.popup .image');
const file = fileInput.files[0]; const file = fileInput.files[0];
if (!file) return; if (!file) return;
const reader = new FileReader(); const reader = new FileReader();
reader.onload = (e) => { reader.onload = (e) => {
imageDiv.innerHTML = ''; // clear previous content imageDiv.innerHTML = ''; // clear previous content
const img = document.createElement('img'); const img = document.createElement('img');
img.src = e.target.result; img.src = e.target.result;
img.alt = 'Uploaded Image'; img.alt = 'Uploaded Image';
img.style.width = '100%'; img.style.width = '100%';
img.style.height = '100%'; img.style.height = '100%';
img.style.objectFit = 'cover'; img.style.objectFit = 'cover';
img.style.borderRadius = '10px'; img.style.borderRadius = '10px';
imageDiv.appendChild(img); imageDiv.appendChild(img);
// Use the original file as the blob and store its MIME type // Use the original file as the blob and store its MIME type
imgBlob = file; imgBlob = file;
imgMimeType = file.type; imgMimeType = file.type;
}; };
reader.readAsDataURL(file); reader.readAsDataURL(file);
}); });

274
js/dashboard-label-studio.js
View File

@@ -1,137 +1,137 @@
// Fetch LabelStudioProjects from backend and render as selectable cards // Fetch LabelStudioProjects from backend and render as selectable cards
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
let projectsList = document.getElementById('projects-list'); let projectsList = document.getElementById('projects-list');
const selectedIds = new Set(); const selectedIds = new Set();
if (!projectsList) { if (!projectsList) {
// Try to create the container if missing // Try to create the container if missing
projectsList = document.createElement('div'); projectsList = document.createElement('div');
projectsList.id = 'projects-list'; projectsList.id = 'projects-list';
document.body.appendChild(projectsList); document.body.appendChild(projectsList);
} }
else{console.log("noep")} else{console.log("noep")}
fetch('/api/label-studio-projects') fetch('/api/label-studio-projects')
.then(res => res.json()) .then(res => res.json())
.then(projects => { .then(projects => {
projectsList.innerHTML = ''; projectsList.innerHTML = '';
if (!projects || projects.length === 0) { if (!projects || projects.length === 0) {
projectsList.innerHTML = '<div>No Label Studio projects found</div>'; projectsList.innerHTML = '<div>No Label Studio projects found</div>';
return; return;
} }
for (const project of projects) { for (const project of projects) {
// Only show card if there is at least one non-empty annotation class // Only show card if there is at least one non-empty annotation class
const annotationClasses = Object.entries(project.annotationCounts || {}) const annotationClasses = Object.entries(project.annotationCounts || {})
.filter(([label, count]) => label && label.trim() !== ''); .filter(([label, count]) => label && label.trim() !== '');
if (annotationClasses.length === 0) continue; if (annotationClasses.length === 0) continue;
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.overflow = 'hidden'; card.style.overflow = 'hidden';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)';
card.style.display = 'flex'; card.style.display = 'flex';
card.style.background = 'white'; card.style.background = 'white';
card.style.cursor = 'pointer'; card.style.cursor = 'pointer';
card.tabIndex = 0; card.tabIndex = 0;
card.setAttribute('role', 'button'); card.setAttribute('role', 'button');
card.setAttribute('aria-label', `Open project ${project.title || project.project_id}`); card.setAttribute('aria-label', `Open project ${project.title || project.project_id}`);
// Selection logic // Selection logic
card.dataset.projectId = project.project_id; card.dataset.projectId = project.project_id;
card.addEventListener('click', () => { card.addEventListener('click', () => {
card.classList.toggle('selected'); card.classList.toggle('selected');
if (card.classList.contains('selected')) { if (card.classList.contains('selected')) {
card.style.background = '#009eac'; // main dif color for card card.style.background = '#009eac'; // main dif color for card
selectedIds.add(project.project_id); selectedIds.add(project.project_id);
} else { } else {
card.style.background = 'white'; // revert card color card.style.background = 'white'; // revert card color
selectedIds.delete(project.project_id); selectedIds.delete(project.project_id);
} }
// Debug: log selected ids array // Debug: log selected ids array
console.log(Array.from(selectedIds)); console.log(Array.from(selectedIds));
}); });
// Info // Info
const infoDiv = document.createElement('div'); const infoDiv = document.createElement('div');
infoDiv.className = 'info'; infoDiv.className = 'info';
infoDiv.style.background = 'rgba(210, 238, 240)'; infoDiv.style.background = 'rgba(210, 238, 240)';
infoDiv.style.flex = '1'; infoDiv.style.flex = '1';
infoDiv.style.padding = '16px'; infoDiv.style.padding = '16px';
infoDiv.innerHTML = ` infoDiv.innerHTML = `
<h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.project_id ?? 'N/A'} &nbsp;&nbsp;&nbsp; ${project.title || 'Untitled'}</h3> <h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.project_id ?? 'N/A'} &nbsp;&nbsp;&nbsp; ${project.title || 'Untitled'}</h3>
<div class="label-classes" style="font-size:1em;"> <div class="label-classes" style="font-size:1em;">
${annotationClasses.map(([label, count]) => `<p>${label}: ${count}</p>`).join('')} ${annotationClasses.map(([label, count]) => `<p>${label}: ${count}</p>`).join('')}
</div> </div>
`; `;
card.appendChild(infoDiv); card.appendChild(infoDiv);
projectsList.appendChild(card); projectsList.appendChild(card);
} }
}) })
.catch(() => { .catch(() => {
projectsList.innerHTML = '<div>Error loading Label Studio projects</div>'; projectsList.innerHTML = '<div>Error loading Label Studio projects</div>';
}); });
// Add Next button at the bottom right of the page // Add Next button at the bottom right of the page
const nextBtn = document.createElement('button'); const nextBtn = document.createElement('button');
nextBtn.id = 'next-btn'; nextBtn.id = 'next-btn';
nextBtn.className = 'button'; nextBtn.className = 'button';
nextBtn.textContent = 'Next'; nextBtn.textContent = 'Next';
nextBtn.style.position = 'fixed'; nextBtn.style.position = 'fixed';
nextBtn.style.right = '32px'; nextBtn.style.right = '32px';
nextBtn.style.bottom = '32px'; nextBtn.style.bottom = '32px';
nextBtn.style.zIndex = '1000'; nextBtn.style.zIndex = '1000';
document.body.appendChild(nextBtn); document.body.appendChild(nextBtn);
// Get training_project_id from URL // Get training_project_id from URL
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const trainingProjectId = urlParams.get('id'); const trainingProjectId = urlParams.get('id');
// Next button click handler // Next button click handler
nextBtn.addEventListener('click', () => { nextBtn.addEventListener('click', () => {
console.log(trainingProjectId) console.log(trainingProjectId)
if (!trainingProjectId) { if (!trainingProjectId) {
alert('No training project selected.'); alert('No training project selected.');
return; return;
} }
if (selectedIds.size === 0) { if (selectedIds.size === 0) {
alert('Please select at least one Label Studio project.'); alert('Please select at least one Label Studio project.');
return; return;
} }
const annotationProjectsJson = JSON.stringify(Array.from(selectedIds)); const annotationProjectsJson = JSON.stringify(Array.from(selectedIds));
fetch('/api/training-project-details', { fetch('/api/training-project-details', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ body: JSON.stringify({
project_id: Number(trainingProjectId), project_id: Number(trainingProjectId),
annotation_projects: Array.from(selectedIds) annotation_projects: Array.from(selectedIds)
}) })
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert('TrainingProjectDetails saved!'); alert('TrainingProjectDetails saved!');
console.log(data); console.log(data);
// Redirect to start-training.html with id // Redirect to start-training.html with id
window.location.href = `/setup-training-project.html?id=${trainingProjectId}`; window.location.href = `/setup-training-project.html?id=${trainingProjectId}`;
}) })
.catch(err => { .catch(err => {
alert('Error saving TrainingProjectDetails'); alert('Error saving TrainingProjectDetails');
console.error(err); console.error(err);
}); });
}); });
// Add description field above the project cards // Add description field above the project cards
const descDiv = document.createElement('div'); const descDiv = document.createElement('div');
descDiv.id = 'dashboard-description'; descDiv.id = 'dashboard-description';
descDiv.style.width = '100%'; descDiv.style.width = '100%';
descDiv.style.maxWidth = '900px'; descDiv.style.maxWidth = '900px';
descDiv.style.margin = '0 auto 24px auto'; descDiv.style.margin = '0 auto 24px auto';
descDiv.style.padding = '18px 24px'; descDiv.style.padding = '18px 24px';
descDiv.style.background = '#eaf7fa'; descDiv.style.background = '#eaf7fa';
descDiv.style.borderRadius = '12px'; descDiv.style.borderRadius = '12px';
descDiv.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)'; descDiv.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)';
descDiv.style.fontSize = '1.15em'; descDiv.style.fontSize = '1.15em';
descDiv.style.color = '#009eac'; descDiv.style.color = '#009eac';
descDiv.style.textAlign = 'center'; descDiv.style.textAlign = 'center';
descDiv.textContent = 'Select one or more Label Studio projects by clicking the cards below. The annotation summary for each project is shown. Click Next to continue.'; descDiv.textContent = 'Select one or more Label Studio projects by clicking the cards below. The annotation summary for each project is shown. Click Next to continue.';
projectsList.parentNode.insertBefore(descDiv, projectsList); projectsList.parentNode.insertBefore(descDiv, projectsList);
}); });

342
js/dashboard.js
View File

@@ -1,171 +1,171 @@
function renderProjects(projects) { function renderProjects(projects) {
const projectsList = document.getElementById('projects-list'); const projectsList = document.getElementById('projects-list');
projectsList.innerHTML = ''; projectsList.innerHTML = '';
if (projects.length === 0) { if (projects.length === 0) {
projectsList.innerHTML = '<div>No projects found</div>'; projectsList.innerHTML = '<div>No projects found</div>';
return; return;
} }
for (const project of projects) { for (const project of projects) {
const labelCounts = project.labelCounts || {}; const labelCounts = project.labelCounts || {};
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.overflow = 'hidden'; card.style.overflow = 'hidden';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)';
card.style.display = 'flex'; card.style.display = 'flex';
card.style.background = 'white'; card.style.background = 'white';
card.style.cursor = 'pointer'; card.style.cursor = 'pointer';
card.tabIndex = 0; card.tabIndex = 0;
card.setAttribute('role', 'button'); card.setAttribute('role', 'button');
card.setAttribute('aria-label', `Open project ${project.title || project.id}`); card.setAttribute('aria-label', `Open project ${project.title || project.id}`);
card.style.position = 'relative'; // For absolute positioning of delete button card.style.position = 'relative'; // For absolute positioning of delete button
card.addEventListener('click', (e) => { card.addEventListener('click', (e) => {
// Prevent click if delete button is pressed // Prevent click if delete button is pressed
if (e.target.classList.contains('delete-btn')) return; if (e.target.classList.contains('delete-btn')) return;
if (project.hasTraining) { if (project.hasTraining) {
window.location.href = `/overview-training.html?id=${project.id}`; window.location.href = `/overview-training.html?id=${project.id}`;
} else if (project.hasDetails) { } else if (project.hasDetails) {
// Find details for this project // Find details for this project
const detailsEntry = window._trainingProjectDetails?.find(d => d.project_id == project.id); const detailsEntry = window._trainingProjectDetails?.find(d => d.project_id == project.id);
if (detailsEntry && Array.isArray(detailsEntry.class_map) && detailsEntry.class_map.length > 0) { if (detailsEntry && Array.isArray(detailsEntry.class_map) && detailsEntry.class_map.length > 0) {
// If classes are assigned, skip to start-training.html // If classes are assigned, skip to start-training.html
window.location.href = `/edit-training.html?id=${project.id}`; window.location.href = `/edit-training.html?id=${project.id}`;
} else { } else {
window.location.href = `/setup-training-project.html?id=${project.id}`; window.location.href = `/setup-training-project.html?id=${project.id}`;
} }
} else { } else {
window.location.href = `/project-details.html?id=${project.id}`; window.location.href = `/project-details.html?id=${project.id}`;
} }
}); });
// Image // Image
let imageHTML = ''; let imageHTML = '';
if (project.project_image) { if (project.project_image) {
imageHTML = `<img src="${project.project_image}" alt="img" style="width:120px;height:120px;object-fit:cover;display:block;" />`; imageHTML = `<img src="${project.project_image}" alt="img" style="width:120px;height:120px;object-fit:cover;display:block;" />`;
} }
const imgContainer = document.createElement('div'); const imgContainer = document.createElement('div');
imgContainer.className = 'img-container'; imgContainer.className = 'img-container';
imgContainer.style.background = '#009eac2d' imgContainer.style.background = '#009eac2d'
imgContainer.style.flex = '0 0 120px'; imgContainer.style.flex = '0 0 120px';
imgContainer.style.display = 'flex'; imgContainer.style.display = 'flex';
imgContainer.style.alignItems = 'center'; imgContainer.style.alignItems = 'center';
imgContainer.style.justifyContent = 'center'; imgContainer.style.justifyContent = 'center';
imgContainer.innerHTML = imageHTML; imgContainer.innerHTML = imageHTML;
// Info // Info
const infoDiv = document.createElement('div'); const infoDiv = document.createElement('div');
infoDiv.className = 'info'; infoDiv.className = 'info';
infoDiv.style.background = '#009eac2d' infoDiv.style.background = '#009eac2d'
infoDiv.style.flex = '1'; infoDiv.style.flex = '1';
infoDiv.style.padding = '16px'; infoDiv.style.padding = '16px';
infoDiv.innerHTML = ` infoDiv.innerHTML = `
<h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.id ?? 'N/A'} &nbsp&nbsp&nbsp ${project.title || 'Untitled'}</h3> <h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.id ?? 'N/A'} &nbsp&nbsp&nbsp ${project.title || 'Untitled'}</h3>
<div class="label-classes" style="font-size:1em;"> <div class="label-classes" style="font-size:1em;">
${getClassesAsParagraphs(project, labelCounts)} ${getClassesAsParagraphs(project, labelCounts)}
</div> </div>
`; `;
// Delete button // Delete button
const deleteBtn = document.createElement('button'); const deleteBtn = document.createElement('button');
deleteBtn.textContent = 'Delete'; deleteBtn.textContent = 'Delete';
deleteBtn.style.width = '70px'; deleteBtn.style.width = '70px';
deleteBtn.style.height = '28px'; deleteBtn.style.height = '28px';
deleteBtn.className = 'button-red delete-btn'; deleteBtn.className = 'button-red delete-btn';
deleteBtn.style.position = 'absolute'; deleteBtn.style.position = 'absolute';
deleteBtn.style.bottom = '0px'; deleteBtn.style.bottom = '0px';
deleteBtn.style.right = '15px'; deleteBtn.style.right = '15px';
deleteBtn.style.zIndex = '2'; deleteBtn.style.zIndex = '2';
deleteBtn.style.fontSize = '14px'; deleteBtn.style.fontSize = '14px';
deleteBtn.style.padding = '0'; deleteBtn.style.padding = '0';
deleteBtn.style.borderRadius = '6px'; deleteBtn.style.borderRadius = '6px';
deleteBtn.style.boxShadow = '0 2px 8px rgba(0,0,0,0.08)'; deleteBtn.style.boxShadow = '0 2px 8px rgba(0,0,0,0.08)';
deleteBtn.style.display = 'flex'; deleteBtn.style.display = 'flex';
deleteBtn.style.alignItems = 'center'; deleteBtn.style.alignItems = 'center';
deleteBtn.style.justifyContent = 'center'; deleteBtn.style.justifyContent = 'center';
deleteBtn.addEventListener('click', function(e) { deleteBtn.addEventListener('click', function(e) {
e.stopPropagation(); e.stopPropagation();
if (confirm('Are you sure you want to delete this training project?')) { if (confirm('Are you sure you want to delete this training project?')) {
fetch(`/api/training-projects/${project.id}`, { method: 'DELETE' }) fetch(`/api/training-projects/${project.id}`, { method: 'DELETE' })
.then(res => { .then(res => {
if (res.ok) { if (res.ok) {
card.remove(); card.remove();
} else { } else {
alert('Failed to delete project.'); alert('Failed to delete project.');
} }
}) })
.catch(() => alert('Failed to delete project.')); .catch(() => alert('Failed to delete project.'));
} }
}); });
card.appendChild(imgContainer); card.appendChild(imgContainer);
card.appendChild(infoDiv); card.appendChild(infoDiv);
card.appendChild(deleteBtn); card.appendChild(deleteBtn);
projectsList.appendChild(card); projectsList.appendChild(card);
} }
} }
// Helper to render classes as <p> elements // Helper to render classes as <p> elements
function getClassesAsParagraphs(project, labelCounts) { function getClassesAsParagraphs(project, labelCounts) {
let classes = []; let classes = [];
let labelConfig = project.parsed_label_config; let labelConfig = project.parsed_label_config;
if (typeof labelConfig === 'string') { if (typeof labelConfig === 'string') {
try { labelConfig = JSON.parse(labelConfig); } catch { labelConfig = null; } try { labelConfig = JSON.parse(labelConfig); } catch { labelConfig = null; }
} }
if (labelConfig) { if (labelConfig) {
Object.values(labelConfig).forEach(cfg => { Object.values(labelConfig).forEach(cfg => {
if (cfg.labels && Array.isArray(cfg.labels)) { if (cfg.labels && Array.isArray(cfg.labels)) {
cfg.labels.forEach(label => { cfg.labels.forEach(label => {
classes.push(label); classes.push(label);
}); });
} }
}); });
} }
if (classes.length === 0 && project.prompts && project.prompts.length > 0) { if (classes.length === 0 && project.prompts && project.prompts.length > 0) {
const prompt = project.prompts[0]; const prompt = project.prompts[0];
if (prompt.output_classes && prompt.output_classes.length > 0) { if (prompt.output_classes && prompt.output_classes.length > 0) {
classes = prompt.output_classes; classes = prompt.output_classes;
} }
} }
if (classes.length === 0 && Object.keys(labelCounts).length > 0) { if (classes.length === 0 && Object.keys(labelCounts).length > 0) {
classes = Object.keys(labelCounts); classes = Object.keys(labelCounts);
} }
return classes.map(cls => `<p>${cls}${labelCounts && labelCounts[cls] !== undefined ? ' ' : ''}</p>`).join(''); return classes.map(cls => `<p>${cls}${labelCounts && labelCounts[cls] !== undefined ? ' ' : ''}</p>`).join('');
} }
// Fetch and render TrainingProjects from the backend // Fetch and render TrainingProjects from the backend
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
Promise.all([ Promise.all([
fetch('/api/training-projects').then(res => res.json()), fetch('/api/training-projects').then(res => res.json()),
fetch('/api/training-project-details').then(res => res.json()), fetch('/api/training-project-details').then(res => res.json()),
fetch('/api/trainings').then(res => res.json()) fetch('/api/trainings').then(res => res.json())
]).then(([projects, details, trainings]) => { ]).then(([projects, details, trainings]) => {
window._trainingProjectDetails = details; // Store globally for click handler window._trainingProjectDetails = details; // Store globally for click handler
// Build a set of project IDs that have details // Build a set of project IDs that have details
const detailsProjectIds = new Set(details.map(d => d.project_id)); const detailsProjectIds = new Set(details.map(d => d.project_id));
// Build a set of project IDs that have trainings // Build a set of project IDs that have trainings
const detailsIdToProjectId = {}; const detailsIdToProjectId = {};
details.forEach(d => { detailsIdToProjectId[d.id] = d.project_id; }); details.forEach(d => { detailsIdToProjectId[d.id] = d.project_id; });
const trainingProjectIds = new Set(trainings.map(t => detailsIdToProjectId[t.project_details_id])); const trainingProjectIds = new Set(trainings.map(t => detailsIdToProjectId[t.project_details_id]));
// Map project_id to id for frontend compatibility // Map project_id to id for frontend compatibility
projects.forEach(project => { projects.forEach(project => {
if (project.project_id !== undefined) project.id = project.project_id; if (project.project_id !== undefined) project.id = project.project_id;
if (Array.isArray(project.classes)) { if (Array.isArray(project.classes)) {
project.labelCounts = {}; project.labelCounts = {};
project.classes.forEach(cls => project.labelCounts[cls] = 0); project.classes.forEach(cls => project.labelCounts[cls] = 0);
} }
// Attach a flag for details existence // Attach a flag for details existence
project.hasDetails = detailsProjectIds.has(project.id); project.hasDetails = detailsProjectIds.has(project.id);
// Attach a flag for training existence // Attach a flag for training existence
project.hasTraining = trainingProjectIds.has(project.id); project.hasTraining = trainingProjectIds.has(project.id);
}); });
renderProjects(projects); renderProjects(projects);
}).catch(err => { }).catch(err => {
document.getElementById('projects-list').innerHTML = '<div>Error loading projects</div>'; document.getElementById('projects-list').innerHTML = '<div>Error loading projects</div>';
}); });
}); });

224
js/settings.js Normal file
View File

@@ -0,0 +1,224 @@
// Settings Modal Management
// Function to open modal
window.openSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'flex';
loadSettings();
}
};
// Function to close modal
window.closeSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'none';
}
};
// Close modal when clicking outside
window.addEventListener('click', function(event) {
const modal = document.getElementById('settings-modal');
if (event.target === modal) {
window.closeSettingsModal();
}
});
// Load settings when modal opens
async function loadSettings() {
try {
const response = await fetch('/api/settings');
if (!response.ok) {
throw new Error('Failed to load settings');
}
const settings = await response.json();
const settingsMap = {};
settings.forEach(s => {
settingsMap[s.key] = s.value;
});
// Populate fields with correct IDs
const labelstudioUrl = document.getElementById('labelstudio-url');
const labelstudioToken = document.getElementById('labelstudio-token');
const yoloxPathInput = document.getElementById('yolox-path');
const yoloxVenvPathInput = document.getElementById('yolox-venv-path');
const yoloxOutputPathInput = document.getElementById('yolox-output-path');
const yoloxDataDirInput = document.getElementById('yolox-data-dir');
if (labelstudioUrl) labelstudioUrl.value = settingsMap.labelstudio_api_url || '';
if (labelstudioToken) labelstudioToken.value = settingsMap.labelstudio_api_token || '';
if (yoloxPathInput) yoloxPathInput.value = settingsMap.yolox_path || '';
if (yoloxVenvPathInput) yoloxVenvPathInput.value = settingsMap.yolox_venv_path || '';
if (yoloxOutputPathInput) yoloxOutputPathInput.value = settingsMap.yolox_output_path || '';
if (yoloxDataDirInput) yoloxDataDirInput.value = settingsMap.yolox_data_dir || '';
} catch (error) {
console.error('Error loading settings:', error);
const saveStatus = document.getElementById('save-status');
if (saveStatus) {
showMessage(saveStatus, 'Fehler beim Laden: ' + error.message, 'error');
}
}
}
// Helper functions
function showMessage(element, message, type) {
if (element) {
element.textContent = message;
element.className = 'status-message ' + type;
element.style.display = 'block';
}
}
function hideMessage(element) {
if (element) {
element.style.display = 'none';
}
}
// Event listeners - wait for DOM to be ready
document.addEventListener('DOMContentLoaded', function() {
// Test Label Studio connection
const testLabelStudioBtn = document.getElementById('test-labelstudio-btn');
if (testLabelStudioBtn) {
testLabelStudioBtn.addEventListener('click', async () => {
const apiUrl = document.getElementById('labelstudio-url').value.trim();
const apiToken = document.getElementById('labelstudio-token').value.trim();
const labelstudioStatus = document.getElementById('labelstudio-status');
const loader = document.getElementById('test-ls-loader');
if (!apiUrl || !apiToken) {
showMessage(labelstudioStatus, 'Bitte geben Sie URL und Token ein', 'error');
return;
}
testLabelStudioBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(labelstudioStatus);
try {
const response = await fetch('/api/settings/test/labelstudio', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ api_url: apiUrl, api_token: apiToken })
});
const result = await response.json();
if (result.success) {
showMessage(labelstudioStatus, '✓ ' + result.message, 'success');
} else {
showMessage(labelstudioStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(labelstudioStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testLabelStudioBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Test YOLOX path
const testYoloxBtn = document.getElementById('test-yolox-btn');
if (testYoloxBtn) {
testYoloxBtn.addEventListener('click', async () => {
const path = document.getElementById('yolox-path').value.trim();
const venvPath = document.getElementById('yolox-venv-path').value.trim();
const yoloxStatus = document.getElementById('yolox-status');
const loader = document.getElementById('test-yolox-loader');
if (!path) {
showMessage(yoloxStatus, 'Bitte geben Sie einen YOLOX Pfad ein', 'error');
return;
}
testYoloxBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(yoloxStatus);
try {
const response = await fetch('/api/settings/test/yolox', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
yolox_path: path,
yolox_venv_path: venvPath
})
});
const result = await response.json();
if (result.success) {
showMessage(yoloxStatus, '✓ ' + result.message, 'success');
} else {
showMessage(yoloxStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(yoloxStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testYoloxBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Save settings
const saveSettingsBtn = document.getElementById('save-settings-btn');
if (saveSettingsBtn) {
saveSettingsBtn.addEventListener('click', async () => {
const labelstudioUrl = document.getElementById('labelstudio-url').value.trim();
const labelstudioToken = document.getElementById('labelstudio-token').value.trim();
const yoloxPathValue = document.getElementById('yolox-path').value.trim();
const yoloxVenvPathValue = document.getElementById('yolox-venv-path').value.trim();
const yoloxOutputPathValue = document.getElementById('yolox-output-path').value.trim();
const yoloxDataDirValue = document.getElementById('yolox-data-dir').value.trim();
const saveStatus = document.getElementById('save-status');
// Validation
if (!labelstudioUrl || !labelstudioToken || !yoloxPathValue || !yoloxVenvPathValue || !yoloxOutputPathValue || !yoloxDataDirValue) {
showMessage(saveStatus, 'Bitte füllen Sie alle Felder aus', 'error');
return;
}
const settings = {
labelstudio_api_url: labelstudioUrl,
labelstudio_api_token: labelstudioToken,
yolox_path: yoloxPathValue,
yolox_venv_path: yoloxVenvPathValue,
yolox_output_path: yoloxOutputPathValue,
yolox_data_dir: yoloxDataDirValue
};
saveSettingsBtn.disabled = true;
const originalText = saveSettingsBtn.textContent;
saveSettingsBtn.textContent = 'Speichern...';
hideMessage(saveStatus);
try {
const response = await fetch('/api/settings', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings)
});
if (!response.ok) {
throw new Error('Failed to save settings');
}
showMessage(saveStatus, '✓ Einstellungen erfolgreich gespeichert!', 'success');
// Close modal after 1.5 seconds
setTimeout(() => {
window.closeSettingsModal();
}, 1500);
} catch (error) {
showMessage(saveStatus, '✗ Fehler beim Speichern: ' + error.message, 'error');
} finally {
saveSettingsBtn.disabled = false;
saveSettingsBtn.textContent = originalText;
}
});
}
});

430
js/setup-training-project.js
View File

@@ -1,216 +1,216 @@
// Fetch and display training project name in nav bar // Fetch and display training project name in nav bar
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const trainingProjectId = urlParams.get('id'); const trainingProjectId = urlParams.get('id');
if (!trainingProjectId) return; if (!trainingProjectId) return;
// Fetch training project, details, and all LabelStudioProjects // Fetch training project, details, and all LabelStudioProjects
Promise.all([ Promise.all([
fetch(`/api/training-projects`).then(res => res.json()), fetch(`/api/training-projects`).then(res => res.json()),
fetch(`/api/training-project-details`).then(res => res.json()), fetch(`/api/training-project-details`).then(res => res.json()),
fetch(`/api/label-studio-projects`).then(res => res.json()) fetch(`/api/label-studio-projects`).then(res => res.json())
]).then(([projects, detailsList, labelStudioProjects]) => { ]).then(([projects, detailsList, labelStudioProjects]) => {
// Find the selected training project // Find the selected training project
const project = projects.find(p => p.project_id == trainingProjectId || p.id == trainingProjectId); const project = projects.find(p => p.project_id == trainingProjectId || p.id == trainingProjectId);
// Find the details entry for this project // Find the details entry for this project
const details = detailsList.find(d => d.project_id == trainingProjectId); const details = detailsList.find(d => d.project_id == trainingProjectId);
if (!project || !details) return; if (!project || !details) return;
// Get the stored classes from training project // Get the stored classes from training project
const storedClasses = Array.isArray(project.classes) ? project.classes : []; const storedClasses = Array.isArray(project.classes) ? project.classes : [];
// Get related LabelStudioProject IDs // Get related LabelStudioProject IDs
const relatedIds = Array.isArray(details.annotation_projects) ? details.annotation_projects : []; const relatedIds = Array.isArray(details.annotation_projects) ? details.annotation_projects : [];
// Filter LabelStudioProjects to only those related // Filter LabelStudioProjects to only those related
const relatedProjects = labelStudioProjects.filter(lp => relatedIds.includes(lp.project_id)); const relatedProjects = labelStudioProjects.filter(lp => relatedIds.includes(lp.project_id));
// Render cards for each related LabelStudioProject // Render cards for each related LabelStudioProject
const detailsDiv = document.getElementById('details'); const detailsDiv = document.getElementById('details');
detailsDiv.innerHTML = ''; detailsDiv.innerHTML = '';
// Find the longest label name for sizing // Find the longest label name for sizing
let maxLabelLength = 0; let maxLabelLength = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
classNames.forEach(className => { classNames.forEach(className => {
if (className && className.trim() !== '' && className.length > maxLabelLength) { if (className && className.trim() !== '' && className.length > maxLabelLength) {
maxLabelLength = className.length; maxLabelLength = className.length;
} }
}); });
}); });
// Use ch unit for width to fit the longest text // Use ch unit for width to fit the longest text
const labelWidth = `${maxLabelLength + 2}ch`; const labelWidth = `${maxLabelLength + 2}ch`;
// Find the longest project name for sizing // Find the longest project name for sizing
let maxProjectNameLength = 0; let maxProjectNameLength = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const nameLength = (lp.title || String(lp.project_id)).length; const nameLength = (lp.title || String(lp.project_id)).length;
if (nameLength > maxProjectNameLength) maxProjectNameLength = nameLength; if (nameLength > maxProjectNameLength) maxProjectNameLength = nameLength;
}); });
const projectNameWidth = `${maxProjectNameLength + 2}ch`; const projectNameWidth = `${maxProjectNameLength + 2}ch`;
// Find the card with the most classes // Find the card with the most classes
let maxClassCount = 0; let maxClassCount = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
if (classNames.length > maxClassCount) maxClassCount = classNames.length; if (classNames.length > maxClassCount) maxClassCount = classNames.length;
}); });
// Set a fixed width for the class rows container // Set a fixed width for the class rows container
const classRowHeight = 38; // px, adjust if needed const classRowHeight = 38; // px, adjust if needed
const classRowsWidth = `${maxClassCount * 180}px`; const classRowsWidth = `${maxClassCount * 180}px`;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
// Get original class names from annotationCounts // Get original class names from annotationCounts
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.margin = '18px 0'; card.style.margin = '18px 0';
card.style.padding = '18px'; card.style.padding = '18px';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)';
// Extra div for project name // Extra div for project name
const nameDiv = document.createElement('div'); const nameDiv = document.createElement('div');
nameDiv.textContent = lp.title || lp.project_id; nameDiv.textContent = lp.title || lp.project_id;
nameDiv.style.fontSize = '1.2em'; nameDiv.style.fontSize = '1.2em';
nameDiv.style.fontWeight = 'bold'; nameDiv.style.fontWeight = 'bold';
nameDiv.style.marginBottom = '12px'; nameDiv.style.marginBottom = '12px';
nameDiv.style.background = '#eaf7fa'; nameDiv.style.background = '#eaf7fa';
nameDiv.style.padding = '8px 16px'; nameDiv.style.padding = '8px 16px';
nameDiv.style.borderRadius = '8px'; nameDiv.style.borderRadius = '8px';
nameDiv.style.width = projectNameWidth; nameDiv.style.width = projectNameWidth;
nameDiv.style.minWidth = projectNameWidth; nameDiv.style.minWidth = projectNameWidth;
nameDiv.style.maxWidth = projectNameWidth; nameDiv.style.maxWidth = projectNameWidth;
nameDiv.style.display = 'inline-block'; nameDiv.style.display = 'inline-block';
card.appendChild(nameDiv); card.appendChild(nameDiv);
// Container for class rows // Container for class rows
const classRowsDiv = document.createElement('div'); const classRowsDiv = document.createElement('div');
classRowsDiv.style.display = 'inline-block'; classRowsDiv.style.display = 'inline-block';
classRowsDiv.style.verticalAlign = 'top'; classRowsDiv.style.verticalAlign = 'top';
classRowsDiv.style.width = classRowsWidth; classRowsDiv.style.width = classRowsWidth;
classNames.forEach(className => { classNames.forEach(className => {
// Row for class name and dropdown // Row for class name and dropdown
const row = document.createElement('div'); const row = document.createElement('div');
row.className = 'class-row'; // Mark as class row row.className = 'class-row'; // Mark as class row
row.style.display = 'flex'; row.style.display = 'flex';
row.style.alignItems = 'center'; row.style.alignItems = 'center';
row.style.marginBottom = '10px'; row.style.marginBottom = '10px';
// Original class name // Original class name
const labelSpan = document.createElement('span'); const labelSpan = document.createElement('span');
labelSpan.textContent = className; labelSpan.textContent = className;
labelSpan.style.fontWeight = 'bold'; labelSpan.style.fontWeight = 'bold';
labelSpan.style.marginRight = '16px'; labelSpan.style.marginRight = '16px';
labelSpan.style.width = labelWidth; labelSpan.style.width = labelWidth;
labelSpan.style.minWidth = labelWidth; labelSpan.style.minWidth = labelWidth;
labelSpan.style.maxWidth = labelWidth; labelSpan.style.maxWidth = labelWidth;
labelSpan.style.display = 'inline-block'; labelSpan.style.display = 'inline-block';
// Dropdown for reassigning // Dropdown for reassigning
const select = document.createElement('select'); const select = document.createElement('select');
select.style.marginLeft = '8px'; select.style.marginLeft = '8px';
select.style.padding = '4px 8px'; select.style.padding = '4px 8px';
select.style.borderRadius = '6px'; select.style.borderRadius = '6px';
select.style.border = '1px solid #009eac'; select.style.border = '1px solid #009eac';
// Add blank item // Add blank item
const blankOption = document.createElement('option'); const blankOption = document.createElement('option');
blankOption.value = ''; blankOption.value = '';
blankOption.textContent = ''; blankOption.textContent = '';
select.appendChild(blankOption); select.appendChild(blankOption);
storedClasses.forEach(cls => { storedClasses.forEach(cls => {
const option = document.createElement('option'); const option = document.createElement('option');
option.value = cls; option.value = cls;
option.textContent = cls; option.textContent = cls;
select.appendChild(option); select.appendChild(option);
}); });
row.appendChild(labelSpan); row.appendChild(labelSpan);
row.appendChild(select); row.appendChild(select);
classRowsDiv.appendChild(row); classRowsDiv.appendChild(row);
}); });
card.appendChild(classRowsDiv); card.appendChild(classRowsDiv);
// Description field (right side, last element) // Description field (right side, last element)
const descDiv = document.createElement('div'); const descDiv = document.createElement('div');
descDiv.className = 'card-description'; descDiv.className = 'card-description';
descDiv.style.flex = '1'; descDiv.style.flex = '1';
descDiv.style.marginLeft = '32px'; descDiv.style.marginLeft = '32px';
descDiv.style.display = 'flex'; descDiv.style.display = 'flex';
descDiv.style.flexDirection = 'column'; descDiv.style.flexDirection = 'column';
descDiv.style.justifyContent = 'flex-start'; descDiv.style.justifyContent = 'flex-start';
descDiv.style.alignItems = 'flex-start'; descDiv.style.alignItems = 'flex-start';
descDiv.style.width = '220px'; descDiv.style.width = '220px';
// Add a label and textarea for description // Add a label and textarea for description
const descLabel = document.createElement('label'); const descLabel = document.createElement('label');
descLabel.textContent = 'Description:'; descLabel.textContent = 'Description:';
descLabel.style.fontWeight = 'bold'; descLabel.style.fontWeight = 'bold';
descLabel.style.marginBottom = '4px'; descLabel.style.marginBottom = '4px';
const descTextarea = document.createElement('textarea'); const descTextarea = document.createElement('textarea');
descTextarea.style.width = '220px'; descTextarea.style.width = '220px';
descTextarea.style.height = '48px'; descTextarea.style.height = '48px';
descTextarea.style.borderRadius = '6px'; descTextarea.style.borderRadius = '6px';
descTextarea.style.border = '1px solid #009eac'; descTextarea.style.border = '1px solid #009eac';
descTextarea.style.padding = '6px'; descTextarea.style.padding = '6px';
descTextarea.style.resize = 'none'; descTextarea.style.resize = 'none';
descTextarea.value = lp.description || ''; descTextarea.value = lp.description || '';
descDiv.appendChild(descLabel); descDiv.appendChild(descLabel);
descDiv.appendChild(descTextarea); descDiv.appendChild(descTextarea);
card.appendChild(descDiv); card.appendChild(descDiv);
detailsDiv.appendChild(card); detailsDiv.appendChild(card);
}); });
// Add Next button at the bottom right of the page // Add Next button at the bottom right of the page
const nextBtn = document.createElement('button'); const nextBtn = document.createElement('button');
nextBtn.id = 'next-btn'; nextBtn.id = 'next-btn';
nextBtn.className = 'button'; nextBtn.className = 'button';
nextBtn.textContent = 'Next'; nextBtn.textContent = 'Next';
nextBtn.style.position = 'fixed'; nextBtn.style.position = 'fixed';
nextBtn.style.right = '32px'; nextBtn.style.right = '32px';
nextBtn.style.bottom = '32px'; nextBtn.style.bottom = '32px';
nextBtn.style.zIndex = '1000'; nextBtn.style.zIndex = '1000';
document.body.appendChild(nextBtn); document.body.appendChild(nextBtn);
// Next button click handler: collect class mappings and update TrainingProjectDetails // Next button click handler: collect class mappings and update TrainingProjectDetails
nextBtn.addEventListener('click', () => { nextBtn.addEventListener('click', () => {
// Array of arrays: [[labelStudioProjectId, [[originalClass, mappedClass], ...]], ...] // Array of arrays: [[labelStudioProjectId, [[originalClass, mappedClass], ...]], ...]
const mappings = []; const mappings = [];
const descriptions = []; const descriptions = [];
detailsDiv.querySelectorAll('.card').forEach((card, idx) => { detailsDiv.querySelectorAll('.card').forEach((card, idx) => {
const projectId = relatedProjects[idx].project_id; const projectId = relatedProjects[idx].project_id;
const classMap = []; const classMap = [];
// Only iterate over actual class rows // Only iterate over actual class rows
card.querySelectorAll('.class-row').forEach(row => { card.querySelectorAll('.class-row').forEach(row => {
const labelSpan = row.querySelector('span'); const labelSpan = row.querySelector('span');
const select = row.querySelector('select'); const select = row.querySelector('select');
if (labelSpan && select) { if (labelSpan && select) {
const className = labelSpan.textContent.trim(); const className = labelSpan.textContent.trim();
const mappedValue = select.value.trim(); const mappedValue = select.value.trim();
if (className !== '' && mappedValue !== '') { if (className !== '' && mappedValue !== '') {
classMap.push([className, mappedValue]); classMap.push([className, mappedValue]);
} }
} }
}); });
mappings.push([projectId, classMap]); mappings.push([projectId, classMap]);
// Get description from textarea // Get description from textarea
const descTextarea = card.querySelector('textarea'); const descTextarea = card.querySelector('textarea');
descriptions.push([projectId, descTextarea ? descTextarea.value : '']); descriptions.push([projectId, descTextarea ? descTextarea.value : '']);
}); });
// Update TrainingProjectDetails in DB // Update TrainingProjectDetails in DB
fetch('/api/training-project-details', { fetch('/api/training-project-details', {
method: 'PUT', method: 'PUT',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ body: JSON.stringify({
project_id: Number(trainingProjectId), project_id: Number(trainingProjectId),
class_map: mappings, class_map: mappings,
description: descriptions // array of [projectId, description] description: descriptions // array of [projectId, description]
}) })
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert('Class assignments and descriptions updated!'); alert('Class assignments and descriptions updated!');
console.log(data); console.log(data);
// Redirect to start-training.html with id // Redirect to start-training.html with id
window.location.href = `/edit-training.html?id=${trainingProjectId}`; window.location.href = `/edit-training.html?id=${trainingProjectId}`;
}) })
.catch(err => { .catch(err => {
alert('Error updating class assignments or descriptions'); alert('Error updating class assignments or descriptions');
console.error(err); console.error(err);
}); });
}); });
}); });
}); });

544
js/start-training.js
View File

@@ -1,272 +1,272 @@
// Render helper descriptions for YOLOX settings and handle form submission // Render helper descriptions for YOLOX settings and handle form submission
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
// Get the form element at the top // Get the form element at the top
const form = document.getElementById('settings-form'); const form = document.getElementById('settings-form');
// Base config state // Base config state
let currentBaseConfig = null; let currentBaseConfig = null;
let baseConfigFields = []; let baseConfigFields = [];
// Define which fields are protected by base config // Define which fields are protected by base config
const protectedFields = [ const protectedFields = [
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr', 'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay', 'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup', 'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees', 'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval' 'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
]; ];
// Map backend field names to frontend field names // Map backend field names to frontend field names
const fieldNameMap = { const fieldNameMap = {
'activation': 'act', // Backend uses 'activation', frontend uses 'act' 'activation': 'act', // Backend uses 'activation', frontend uses 'act'
'nms_thre': 'nmsthre' 'nms_thre': 'nmsthre'
}; };
// Function to load base config for selected model // Function to load base config for selected model
function loadBaseConfig(modelName) { function loadBaseConfig(modelName) {
if (!modelName) return Promise.resolve(null); if (!modelName) return Promise.resolve(null);
return fetch(`/api/base-config/${modelName}`) return fetch(`/api/base-config/${modelName}`)
.then(res => { .then(res => {
if (!res.ok) throw new Error('Base config not found'); if (!res.ok) throw new Error('Base config not found');
return res.json(); return res.json();
}) })
.catch(err => { .catch(err => {
console.warn(`Could not load base config for ${modelName}:`, err); console.warn(`Could not load base config for ${modelName}:`, err);
return null; return null;
}); });
} }
// Function to apply base config to form fields // Function to apply base config to form fields
function applyBaseConfig(config, isCocoMode) { function applyBaseConfig(config, isCocoMode) {
const infoBanner = document.getElementById('base-config-info'); const infoBanner = document.getElementById('base-config-info');
const modelNameSpan = document.getElementById('base-config-model'); const modelNameSpan = document.getElementById('base-config-model');
if (!config || !isCocoMode) { if (!config || !isCocoMode) {
// Hide info banner // Hide info banner
if (infoBanner) infoBanner.style.display = 'none'; if (infoBanner) infoBanner.style.display = 'none';
// Remove grey styling and enable all fields // Remove grey styling and enable all fields
protectedFields.forEach(fieldName => { protectedFields.forEach(fieldName => {
const input = form.querySelector(`[name="${fieldName}"]`); const input = form.querySelector(`[name="${fieldName}"]`);
if (input) { if (input) {
input.disabled = false; input.disabled = false;
input.style.backgroundColor = '#f8f8f8'; input.style.backgroundColor = '#f8f8f8';
input.style.color = '#333'; input.style.color = '#333';
input.style.cursor = 'text'; input.style.cursor = 'text';
input.title = ''; input.title = '';
} }
}); });
baseConfigFields = []; baseConfigFields = [];
return; return;
} }
// Show info banner // Show info banner
if (infoBanner) { if (infoBanner) {
infoBanner.style.display = 'block'; infoBanner.style.display = 'block';
const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model'; const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model';
if (modelNameSpan) modelNameSpan.textContent = modelName; if (modelNameSpan) modelNameSpan.textContent = modelName;
} }
// Apply base config values and grey out fields // Apply base config values and grey out fields
baseConfigFields = []; baseConfigFields = [];
Object.entries(config).forEach(([key, value]) => { Object.entries(config).forEach(([key, value]) => {
// Map backend field name to frontend field name if needed // Map backend field name to frontend field name if needed
const frontendFieldName = fieldNameMap[key] || key; const frontendFieldName = fieldNameMap[key] || key;
if (protectedFields.includes(frontendFieldName)) { if (protectedFields.includes(frontendFieldName)) {
const input = form.querySelector(`[name="${frontendFieldName}"]`); const input = form.querySelector(`[name="${frontendFieldName}"]`);
if (input) { if (input) {
baseConfigFields.push(frontendFieldName); baseConfigFields.push(frontendFieldName);
// Set value based on type // Set value based on type
if (input.type === 'checkbox') { if (input.type === 'checkbox') {
input.checked = Boolean(value); input.checked = Boolean(value);
} else if (Array.isArray(value)) { } else if (Array.isArray(value)) {
input.value = value.join(','); input.value = value.join(',');
} else { } else {
input.value = value; input.value = value;
} }
// Grey out and disable // Grey out and disable
input.disabled = true; input.disabled = true;
input.style.backgroundColor = '#d3d3d3'; input.style.backgroundColor = '#d3d3d3';
input.style.color = '#666'; input.style.color = '#666';
input.style.cursor = 'not-allowed'; input.style.cursor = 'not-allowed';
// Add title tooltip // Add title tooltip
const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model'; const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model';
input.title = `Protected by base config for ${modelName}. Switch to "Train from sketch" to customize.`; input.title = `Protected by base config for ${modelName}. Switch to "Train from sketch" to customize.`;
} }
} }
}); });
console.log(`Applied base config. Protected fields: ${baseConfigFields.join(', ')}`); console.log(`Applied base config. Protected fields: ${baseConfigFields.join(', ')}`);
} }
// Function to update form based on transfer learning mode // Function to update form based on transfer learning mode
function updateTransferLearningMode() { function updateTransferLearningMode() {
const transferLearning = document.getElementById('transfer-learning'); const transferLearning = document.getElementById('transfer-learning');
const selectModel = document.getElementById('select-model'); const selectModel = document.getElementById('select-model');
const selectModelRow = document.getElementById('select-model-row'); const selectModelRow = document.getElementById('select-model-row');
if (!transferLearning || !selectModel) return; if (!transferLearning || !selectModel) return;
const isCocoMode = transferLearning.value === 'coco'; const isCocoMode = transferLearning.value === 'coco';
const isCustomMode = transferLearning.value === 'custom'; const isCustomMode = transferLearning.value === 'custom';
const isSketchMode = transferLearning.value === 'sketch'; const isSketchMode = transferLearning.value === 'sketch';
const modelName = selectModel.value; const modelName = selectModel.value;
// Show/hide select model based on transfer learning mode // Show/hide select model based on transfer learning mode
if (selectModelRow) { if (selectModelRow) {
if (isSketchMode) { if (isSketchMode) {
selectModelRow.style.display = 'none'; selectModelRow.style.display = 'none';
} else { } else {
selectModelRow.style.display = ''; selectModelRow.style.display = '';
} }
} }
if (isCocoMode && modelName) { if (isCocoMode && modelName) {
// Load and apply base config // Load and apply base config
loadBaseConfig(modelName).then(config => { loadBaseConfig(modelName).then(config => {
currentBaseConfig = config; currentBaseConfig = config;
applyBaseConfig(config, true); applyBaseConfig(config, true);
}); });
} else { } else {
// Clear base config // Clear base config
currentBaseConfig = null; currentBaseConfig = null;
applyBaseConfig(null, false); applyBaseConfig(null, false);
} }
} }
// Listen for changes to transfer learning dropdown // Listen for changes to transfer learning dropdown
const transferLearningSelect = document.getElementById('transfer-learning'); const transferLearningSelect = document.getElementById('transfer-learning');
if (transferLearningSelect) { if (transferLearningSelect) {
transferLearningSelect.addEventListener('change', updateTransferLearningMode); transferLearningSelect.addEventListener('change', updateTransferLearningMode);
} }
// Listen for changes to model selection // Listen for changes to model selection
const modelSelect = document.getElementById('select-model'); const modelSelect = document.getElementById('select-model');
if (modelSelect) { if (modelSelect) {
modelSelect.addEventListener('change', updateTransferLearningMode); modelSelect.addEventListener('change', updateTransferLearningMode);
} }
// Initial update on page load // Initial update on page load
setTimeout(updateTransferLearningMode, 100); setTimeout(updateTransferLearningMode, 100);
// Auto-set num_classes from training_project classes array // Auto-set num_classes from training_project classes array
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id'); const projectId = urlParams.get('id');
if (projectId && form) { if (projectId && form) {
fetch('/api/training-projects') fetch('/api/training-projects')
.then(res => res.json()) .then(res => res.json())
.then(projects => { .then(projects => {
const project = projects.find(p => p.project_id == projectId || p.id == projectId); const project = projects.find(p => p.project_id == projectId || p.id == projectId);
if (project && project.classes) { if (project && project.classes) {
let classesArr = project.classes; let classesArr = project.classes;
// If classes is a stringified JSON, parse it // If classes is a stringified JSON, parse it
if (typeof classesArr === 'string') { if (typeof classesArr === 'string') {
try { try {
classesArr = JSON.parse(classesArr); classesArr = JSON.parse(classesArr);
} catch (e) { } catch (e) {
classesArr = []; classesArr = [];
} }
} }
let numClasses = 0; let numClasses = 0;
if (Array.isArray(classesArr)) { if (Array.isArray(classesArr)) {
numClasses = classesArr.length; numClasses = classesArr.length;
} else if (typeof classesArr === 'object' && classesArr !== null) { } else if (typeof classesArr === 'object' && classesArr !== null) {
numClasses = Object.keys(classesArr).length; numClasses = Object.keys(classesArr).length;
} }
// Fix: Only set num_classes if input exists // Fix: Only set num_classes if input exists
const numClassesInput = form.querySelector('[name="num_classes"]'); const numClassesInput = form.querySelector('[name="num_classes"]');
if (numClassesInput) { if (numClassesInput) {
numClassesInput.value = numClasses; numClassesInput.value = numClasses;
numClassesInput.readOnly = true; numClassesInput.readOnly = true;
numClassesInput.dispatchEvent(new Event('input')); numClassesInput.dispatchEvent(new Event('input'));
} }
} }
}); });
} }
// Handle form submission // Handle form submission
form.addEventListener('submit', function(e) { form.addEventListener('submit', function(e) {
console.log("Form submitted"); console.log("Form submitted");
e.preventDefault(); e.preventDefault();
// Temporarily enable disabled fields so they get included in FormData // Temporarily enable disabled fields so they get included in FormData
const disabledInputs = []; const disabledInputs = [];
form.querySelectorAll('input[disabled], select[disabled]').forEach(input => { form.querySelectorAll('input[disabled], select[disabled]').forEach(input => {
input.disabled = false; input.disabled = false;
disabledInputs.push(input); disabledInputs.push(input);
}); });
const formData = new FormData(form); const formData = new FormData(form);
const settings = {}; const settings = {};
let fileToUpload = null; let fileToUpload = null;
for (const [key, value] of formData.entries()) { for (const [key, value] of formData.entries()) {
if (key === 'model_upload' && form.elements[key].files.length > 0) { if (key === 'model_upload' && form.elements[key].files.length > 0) {
fileToUpload = form.elements[key].files[0]; fileToUpload = form.elements[key].files[0];
continue; continue;
} }
if (key === 'ema' || key === 'enable_mixup' || key === 'save_history_ckpt') { if (key === 'ema' || key === 'enable_mixup' || key === 'save_history_ckpt') {
settings[key] = form.elements[key].checked; settings[key] = form.elements[key].checked;
} else if (key === 'scale' || key === 'mosaic_scale' || key === 'mixup_scale' || key === 'input_size' || key === 'test_size') { } else if (key === 'scale' || key === 'mosaic_scale' || key === 'mixup_scale' || key === 'input_size' || key === 'test_size') {
settings[key] = value.split(',').map(v => parseFloat(v.trim())); settings[key] = value.split(',').map(v => parseFloat(v.trim()));
} else if (!isNaN(value) && value !== '') { } else if (!isNaN(value) && value !== '') {
settings[key] = parseFloat(value); settings[key] = parseFloat(value);
} else { } else {
settings[key] = value; settings[key] = value;
} }
} }
// Re-disable the inputs // Re-disable the inputs
disabledInputs.forEach(input => { disabledInputs.forEach(input => {
input.disabled = true; input.disabled = true;
}); });
// Attach project id from URL // Attach project id from URL
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id'); const projectId = urlParams.get('id');
if (projectId) settings.project_id = Number(projectId); if (projectId) settings.project_id = Number(projectId);
// First, send settings JSON (without file) // First, send settings JSON (without file)
fetch('/api/yolox-settings', { fetch('/api/yolox-settings', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings) body: JSON.stringify(settings)
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
// If file selected, send it as binary // If file selected, send it as binary
if (fileToUpload) { if (fileToUpload) {
const reader = new FileReader(); const reader = new FileReader();
reader.onload = function(e) { reader.onload = function(e) {
fetch(`/api/yolox-settings/upload?project_id=${settings.project_id}`, { fetch(`/api/yolox-settings/upload?project_id=${settings.project_id}`, {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/octet-stream' }, headers: { 'Content-Type': 'application/octet-stream' },
body: e.target.result body: e.target.result
}) })
.then(res => res.json()) .then(res => res.json())
.then(data2 => { .then(data2 => {
alert('YOLOX settings and model file saved!'); alert('YOLOX settings and model file saved!');
window.location.href = `/overview-training.html?id=${settings.project_id}`; window.location.href = `/overview-training.html?id=${settings.project_id}`;
}) })
.catch(err => { .catch(err => {
alert('Error uploading model file'); alert('Error uploading model file');
console.error(err); console.error(err);
}); });
}; };
reader.readAsArrayBuffer(fileToUpload); reader.readAsArrayBuffer(fileToUpload);
} else { } else {
alert('YOLOX settings saved!'); alert('YOLOX settings saved!');
window.location.href = `/overview-training.html?id=${settings.project_id}`; window.location.href = `/overview-training.html?id=${settings.project_id}`;
} }
}) })
.catch(err => { .catch(err => {
alert('Error saving YOLOX settings'); alert('Error saving YOLOX settings');
console.error(err); console.error(err);
}); });
}); });
}); });

20
js/storage.js
View File

@@ -1,11 +1,11 @@
// js/storage.js // js/storage.js
export function getStoredProjects() { export function getStoredProjects() {
try { try {
return JSON.parse(localStorage.getItem('ls_projects') || '{}'); return JSON.parse(localStorage.getItem('ls_projects') || '{}');
} catch (e) { } catch (e) {
return {}; return {};
} }
} }
export function setStoredProjects(projectsObj) { export function setStoredProjects(projectsObj) {
localStorage.setItem('ls_projects', JSON.stringify(projectsObj)); localStorage.setItem('ls_projects', JSON.stringify(projectsObj));
} }

577
overview-training.html
View File

@@ -1,254 +1,325 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" <label id="project-title-label"
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label> style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button> Training Project</button>
<button id="seed-db-btn" class="button"> <button id="seed-db-btn" class="button">
Seed Database Seed Database
<div class="loader" id="loader" style="display: none"></div> <div class="loader" id="loader" style="display: none"></div>
</div> </button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
</button>
<button id="generate-yolox-json-btn" class="button"> </div>
Generate YOLOX JSON </button>
</button>
</button>
</button> <button id="setup-details" class="button">
<button id="setup-details" class="button"> Show Details
Show Details </button>
</button>
<script>
<script> document.getElementById('seed-db-btn').addEventListener('click', function () {
document.getElementById('seed-db-btn').addEventListener('click', function () { const elLoader = document.getElementById("loader")
const elLoader = document.getElementById("loader") elLoader.style.display = "inherit"
elLoader.style.display = "inherit"
fetch('/api/seed')
fetch('/api/seed') .finally(() => {
.finally(() => { // Instead of hiding loader immediately, poll /api/update-status until done
// Instead of hiding loader immediately, poll /api/update-status until done function pollStatus() {
function pollStatus() { fetch('/api/update-status')
fetch('/api/update-status') .then(res => res.json())
.then(res => res.json()) .then(status => {
.then(status => { if (status && status.running) {
if (status && status.running) { // Still running, poll again after short delay
// Still running, poll again after short delay setTimeout(pollStatus, 5000);
setTimeout(pollStatus, 5000); } else {
} else { elLoader.style.display = "none";
elLoader.style.display = "none"; }
} })
}) .catch(() => {
.catch(() => { elLoader.style.display = "none";
elLoader.style.display = "none"; });
}); }
} pollStatus();
pollStatus(); })
}) });
});
// Show loader if backend is still processing on page load
// Show loader if backend is still processing on page load
function pollStatus() {
function pollStatus() { const elLoader = document.getElementById("loader");
const elLoader = document.getElementById("loader"); fetch('/api/update-status')
fetch('/api/update-status') .then(res => res.json())
.then(res => res.json())
.then(status => {
.then(status => { if (status && status.running) {
if (status && status.running) { elLoader.style.display = "inherit";
elLoader.style.display = "inherit"; setTimeout(pollStatus, 5000);
setTimeout(pollStatus, 5000); } else {
} else { elLoader.style.display = "none";
elLoader.style.display = "none"; }
} })
}) .catch(() => {
.catch(() => { elLoader.style.display = "none";
elLoader.style.display = "none"; });
}); }
}
</script>
</script> <script>
<script> // Declare urlParams and projectId once
// Declare urlParams and projectId once const urlParams = new URLSearchParams(window.location.search);
const urlParams = new URLSearchParams(window.location.search); const projectId = urlParams.get('id');
const projectId = urlParams.get('id'); // Set project title in header
// Set project title in header fetch('/api/training-projects')
fetch('/api/training-projects') .then(res => res.json())
.then(res => res.json()) .then(projects => {
.then(projects => { const project = projects.find(p => p.project_id == projectId || p.id == projectId);
const project = projects.find(p => p.project_id == projectId || p.id == projectId); if (project) {
if (project) { const titleLabel = document.getElementById('project-title-label');
const titleLabel = document.getElementById('project-title-label'); if (titleLabel) titleLabel.textContent = '/' + project.title;
if (titleLabel) titleLabel.textContent = '/' + project.title; }
} });
}); // Render trainings
// Render trainings function renderTrainings(trainings) {
function renderTrainings(trainings) { const list = document.getElementById('projects-list');
const list = document.getElementById('projects-list'); list.innerHTML = '';
list.innerHTML = ''; if (!Array.isArray(trainings) || trainings.length === 0) {
if (!Array.isArray(trainings) || trainings.length === 0) { list.innerHTML = '<div style="color:#009eac;padding:16px;">No trainings found for this project.</div>';
list.innerHTML = '<div style="color:#009eac;padding:16px;">No trainings found for this project.</div>'; return;
return; }
} trainings.forEach(training => {
trainings.forEach(training => { const card = document.createElement('div');
const card = document.createElement('div'); card.className = 'dataset-card';
card.className = 'dataset-card'; card.style = 'border:1px solid #009eac;padding:12px;margin:8px;border-radius:8px;background:#eaf7fa;position:relative;min-width:320px;min-height:120px;display:flex;flex-direction:row;justify-content:space-between;align-items:stretch;';
card.style = 'border:1px solid #009eac;padding:12px;margin:8px;border-radius:8px;background:#eaf7fa;position:relative;min-width:320px;min-height:120px;display:flex;flex-direction:row;justify-content:space-between;align-items:stretch;';
// Info section (left)
// Info section (left) const infoDiv = document.createElement('div');
const infoDiv = document.createElement('div'); infoDiv.style = 'flex:1; text-align:left;';
infoDiv.style = 'flex:1; text-align:left;'; infoDiv.innerHTML = `<b>${training.exp_name || 'Training'}</b><br>Epochs: ${training.max_epoch}<br>Depth: ${training.depth}<br>Width: ${training.width}<br>Activation: ${training.activation || training.act || ''}`;
infoDiv.innerHTML = `<b>${training.exp_name || 'Training'}</b><br>Epochs: ${training.max_epoch}<br>Depth: ${training.depth}<br>Width: ${training.width}<br>Activation: ${training.activation || training.act || ''}`;
// Buttons section (right)
// Buttons section (right) const btnDiv = document.createElement('div');
const btnDiv = document.createElement('div'); btnDiv.style = 'display:flex;flex-direction:column;align-items:flex-end;gap:8px;min-width:160px;';
btnDiv.style = 'display:flex;flex-direction:column;align-items:flex-end;gap:8px;min-width:160px;';
// Start Training button
// Start Training button const startBtn = document.createElement('button');
const startBtn = document.createElement('button'); startBtn.textContent = 'Start YOLOX Training';
startBtn.textContent = 'Start YOLOX Training'; startBtn.style = 'background:#009eac;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
startBtn.style = 'background:#009eac;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;'; startBtn.onclick = function() {
startBtn.onclick = function() { startBtn.disabled = true;
startBtn.disabled = true; fetch('/api/start-yolox-training', {
fetch('/api/start-yolox-training', { method: 'POST',
method: 'POST', headers: { 'Content-Type': 'application/json' },
headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ project_id: projectId, training_id: training.id })
body: JSON.stringify({ project_id: projectId, training_id: training.id }) })
}) .then(res => res.json())
.then(res => res.json()) .then(result => {
.then(result => { alert(result.message || 'Training started');
alert(result.message || 'Training started'); startBtn.disabled = false;
startBtn.disabled = false; })
}) .catch(() => {
.catch(() => { alert('Failed to start training');
alert('Failed to start training'); startBtn.disabled = false;
startBtn.disabled = false; });
}); };
}; btnDiv.appendChild(startBtn);
btnDiv.appendChild(startBtn);
// View Log button
// View Log button const logBtn = document.createElement('button');
const logBtn = document.createElement('button'); logBtn.textContent = 'View Training Log';
logBtn.textContent = 'View Training Log'; logBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
logBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;'; logBtn.onclick = function() {
logBtn.onclick = function() { showLogModal(training.id);
showLogModal(training.id); };
}; btnDiv.appendChild(logBtn);
btnDiv.appendChild(logBtn);
// Remove button
// Remove button const removeBtn = document.createElement('button');
const removeBtn = document.createElement('button'); removeBtn.textContent = 'Remove';
removeBtn.textContent = 'Remove'; removeBtn.style = 'background:#ff4d4f;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
removeBtn.style = 'background:#ff4d4f;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;'; removeBtn.onclick = function() {
removeBtn.onclick = function() { if (confirm('Are you sure you want to delete this training?')) {
if (confirm('Are you sure you want to delete this training?')) { fetch(`/api/trainings/${training.id}`, { method: 'DELETE' })
fetch(`/api/trainings/${training.id}`, { method: 'DELETE' }) .then(res => res.json())
.then(res => res.json()) .then(result => {
.then(result => { alert(result.message || 'Training deleted');
alert(result.message || 'Training deleted'); fetchTrainings(); // Refresh list
fetchTrainings(); // Refresh list })
}) .catch(() => alert('Failed to delete training'));
.catch(() => alert('Failed to delete training')); }
} };
}; btnDiv.appendChild(removeBtn);
btnDiv.appendChild(removeBtn);
card.appendChild(infoDiv);
card.appendChild(infoDiv); card.appendChild(btnDiv);
card.appendChild(btnDiv); list.appendChild(card);
list.appendChild(card); });
}); // Modal for log display
// Modal for log display if (!document.getElementById('log-modal')) {
if (!document.getElementById('log-modal')) { const modal = document.createElement('div');
const modal = document.createElement('div'); modal.id = 'log-modal';
modal.id = 'log-modal'; modal.style = 'display:none;position:fixed;top:0;left:0;width:100vw;height:100vh;background:rgba(0,0,0,0.5);z-index:9999;justify-content:center;align-items:center;';
modal.style = 'display:none;position:fixed;top:0;left:0;width:100vw;height:100vh;background:rgba(0,0,0,0.5);z-index:9999;justify-content:center;align-items:center;'; modal.innerHTML = `<div style="background:#fff;padding:24px;border-radius:8px;max-width:800px;width:90vw;max-height:80vh;overflow:auto;position:relative;"><pre id='log-content' style='font-size:13px;white-space:pre-wrap;word-break:break-all;max-height:60vh;overflow:auto;background:#f7f7f7;padding:12px;border-radius:6px;'></pre><button id='close-log-modal' style='position:absolute;top:8px;right:8px;background:#009eac;color:#fff;border:none;border-radius:4px;padding:6px 12px;cursor:pointer;'>Close</button></div>`;
modal.innerHTML = `<div style="background:#fff;padding:24px;border-radius:8px;max-width:800px;width:90vw;max-height:80vh;overflow:auto;position:relative;"><pre id='log-content' style='font-size:13px;white-space:pre-wrap;word-break:break-all;max-height:60vh;overflow:auto;background:#f7f7f7;padding:12px;border-radius:6px;'></pre><button id='close-log-modal' style='position:absolute;top:8px;right:8px;background:#009eac;color:#fff;border:none;border-radius:4px;padding:6px 12px;cursor:pointer;'>Close</button></div>`; document.body.appendChild(modal);
document.body.appendChild(modal); document.getElementById('close-log-modal').onclick = function() {
document.getElementById('close-log-modal').onclick = function() { modal.style.display = 'none';
modal.style.display = 'none'; };
}; }
}
// Function to show log modal and poll log
// Function to show log modal and poll log function showLogModal(trainingId) {
function showLogModal(trainingId) { const modal = document.getElementById('log-modal');
const modal = document.getElementById('log-modal'); const logContent = document.getElementById('log-content');
const logContent = document.getElementById('log-content'); modal.style.display = 'flex';
modal.style.display = 'flex'; function fetchLog() {
function fetchLog() { fetch(`/api/training-log?project_id=${projectId}&training_id=${trainingId}`)
fetch(`/api/training-log?project_id=${projectId}&training_id=${trainingId}`) .then(res => res.json())
.then(res => res.json()) .then(data => {
.then(data => { logContent.textContent = data.log || 'No log found.';
logContent.textContent = data.log || 'No log found.'; })
}) .catch(() => {
.catch(() => { logContent.textContent = 'Failed to fetch log.';
logContent.textContent = 'Failed to fetch log.'; });
}); }
} fetchLog();
fetchLog(); // Poll every 5 seconds while modal is open
// Poll every 5 seconds while modal is open let poller = setInterval(() => {
let poller = setInterval(() => { if (modal.style.display === 'flex') fetchLog();
if (modal.style.display === 'flex') fetchLog(); else clearInterval(poller);
else clearInterval(poller); }, 5000);
}, 5000); }
} }
} // Fetch trainings for project
// Fetch trainings for project function fetchTrainings() {
function fetchTrainings() { if (!projectId) return;
if (!projectId) return; fetch(`/api/trainings?project_id=${projectId}`)
fetch(`/api/trainings?project_id=${projectId}`) .then(res => res.json())
.then(res => res.json()) .then(trainings => {
.then(trainings => { renderTrainings(trainings);
renderTrainings(trainings); });
}); }
} window.addEventListener('DOMContentLoaded', fetchTrainings);
window.addEventListener('DOMContentLoaded', fetchTrainings); </script>
document.getElementById('generate-yolox-json-btn').addEventListener('click', function () { <div style="padding: 16px; text-align: left;">
fetch('/api/generate-yolox-json', { <button id="create-new-training-btn" class="button" style="background:#009eac;color:white;">
method: 'POST', + Create New Training
headers: { 'Content-Type': 'application/json' }, </button>
body: JSON.stringify({ project_id: projectId }) </div>
}) <div id="projects-list"></div>
.then(res => res.json()) </div>
.then(result => {
alert('YOLOX JSON generated!'); <script>
}) // Create New Training button handler
.catch(err => { document.addEventListener('DOMContentLoaded', function() {
alert('Failed to generate YOLOX JSON'); document.getElementById('create-new-training-btn').addEventListener('click', function() {
}); if (!projectId) {
}); alert('No project selected');
</script> return;
<div id="projects-list"></div> }
</div> // Navigate to edit-training page to configure new training parameters
</body> // This will reuse existing project details and class mappings
window.location.href = `/edit-training.html?project_id=${projectId}`;
});
});
</script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html> </html>

297
project-details.html
View File

@@ -1,116 +1,183 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon> style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label> <label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" <button id="Add Training Project" onclick="window.location.href='/add-project.html'"
class="button-red">Add Training Project</button> class="button-red">Add Training Project</button>
<button id="seed-db-btn" class="button"> <button id="seed-db-btn" class="button">
Seed Database Seed Database
<div class="loader" id="loader" style="display: none"></div> <div class="loader" id="loader" style="display: none"></div>
</button> </button>
</div> <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
</div>
<div id="projects-list">
<div id="projects-list">
</div>
<script> </div>
document.getElementById('seed-db-btn').addEventListener('click', function () { <script>
const elLoader = document.getElementById("loader") document.getElementById('seed-db-btn').addEventListener('click', function () {
elLoader.style.display = "inherit" const elLoader = document.getElementById("loader")
elLoader.style.display = "inherit"
fetch('/api/seed')
.finally(() => { fetch('/api/seed')
// Instead of hiding loader immediately, poll /api/update-status until done .finally(() => {
function pollStatus() { // Instead of hiding loader immediately, poll /api/update-status until done
fetch('/api/update-status') function pollStatus() {
.then(res => res.json()) fetch('/api/update-status')
.then(status => { .then(res => res.json())
if (status && status.running) { .then(status => {
// Still running, poll again after short delay if (status && status.running) {
setTimeout(pollStatus, 5000); // Still running, poll again after short delay
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
pollStatus(); }
}) pollStatus();
}); })
});
// Show loader if backend is still processing on page load
// Show loader if backend is still processing on page load
function pollStatus() {
const elLoader = document.getElementById("loader"); function pollStatus() {
fetch('/api/update-status') const elLoader = document.getElementById("loader");
.then(res => res.json()) fetch('/api/update-status')
.then(res => res.json())
.then(status => {
if (status && status.running) { .then(status => {
elLoader.style.display = "inherit"; if (status && status.running) {
setTimeout(pollStatus, 5000); elLoader.style.display = "inherit";
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
}
</script>
<script> </script>
const urlParams = new URLSearchParams(window.location.search); <script>
const projectId = urlParams.get('id'); const urlParams = new URLSearchParams(window.location.search);
fetch('/api/training-projects') const projectId = urlParams.get('id');
.then(res => res.json()) fetch('/api/training-projects')
.then(projects => { .then(res => res.json())
const project = projects.find(p => p.project_id == projectId || p.id == projectId); .then(projects => {
if (project) { const project = projects.find(p => p.project_id == projectId || p.id == projectId);
document.getElementById('project-title-label').textContent = '/' + project.title; if (project) {
} document.getElementById('project-title-label').textContent = '/' + project.title;
}); }
</script> });
</script>
</div>
<script src="./js/dashboard-label-studio.js"></script> </div>
</body> <script src="./js/dashboard-label-studio.js"></script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html> </html>

236
settings.html Normal file
View File

@@ -0,0 +1,236 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Settings - mb ai Trainer</title>
<link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" />
<style>
.modal {
display: none;
position: fixed;
z-index: 1000;
left: 0;
top: 0;
width: 100%;
height: 100%;
overflow: auto;
background-color: rgba(0,0,0,0.4);
}
.modal-content {
background-color: #fefefe;
margin: 5% auto;
padding: 20px;
border: 1px solid #888;
border-radius: 8px;
width: 600px;
max-width: 90%;
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
border-bottom: 2px solid #e0e0e0;
padding-bottom: 10px;
}
.modal-header h2 {
margin: 0;
color: #333;
}
.close {
color: #aaa;
font-size: 28px;
font-weight: bold;
cursor: pointer;
}
.close:hover,
.close:focus {
color: #000;
}
.settings-section {
margin-bottom: 25px;
}
.settings-section h3 {
margin-bottom: 10px;
color: #555;
font-size: 18px;
}
.setting-row {
margin-bottom: 15px;
}
.setting-row label {
display: block;
margin-bottom: 5px;
font-weight: 600;
color: #333;
}
.setting-row input {
width: 100%;
padding: 8px;
border: 1px solid #ddd;
border-radius: 4px;
box-sizing: border-box;
}
.setting-row small {
display: block;
margin-top: 4px;
color: #666;
font-size: 12px;
}
.button-row {
display: flex;
gap: 10px;
margin-top: 10px;
}
.test-button {
padding: 6px 12px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
}
.test-button:hover {
background-color: #45a049;
}
.test-button:disabled {
background-color: #cccccc;
cursor: not-allowed;
}
.save-button {
padding: 10px 20px;
background-color: #2196F3;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 16px;
}
.save-button:hover {
background-color: #0b7dda;
}
.status-message {
padding: 10px;
margin-top: 10px;
border-radius: 4px;
font-size: 14px;
display: none;
}
.status-success {
background-color: #d4edda;
color: #155724;
border: 1px solid #c3e6cb;
}
.status-error {
background-color: #f8d7da;
color: #721c24;
border: 1px solid #f5c6cb;
}
.loader-small {
border: 3px solid #f3f3f3;
border-top: 3px solid #3498db;
border-radius: 50%;
width: 16px;
height: 16px;
animation: spin 1s linear infinite;
display: inline-block;
margin-left: 8px;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
</style>
</head>
<body>
<!-- Settings Modal -->
<div id="settingsModal" class="modal">
<div class="modal-content">
<div class="modal-header">
<h2>⚙️ Global Settings</h2>
<span class="close">&times;</span>
</div>
<div class="settings-section">
<h3>Label Studio Connection</h3>
<div class="setting-row">
<label for="labelstudio_api_url">API URL:</label>
<input type="text" id="labelstudio_api_url" placeholder="http://192.168.1.19:8080/api" />
<small>Enter the base URL of your Label Studio API</small>
</div>
<div class="setting-row">
<label for="labelstudio_api_token">API Token:</label>
<input type="text" id="labelstudio_api_token" placeholder="Your API token" />
<small>Find your API token in Label Studio Account settings</small>
</div>
<div class="button-row">
<button class="test-button" id="testLabelStudioBtn">Test Connection</button>
</div>
<div id="labelstudioTestResult" class="status-message"></div>
</div>
<div class="settings-section">
<h3>YOLOX Installation</h3>
<div class="setting-row">
<label for="yolox_path">YOLOX Path:</label>
<input type="text" id="yolox_path" placeholder="C:/YOLOX" />
<small>Enter the path to your YOLOX installation directory</small>
</div>
<div class="setting-row">
<label for="yolox_venv_path">YOLOX Virtual Environment Path:</label>
<input type="text" id="yolox_venv_path" placeholder="e.g., /path/to/yolox_venv" />
<small>Path to YOLOX venv folder or activation script</small>
</div>
<div class="setting-row">
<label for="yolox_output_path">YOLOX Output Folder:</label>
<input type="text" id="yolox_output_path" placeholder="./backend" />
<small>Folder where experiment files (exp.py, exp_infer.py) and JSON files will be saved</small>
</div>
<div class="button-row">
<button class="test-button" id="testYoloxBtn">Verify Path</button>
</div>
<div id="yoloxTestResult" class="status-message"></div>
</div>
<div style="text-align: right; margin-top: 20px; padding-top: 20px; border-top: 1px solid #e0e0e0;">
<button class="save-button" id="saveSettingsBtn">Save Settings</button>
</div>
<div id="saveResult" class="status-message"></div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html>

283
setup-training-project.html
View File

@@ -1,109 +1,176 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label> <label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
Training Project</button> Training Project</button>
<button id="seed-db-btn" class="button"> <button id="seed-db-btn" class="button">
Seed Database Seed Database
<div class="loader" id="loader" style="display: none"></div> <div class="loader" id="loader" style="display: none"></div>
</button> </button>
</div> <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
</div>
</div>
<div id="details"> </div>
<script src="js/setup-training-project.js"></script> <div id="details">
</div> <script src="js/setup-training-project.js"></script>
<script> </div>
document.getElementById('seed-db-btn').addEventListener('click', function () { <script>
const elLoader = document.getElementById("loader") document.getElementById('seed-db-btn').addEventListener('click', function () {
elLoader.style.display = "inherit" const elLoader = document.getElementById("loader")
elLoader.style.display = "inherit"
fetch('/api/seed')
.finally(() => { fetch('/api/seed')
// Instead of hiding loader immediately, poll /api/update-status until done .finally(() => {
function pollStatus() { // Instead of hiding loader immediately, poll /api/update-status until done
fetch('/api/update-status') function pollStatus() {
.then(res => res.json()) fetch('/api/update-status')
.then(status => { .then(res => res.json())
if (status && status.running) { .then(status => {
// Still running, poll again after short delay if (status && status.running) {
setTimeout(pollStatus, 5000); // Still running, poll again after short delay
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
pollStatus(); }
}) pollStatus();
}); })
});
// Show loader if backend is still processing on page load
// Show loader if backend is still processing on page load
function pollStatus() {
const elLoader = document.getElementById("loader"); function pollStatus() {
fetch('/api/update-status') const elLoader = document.getElementById("loader");
.then(res => res.json()) fetch('/api/update-status')
.then(res => res.json())
.then(status => {
if (status && status.running) { .then(status => {
elLoader.style.display = "inherit"; if (status && status.running) {
setTimeout(pollStatus, 5000); elLoader.style.display = "inherit";
} else { setTimeout(pollStatus, 5000);
elLoader.style.display = "none"; } else {
} elLoader.style.display = "none";
}) }
.catch(() => { })
elLoader.style.display = "none"; .catch(() => {
}); elLoader.style.display = "none";
} });
}
</script>
<script> </script>
const urlParams = new URLSearchParams(window.location.search); <script>
const projectId = urlParams.get('id'); const urlParams = new URLSearchParams(window.location.search);
fetch('/api/training-projects') const projectId = urlParams.get('id');
.then(res => res.json()) fetch('/api/training-projects')
.then(projects => { .then(res => res.json())
const project = projects.find(p => p.project_id == projectId || p.id == projectId); .then(projects => {
if (project) { const project = projects.find(p => p.project_id == projectId || p.id == projectId);
document.getElementById('project-title-label').textContent = '/' + project.title; if (project) {
} document.getElementById('project-title-label').textContent = '/' + project.title;
}); }
</script> });
</script>
</div>
</body> </div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html> </html>

1442
style.css
View File

File diff suppressed because it is too large Load Diff

32
styleguide.css
View File

@@ -1,16 +1,16 @@
:root { :root {
--m3-body-small-font-family: "Roboto", Helvetica; --m3-body-small-font-family: "Roboto", Helvetica;
--m3-body-small-font-weight: 400; --m3-body-small-font-weight: 400;
--m3-body-small-font-size: 12px; --m3-body-small-font-size: 12px;
--m3-body-small-letter-spacing: 0.4000000059604645px; --m3-body-small-letter-spacing: 0.4000000059604645px;
--m3-body-small-line-height: 16px; --m3-body-small-line-height: 16px;
--m3-body-small-font-style: normal; --m3-body-small-font-style: normal;
--title2-regular-font-family: "SF Pro", Helvetica; --title2-regular-font-family: "SF Pro", Helvetica;
--title2-regular-font-weight: 400; --title2-regular-font-weight: 400;
--title2-regular-font-size: 22px; --title2-regular-font-size: 22px;
--title2-regular-letter-spacing: -0.25999999046325684px; --title2-regular-letter-spacing: -0.25999999046325684px;
--title2-regular-line-height: 28px; --title2-regular-line-height: 28px;
--title2-regular-font-style: normal; --title2-regular-font-style: normal;
--minus-for-button-size: 30px; --minus-for-button-size: 30px;
} }

78
text.css
View File

@@ -1,40 +1,40 @@
.popup .upload-button { .popup .upload-button {
width: 80px; width: 80px;
height: 25px; height: 25px;
background-color: #4cdb0085; background-color: #4cdb0085;
border-radius: 5px; border-radius: 5px;
display: flex; display: flex;
justify-content: center; justify-content: center;
align-items: center; align-items: center;
cursor: pointer; cursor: pointer;
position: absolute; position: absolute;
top: 303px; top: 303px;
left: 128px; left: 128px;
box-sizing: border-box; box-sizing: border-box;
} }
.popup .upload-button:hover{ .popup .upload-button:hover{
width: 80px; width: 80px;
height: 25px; height: 25px;
background-color: #36990085; background-color: #36990085;
border-radius: 5px; border-radius: 5px;
display: flex; display: flex;
justify-content: center; justify-content: center;
align-items: center; align-items: center;
cursor: pointer; cursor: pointer;
position: absolute; position: absolute;
top: 303px; top: 303px;
left: 128px; left: 128px;
box-sizing: border-box; box-sizing: border-box;
} }
.popup .upload-button-text { .popup .upload-button-text {
font-family: var(--m3-body-small-font-family); font-family: var(--m3-body-small-font-family);
font-weight: var(--m3-body-small-font-weight); font-weight: var(--m3-body-small-font-weight);
color: #000000; color: #000000;
font-size: var(--m3-body-small-font-size); font-size: var(--m3-body-small-font-size);
letter-spacing: var(--m3-body-small-letter-spacing); letter-spacing: var(--m3-body-small-letter-spacing);
line-height: var(--m3-body-small-line-height); line-height: var(--m3-body-small-line-height);
white-space: nowrap; white-space: nowrap;
font-style: var(--m3-body-small-font-style); font-style: var(--m3-body-small-font-style);
} }